Good evening,
I’ve upgraded a v3.6.6 setup to v4.0.1 and some tickets being
created via email are causing perl process to hang (or appear to
hang) with 99% CPU (& ticket doesn’t get created either). It
doesn’t happen with all messages and I’m stuck trying to work
out why some messages are causing it.
The upgrade is using a fresh rt4 directory. The RT_SiteConfig
file was re-created (carefully). I created a new database,
imported from old database, and ran the various db upgrade processes.
I’m using the following plugins:
Set(@MailPlugins,(qw(Auth::MailFrom Filter::TakeAction)));
Set(@Plugins,(qw(RT::Extension::CommandByMail)));
I get the following in the error_log:
[Mon Aug 01 18:38:45 2011] [error] [client 150.101.114.91]
FastCGI: comm with server “/usr/local/rt4/sbin/rt-server.fcgi”
aborted: idle timeout (300 sec)
[Mon Aug 01 18:38:45 2011] [error] [client 150.101.114.91]
FastCGI: incomplete headers (0 bytes) received from server “/usr/local/rt4/sbin/rt-server.fcgi”
[Mon Aug 01 18:38:45 2011] [error] [client 150.101.114.91]
FastCGI: server “/usr/local/rt4/sbin/rt-server.fcgi” stderr:
[Mon Aug 1 08:38:45 2011] [error]: Couldn’t find sender’s
address (/usr/local/rt4/sbin/…/lib/RT/Interface/Email/Auth/MailFrom.pm:74)
[Mon Aug 01 18:38:45 2011] [error] [client 150.101.114.91]
FastCGI: server “/usr/local/rt4/sbin/rt-server.fcgi” stderr:
[Mon Aug 1 08:38:45 2011] [error]: Could not record email:
Could not load a valid user (/usr/local/rt4/share/html/REST/1.0/NoAuth/mail-gateway:75)
I added some warn statements to GetCurrentUser sub in
MailFrom.pm, but warn statements don’t get logged. So I’m
apparently not understanding the code flow.
Can anyone suggest what I need to do to troubleshoot this further.
Thanks,
Charlie
Ꮚ Charlie Garrison ♊ <garrison@zeta.org.au>
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
〠 http://www.ietf.org/rfc/rfc1855.txt
Good evening,On 1/08/11 at 7:31 PM +1000, Charlie Garrison garrison@zeta.org.au wrote:
I’ve upgraded a v3.6.6 setup to v4.0.1 and some tickets being
created via email are causing perl process to hang (or appear
to hang) with 99% CPU (& ticket doesn’t get created either). It
doesn’t happen with all messages and I’m stuck trying to work
out why some messages are causing it.
After doing more testing it appears that multi-part messages (or
messages with attachements) are causing the RT processes to
hang. Can anyone think of a reason for that, or how I can
troubleshoot it?
Thanks,
Charlie
Ꮚ Charlie Garrison ♊ <garrison@zeta.org.au>
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
〠 http://www.ietf.org/rfc/rfc1855.txt
After doing more testing it appears that multi-part messages (or
messages with attachements) are causing the RT processes to hang. Can
anyone think of a reason for that, or how I can troubleshoot it?
Are there any other log messages? Can you capture a multipart message
that causes it consistently?
Thomas
Good morning,On 2/08/11 at 9:50 AM -0400, Thomas Sibley trs@bestpractical.com wrote:
On 08/02/2011 08:39 AM, Charlie Garrison wrote:
After doing more testing it appears that multi-part messages (or
messages with attachements) are causing the RT processes to hang. Can
anyone think of a reason for that, or how I can troubleshoot it?
Are there any other log messages? Can you capture a multipart message
that causes it consistently?
There are no other log messages. Here are the log messages
associated with the last test:
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91]
FastCGI: incomplete headers (0 bytes) received from server “/usr/local/rt4/sbin/rt-server.fcgi”
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91]
FastCGI: server “/usr/local/rt4/sbin/rt-server.fcgi” stderr:
[Tue Aug 2 14:31:44 2011] [error]: Couldn’t find sender’s
address (/usr/local/rt4/sbin/…/lib/RT/Interface/Email/Auth/MailFrom.pm:74)
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91]
FastCGI: server “/usr/local/rt4/sbin/rt-server.fcgi” stderr:
[Tue Aug 2 14:31:44 2011] [error]: Could not record email:
Could not load a valid user (/usr/local/rt4/share/html/REST/1.0/NoAuth/mail-gateway:75)
And I’ve attached a copy of the message I just sent to RT.
If I send that same message as plain-text it makes it through
and creates a ticket.
Thanks,
Charlie
Ꮚ Charlie Garrison ♊ <garrison@zeta.org.au>
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
〠 http://www.ietf.org/rfc/rfc1855.txt
Multi-part test.zip (1.05 KB)
Good morning,
After doing more testing it appears that multi-part messages (or
messages with attachements) are causing the RT processes to hang. Can
anyone think of a reason for that, or how I can troubleshoot it?
Are there any other log messages? Can you capture a multipart message
that causes it consistently?
There are no other log messages. Here are the log messages
associated with the last test:
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91]
FastCGI: incomplete headers (0 bytes) received from server “/usr/local/rt4/sbin/rt-server.fcgi”
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91]
FastCGI: server “/usr/local/rt4/sbin/rt-server.fcgi” stderr:
[Tue Aug 2 14:31:44 2011] [error]: Couldn’t find sender’s
address (/usr/local/rt4/sbin/…/lib/RT/Interface/Email/Auth/MailFrom.pm:74)
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91]
FastCGI: server “/usr/local/rt4/sbin/rt-server.fcgi” stderr:
[Tue Aug 2 14:31:44 2011] [error]: Could not record email:
Could not load a valid user (/usr/local/rt4/share/html/REST/1.0/NoAuth/mail-gateway:75)
And I’ve included (below) a copy of the message I just sent to
RT. (I tried sending it as an attachment but it bounced.)
If I send that same message as plain-text it makes it through
and creates a ticket.
Thanks,
Charlie
Ꮚ Charlie Garrison ♊ <garrison@zeta.org.au>
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
〠 http://www.ietf.org/rfc/rfc1855.txt
===== Copy of test message =====
Content-Type: multipart/alternative;
boundary=“Apple-Mail=_5359C68F-8205-464A-A857-DC42CEAF7D65”
X-Smtp-Server: mail.garrison.com.au
X-Universally-Unique-Identifier: 44cc0d4f-9163-4758-9839-6076e27ed1f6
Message-Id: 3A76CDFC-FD35-47F0-A5F5-F0F59921EBCA@zeta.org.au
Mime-Version: 1.0 (Apple Message framework v1244.3)
–Apple-Mail=_5359C68F-8205-464A-A857-DC42CEAF7D65
Content-Transfer-Encoding: 7bit
Content-Type: text/plain;
charset=us-ascii
This is a multi-part test of creating a ticket via email.
Charlie
–Apple-Mail=_5359C68F-8205-464A-A857-DC42CEAF7D65
Content-Transfer-Encoding: 7bit
Content-Type: text/html;
charset=us-ascii
This is a
multi-part test of creating a ticket via email.
Charlie
--Apple-Mail=_5359C68F-8205-464A-A857-DC42CEAF7D65--
There are no other log messages. Here are the log messages associated
with the last test:
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91] FastCGI:
incomplete headers (0 bytes) received from server
“/usr/local/rt4/sbin/rt-server.fcgi”
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91] FastCGI:
server “/usr/local/rt4/sbin/rt-server.fcgi” stderr: [Tue Aug 2 14:31:44
2011] [error]: Couldn’t find sender’s address
(/usr/local/rt4/sbin/…/lib/RT/Interface/Email/Auth/MailFrom.pm:74)
[Wed Aug 03 00:31:44 2011] [error] [client 150.101.114.91] FastCGI:
server “/usr/local/rt4/sbin/rt-server.fcgi” stderr: [Tue Aug 2 14:31:44
2011] [error]: Could not record email: Could not load a valid user
(/usr/local/rt4/share/html/REST/1.0/NoAuth/mail-gateway:75)
There’s nothing RT does after spitting out these errors except print
“not ok” and an error message for bin/rt-mailgate to pickup (the
mailgate then exits).
I’ll note that “Couldn’t find sender’s address” means that RT couldn’t
find a From: address, but your message contains one and stock RT finds
it. Either the log messages above aren’t for the test message you
attached, or your test message is getting mangled by something else
before it ever gets to RT.
Do you have anything interesting in your mail flow before RT? To
further debug, you could change your mailgate alias to
| tee /tmp/output | rt-mailgate ...
to capture the message as it is piped into mailgate.
And I’ve attached a copy of the message I just sent to RT.
The attached message doesn’t make RT blow up on a vanilla system, so
it’s specific to your install.
Thomas
Good evening,On 5/08/11 at 11:17 AM -0400, Thomas Sibley trs@bestpractical.com wrote:
And I’ve attached a copy of the message I just sent to RT.
The attached message doesn’t make RT blow up on a vanilla system, so
it’s specific to your install.
Thanks for looking at the problem. I’ve continued my
troubleshooting, and today I found the culprit. There was a
scrip which was spinning (while loop). I disabled that and
everything is working fine.
I didn’t investigate why that scrip wasn’t working since it’s
not needed any longer anyway. (I saw it contained a while loop
and considered it a suspect.)
Thanks again for your help.
Charlie
Ꮚ Charlie Garrison ♊ <garrison@zeta.org.au>
O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
〠 http://www.ietf.org/rfc/rfc1855.txt
Please keep messages on the list so people can find resolutions later.On 08/07/2011 05:55 AM, Charlie Garrison wrote:
Thanks for looking at the problem. I’ve continued my troubleshooting,
and today I found the culprit. There was a scrip which was spinning
(while loop). I disabled that and everything is working fine.
I didn’t investigate why that scrip wasn’t working since it’s not needed
any longer anyway. (I saw it contained a while loop and considered it a
suspect.)
I’m glad you found the problem, and that it wasn’t an issue with RT itself.
Thomas