Emails on ticket updates not being sent in 4.0.6

RT Users
21

I’ll point out, too, for more data … that before I did the
patching yesterday, I did a:

mkdir SAVE
cp -rp bin lib share local etc SAVE

Looking at the new lib/RT/Interface/Email.pm from the patch, and
looking at the diff IN the patch file itself, I saw nothing
requiring any new code or DB references, so tried dropping the
old “SAVE/lib/RT/Interface/Email.pm” file back in place. It bought
me nothing.

Did you restart apache after this? We see no reason why this shouldn’t
work, and it works locally for us. (And if you don’t use RT’s VERP
support, it won’t even leave you vulnerable.)

Can you try it again, making sure to restart apache, and verify if it
does or does not work?

Thomas

22

FWIW, we are seeing the same sendmail problem with 3.8.10 patched
with latest patch set.

We have replicated this problem when running 3.8.12, or 3.8.x + the
security patches, when running under mod_perl

WHEW. Good to know.

I don’t know why, but I can’t get my $RT::Logger->crit(“…”)
calls to spit out anything from inside Email.pm

Yes, I’m desperate here :slight_smile:

23

FWIW, we are seeing the same sendmail problem with 3.8.10 patched
with latest patch set.

We have replicated this problem when running 3.8.12, or 3.8.x + the
security patches, when running under mod_perl

I also had this problem when I upgraded from 3.8.9 to 3.8.12 yesterday.
I had to rollback the upgrade in order to get operational again.

It sounds like Best Practical has this under control now, but I’d be happy
to send any additional data to help.

Thanks,
Chris

24

I’ll point out, too, for more data … that before I did the
patching yesterday, I did a:

 mkdir SAVE
 cp -rp bin lib share local etc SAVE

Looking at the new lib/RT/Interface/Email.pm from the patch, and
looking at the diff IN the patch file itself, I saw nothing
requiring any new code or DB references, so tried dropping the
old “SAVE/lib/RT/Interface/Email.pm” file back in place. It bought
me nothing.

Did you restart apache after this? We see no reason why this shouldn’t
work, and it works locally for us. (And if you don’t use RT’s VERP
support, it won’t even leave you vulnerable.)

Can you try it again, making sure to restart apache, and verify if it
does or does not work?

I did restart Apache, however I remembering now that we have
quite a bit of the correspond scrips disabled (but no, this
has not just been a “Problem Exists Between Chair and Keyboard”
situation).

Tickling some RT email via a ticket update (owner change) does
in fact send email properly with the old Email.pm in place.

25

I also had this problem when I upgraded from 3.8.9 to 3.8.12 yesterday.
I had to rollback the upgrade in order to get operational again.

It’s great to hear about these sorts of things when they happen. If
there’s a next time, don’t hesitate to make noise if an upgrade causes
fundamental failures like this.

It sounds like Best Practical has this under control now, but I’d be happy
to send any additional data to help.

Any additional data may help. If you can send along a saved copy of
your System Configuration page from RT, that would be great. If you
still have example errors from the error log, those would also be useful.

Thomas

26

So it looks like the reason is that sendmail (exim) isn’t receiving a
recipient address:

Bounce emails to the rt user on each event contain: “A message that
you sent contained no recipient addresses, and therefore no delivery
could be attempted.”

However, if I change $MailCommand to ‘sendmail’ instead of
‘sendmailpipe’ (and remove the -t arg from $SendmailArguments) email
updates work again.

Not sure what the difference between these two methods is but
Set($MailCommand , ‘sendmail’); has fixed the issue for me.

That’s mighty strange. Can you send us both your non-working and
working RT 4.0.6 RT_SiteConfig.pm? Please be sure to redact any
passwords contained in them.

Working config mailed to your address. Diff between working and non-working is

diff RT_SiteConfig.pm RT_SiteConfig.pm.broken

23c23
< Set($MailCommand , ‘sendmail’);

Set($MailCommand , ‘sendmailpipe’);
25c25
< Set($SendmailArguments , “-oi”);

27

It’s great to hear about these sorts of things when they happen. If
there’s a next time, don’t hesitate to make noise if an upgrade causes
fundamental failures like this.

I apologize for the delay; I fully intended to send this yesterday but time
did not permit.

Any additional data may help. If you can send along a saved copy of
your System Configuration page from RT, that would be great. If you
still have example errors from the error log, those would also be useful.

If it helps, this is a Debian Squeeze system.

The RT error was:

[Tue May 22 19:36:46 2012] [info]: rt-3.8.12-24249-1337715406-1496.67623-81-0@rt.snl.salk.edu #67623/656937 - Scrip 81 Autoreply (/opt/rt3/bin/…/lib/RT/Action/SendEmail.pm:302)
[Tue May 22 19:36:46 2012] [crit]: rt-3.8.12-24249-1337715406-1496.67623-81-0@rt.snl.salk.edu: Could not send mail with command /usr/sbin/sendmail -oi -t: program unexpectedly closed pipe at /opt/rt3/bin/…/lib/RT/Interface/Email.pm line 451, line 1.

Stack:
[/opt/rt3/bin/…/lib/RT/Interface/Email.pm:451]
[/usr/lib/perl/5.10/IO/Handle.pm:148]
[/usr/share/perl5/MIME/Decoder/NBit.pm:151]
[/usr/share/perl5/MIME/Decoder.pm:258]
[/usr/share/perl5/MIME/Entity.pm:1883]
[/usr/share/perl5/MIME/Entity.pm:1853]
[/usr/share/perl5/MIME/Entity.pm:1773]
[/opt/rt3/bin/…/lib/RT/Interface/Email.pm:458]
[/opt/rt3/bin/…/lib/RT/Action/SendEmail.pm:309]
[/opt/rt3/bin/…/lib/RT/Action/SendEmail.pm:129]
[/opt/rt3/bin/…/lib/RT/ScripAction_Overlay.pm:238]
[/opt/rt3/bin/…/lib/RT/Scrip_Overlay.pm:477]
[/opt/rt3/bin/…/lib/RT/Scrips_Overlay.pm:205]
[/opt/rt3/bin/…/lib/RT/Transaction_Overlay.pm:188]
[/opt/rt3/bin/…/lib/RT/Record.pm:1457]
[/opt/rt3/bin/…/lib/RT/Ticket_Overlay.pm:660]
[/opt/rt3/bin/…/lib/RT/Interface/Email.pm:1472]
[/opt/rt3/share/html/REST/1.0/NoAuth/mail-gateway:61] (/opt/rt3/bin/…/lib/RT/Interface/Email.pm:472)

The exim error was:

2012-05-22 12:33:06 1SWupO-0006Sc-LX Message abandoned: Input read error (Bad file descriptor) while receiving message from www-data

Our sendmail config is set in RT_Config.pm (not set in SiteConfig):

Set($MailCommand , ‘sendmailpipe’);
…
Set($SendmailArguments , “-oi -t”);

Attached is the full, working, 3.8.9 configuration. I did not save the configuration from 3.8.12, but I changed almost nothing (save for installing the latest fcgi from CPAN) during the upgrade.

System Configuration.html (95.1 KB)

28

I believe that this affected me too. I performed the upgrade from RT 4.0.5 to 4.0.6 yesterday and email stopped.

The workaround I found was overriding the mail command in my RT_SiteConfig.pm, as mentioned in this thread, but from the default (‘sendmailpipe’):

Set( $MailCommand, ‘sendmail’ );

My environment is FreeBSD 9.0-STABLE, stock FreeBSD Sendmail 8.14.5/8.14.5, Apache 2.2.22, Perl v5.12.4, mod_perl 2-2.0.6.

Here are additional symptoms I observed.

In my Request Tracker log:

[Fri Jun 1 18:35:47 2012] [info]: <rt-4.0.6-26519-1338575747-802.33892-4-0@son.rochester.eduhttp://rochester.edu> #33892/384246 - Scrip 4 Notify AdminCcs On Create (/usr/local/lib/perl5/site_perl/5.12.4/RT/Action/SendEmail.pm:301)
[Fri Jun 1 18:35:47 2012] [critical]: <rt-4.0.6-26519-1338575747-802.33892-4-0@son.rochester.eduhttp://son.rochester.edu>: Could not send mail with command /usr/sbin/sendmail -oi -t: program unexpectedly closed pipe at /usr/local/lib/perl5/site_perl/5.12.4/RT/Interface/Email.pm line 449.

Stack:
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Interface/Email.pm:449]
[/usr/local/lib/perl5/5.12.4/mach/IO/Handle.pm:406]
[/usr/local/lib/perl5/site_perl/5.12.4/MIME/Head.pm:535]
[/usr/local/lib/perl5/site_perl/5.12.4/Mail/Internet.pm:112]
[/usr/local/lib/perl5/site_perl/5.12.4/MIME/Entity.pm:1770]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Interface/Email.pm:456]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Action/SendEmail.pm:308]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Action/SendEmail.pm:128]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/ScripAction.pm:232]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Scrip.pm:475]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Scrips.pm:198]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Transaction.pm:201]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Record.pm:1459]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Ticket.pm:676]
[/usr/local/lib/perl5/site_perl/5.12.4/RT/Interface/Email.pm:1522]
[/usr/local/share/rt40/html/REST/1.0/NoAuth/mail-gateway:61] (/usr/local/lib/perl5/site_perl/5.12.4/RT/Interface/Email.pm:470)

However, failures were inconsistent – some had false positives:

[Fri Jun 1 17:11:54 2012] [info]: <rt-4.0.6-26923-1338570714-238.33891-4-0@son.rochester.eduhttp://rochester.edu> #33891/384223 - Scrip 4 Notify AdminCcs On Create (/usr/local/lib/perl5/site_perl/5.12.4/RT/Action/SendEmail.pm:301)
[Fri Jun 1 17:11:54 2012] [info]: <rt-4.0.6-26923-1338570714-238.33891-4-0@son.rochester.eduhttp://rochester.edu> sent Bcc: Michael_Fisher@URMC.Rochester.edumailto:Michael_Fisher@URMC.Rochester.edu (/usr/local/lib/perl5/site_perl/5.12.4/RT/Action/SendEmail.pm:332)

In my mail log, I was seeing entries like this (note that the sender/recipient email addresses are missing and defaulted to the web server process user):

Jun 1 13:11:54 rt sendmail[30898]: q51HBsZ8030898: from=www, size=0, class=0, nrcpts=0, msgid=<201206011711.q51HBsZ8030898@son.rochester.edumailto:201206011711.q51HBsZ8030898@son.rochester.edu>, relay=www@localhost
Jun 1 13:11:54 rt sendmail[30899]: q51HBsnA030899: from=www, size=0, class=0, nrcpts=0, msgid=<201206011711.q51HBsnA030899@son.rochester.edumailto:201206011711.q51HBsnA030899@son.rochester.edu>, relay=www@localhost
Jun 1 14:35:47 rt sendmail[34095]: q51IZlaf034095: from=www, size=0, class=0, nrcpts=0, msgid=<201206011835.q51IZlaf034095@son.rochester.edumailto:201206011835.q51IZlaf034095@son.rochester.edu>, relay=www@localhost

After switching to ‘sendmail’ mode, I can tell no client-side difference between having Set($SendmailArguments, ‘-oi’ ); and Set( $SendmailArguments, ‘-oi -t’ ); I am using the former, per Thomas Sibley’s recommendation of 5/23.

Thanks for making and sharing Request Tracker. Please let me know if there’s anything else that I can do to help.

Mike Fisher Michael_Fisher@urmc.rochester.edumailto:Michael_Fisher@urmc.rochester.edu 585-275-6884
University of Rochester School of NursingOn May 23, 2012, at 7:01 , Niall Wilson wrote:

Hi,

I’ve just updated to 4.0.6 and ticket updates are no longer being emailed. This does not seem to be related to the noted FCGI.pm problem as (1) I’m using mod_perl and (2) I updated the installed FCGI to v 0.74 just in case.

If I revert to 4.0.5 email works fine.

Sample debug output is :

[Wed May 23 10:52:07 2012] [debug]: Working on mailfield To; recipients are (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:660)
[Wed May 23 10:52:07 2012] [debug]: Subject: [ICHEC Systems Team #416] [Comment] ICHEC email address: forward, auto-reply and mailing-list unsubscription
From: “Niall Wilson via ICHEC RT” <support-systems-comment@ichec.iemailto:support-systems-comment@ichec.ie>
Reply-To: support-systems-comment@ichec.iemailto:support-systems-comment@ichec.ie
In-Reply-To: <rt-4.0.6-14259-1337765913-158.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337765913-158.416-8-0@ichec.ie>
References: <RT-Ticket-416@ichec.iemailto:RT-Ticket-416@ichec.ie> <rt-4.0.6-14259-1337765913-158.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337765913-158.416-8-0@ichec.ie>
Message-ID: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie>
Precedence: bulk
X-RT-Loop-Prevention: ICHEC Support
RT-Ticket: ICHEC Support #416
Managed-by: RT 4.0.6 (Request Tracker — Best Practical Solutions)
RT-Originator: niall.wilson@ichec.iemailto:niall.wilson@ichec.ie
Bcc: niall.wilson@ichec.iemailto:niall.wilson@ichec.ie
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: utf-8 (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:667)
[Wed May 23 10:52:07 2012] [debug]: Removing deferred recipients from To: line (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:690)
[Wed May 23 10:52:07 2012] [debug]: Setting deferred recipients for attribute creation (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:699)
[Wed May 23 10:52:07 2012] [debug]: Working on mailfield Cc; recipients are (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:660)
[Wed May 23 10:52:07 2012] [debug]: Subject: [ICHEC Systems Team #416] [Comment] ICHEC email address: forward, auto-reply and mailing-list unsubscription
From: “Niall Wilson via ICHEC RT” <support-systems-comment@ichec.iemailto:support-systems-comment@ichec.ie>
Reply-To: support-systems-comment@ichec.iemailto:support-systems-comment@ichec.ie
In-Reply-To: <rt-4.0.6-14259-1337765913-158.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337765913-158.416-8-0@ichec.ie>
References: <RT-Ticket-416@ichec.iemailto:RT-Ticket-416@ichec.ie> <rt-4.0.6-14259-1337765913-158.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337765913-158.416-8-0@ichec.ie>
Message-ID: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie>
Precedence: bulk
X-RT-Loop-Prevention: ICHEC Support
RT-Ticket: ICHEC Support #416
Managed-by: RT 4.0.6 (Request Tracker — Best Practical Solutions)
RT-Originator: niall.wilson@ichec.iemailto:niall.wilson@ichec.ie
Bcc: niall.wilson@ichec.iemailto:niall.wilson@ichec.ie
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: utf-8 (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:667)
[Wed May 23 10:52:07 2012] [debug]: Removing deferred recipients from Cc: line (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:690)
[Wed May 23 10:52:07 2012] [debug]: Setting deferred recipients for attribute creation (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:699)
[Wed May 23 10:52:07 2012] [debug]: Working on mailfield Bcc; recipients are niall.wilson@ichec.iemailto:niall.wilson@ichec.ie (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:660)
[Wed May 23 10:52:07 2012] [debug]: Subject: [ICHEC Systems Team #416] [Comment] ICHEC email address: forward, auto-reply and mailing-list unsubscription
From: “Niall Wilson via ICHEC RT” <support-systems-comment@ichec.iemailto:support-systems-comment@ichec.ie>
Reply-To: support-systems-comment@ichec.iemailto:support-systems-comment@ichec.ie
In-Reply-To: <rt-4.0.6-14259-1337765913-158.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337765913-158.416-8-0@ichec.ie>
References: <RT-Ticket-416@ichec.iemailto:RT-Ticket-416@ichec.ie> <rt-4.0.6-14259-1337765913-158.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337765913-158.416-8-0@ichec.ie>
Message-ID: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie>
Precedence: bulk
X-RT-Loop-Prevention: ICHEC Support
RT-Ticket: ICHEC Support #416
Managed-by: RT 4.0.6 (Request Tracker — Best Practical Solutions)
RT-Originator: niall.wilson@ichec.iemailto:niall.wilson@ichec.ie
Bcc: niall.wilson@ichec.iemailto:niall.wilson@ichec.ie
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: utf-8 (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:667)
[Wed May 23 10:52:07 2012] [debug]: Got user mail preference ‘Individual messages’ for user niall.wilson@ichec.iemailto:niall.wilson@ichec.ie (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:681)
[Wed May 23 10:52:07 2012] [debug]: Removing deferred recipients from Bcc: line (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:690)
[Wed May 23 10:52:07 2012] [debug]: Setting deferred recipients for attribute creation (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:699)
[Wed May 23 10:52:07 2012] [debug]: No recipients found for deferred delivery on transaction #8091 (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:712)
[Wed May 23 10:52:07 2012] [info]: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie> #416/8091 - Scrip 8 On Comment Notify AdminCcs as Comment (/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:301)
[Wed May 23 10:52:07 2012] [error]: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie>: /usr/sbin/sendmail -oi -t exited with code 1 (/opt/rt4/sbin/…/lib/RT/Interface/Email.pm:465)
[Wed May 23 10:52:07 2012] [crit]: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie>: Could not send mail with command /usr/sbin/sendmail -oi -t: <rt-4.0.6-14259-1337770327-43.416-8-0@ichec.iemailto:rt-4.0.6-14259-1337770327-43.416-8-0@ichec.ie>: /usr/sbin/sendmail -oi -t exited with code 1 at /opt/rt4/sbin/…/lib/RT/Interface/Email.pm line 466.

Stack:
[/opt/rt4/sbin/…/lib/RT/Interface/Email.pm:466]
[/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:308]
[/opt/rt4/sbin/…/lib/RT/Action/SendEmail.pm:128]
[/opt/rt4/sbin/…/lib/RT/ScripAction.pm:232]
[/opt/rt4/sbin/…/lib/RT/Scrip.pm:475]
[/opt/rt4/sbin/…/lib/RT/Scrips.pm:198]
[/opt/rt4/sbin/…/lib/RT/Transaction.pm:201]
[/opt/rt4/sbin/…/lib/RT/Record.pm:1459]
[/opt/rt4/sbin/…/lib/RT/Ticket.pm:2243]
[/opt/rt4/sbin/…/lib/RT/Ticket.pm:2103]
[/opt/rt4/sbin/…/lib/RT/Interface/Web.pm:1856]
[/opt/rt4/share/html/Ticket/Display.html:164]
[/opt/rt4/share/html/Ticket/Update.html:328]
[/opt/rt4/share/html/Ticket/autohandler:19]
[/opt/rt4/sbin/…/lib/RT/Interface/Web.pm:568]
[/opt/rt4/sbin/…/lib/RT/Interface/Web.pm:318]
[/opt/rt4/share/html/autohandler:53] (/opt/rt4/sbin/…/lib/RT/Interface/Email.pm:470)

Niall Wilson
ICHEC Systems Team

29

I believe that this affected me too. I performed the upgrade from RT
4.0.5 to 4.0.6 yesterday and email stopped.

The workaround I found was overriding the mail command in my
RT_SiteConfig.pm, as mentioned in this thread, but from the default
(‘sendmailpipe’):

Set( $MailCommand, 'sendmail' );

[snip]

My environment is FreeBSD 9.0-STABLE, stock FreeBSD Sendmail
8.14.5/8.14.5, Apache 2.2.22, Perl v5.12.4, mod_perl 2-2.0.6.

This means you’re running RT 4 under mod_perl, but under SetHandler
perl-script instead of the recommended SetHandler modperl (see
docs/web_deployment.pod).

I suggest switching back to sendmailpipe for your $MailCommand and
switching to SetHandler modperl, as SetHandler perl-script is
unsupported under RT 4.

Thomas

30

I believe that this affected me too. I performed the upgrade from RT
4.0.5 to 4.0.6 yesterday and email stopped.

The workaround I found was overriding the mail command in my
RT_SiteConfig.pm, as mentioned in this thread, but from the default
(‘sendmailpipe’):

Set( $MailCommand, ‘sendmail’ );

[snip]
My environment is FreeBSD 9.0-STABLE, stock FreeBSD Sendmail
8.14.5/8.14.5, Apache 2.2.22, Perl v5.12.4, mod_perl 2-2.0.6.

This means you’re running RT 4 under mod_perl, but under SetHandler
perl-script instead of the recommended SetHandler modperl (see
docs/web_deployment.pod).

I suggest switching back to sendmailpipe for your $MailCommand and
switching to SetHandler modperl, as SetHandler perl-script is
unsupported under RT 4.

Thomas,

Thank you very much – this configuration change (and reversion of MailCommand) solved the problem. Great catch!

Mike Fisher Michael_Fisher@urmc.rochester.edumailto:Michael_Fisher@urmc.rochester.edu 585-275-6884
University of Rochester School of Nursing