Is there any disadvantage of "Precedence: bulk" in RT emails header

Hi,

Recently we got into problem where spamcop blacklisted our RT mail server IP
(rt using another host for relay emails) then we start using the same host
MTA (postfix) where RT is installed.

Now yahoo is Rate limiting emails from our RT which causing delay in emails.

Now boss is asking to remove/patch RT to do not add “Precedence: bulk” to
header of emails, i wonders does it will do any good? coz as far i can
understand “precendence: bulk” is nice thing

"that some autoreply software uses that header to avoid sending
vacation/out-of-office messages to automated mailers.

Thankfully those some autorepliers should also be smart enough to not
send more than one such message to any specific address, which would
avoid nasty mail loops."

So what you guys thinks about removing the “Precedence: bulk” from RT email
header if yes then how to remove it?

Thanks.
Askar

Asrai khn wrote:

Recently we got into problem where spamcop blacklisted our RT mail server IP
(rt using another host for relay emails) then we start using the same host
MTA (postfix) where RT is installed.

Now yahoo is Rate limiting emails from our RT which causing delay in emails.

That won’t be because you’re adding a “Precedence: bulk” header.
That’ll be because you were auto-replying to too much spam.

We tend to be fairly aggressive on our spam checking when it’s going
into RT, with procmail dumping the mail into a separate mail folder if
it even suspects that the mail is spam.

We also send our RT mail out through our main mail relays which means
that if we do auto-reply to spam then the spam/ham ratio is much lower,
since our staff and students are sending legitimate mail out to yahoo as
well.

Mark
Mark Chappell
Unix Systems Administrator

We tend to be fairly aggressive on our spam checking when it’s going
into RT, with procmail dumping the mail into a separate mail folder if
it even suspects that the mail is spam.

Hi Mark,

How you achieve this ? imean would you care to share the process/script of
dumping spam to separate email folder?

We also send our RT mail out through our main mail relays which means
that if we do auto-reply to spam then the spam/ham ratio is much lower,
since our staff and students are sending legitimate mail out to yahoo as
well.

Most of our emails is business related which mean via RT, and staff do not
sent email frm there email addresses direct to yahoo but only via RT.

Thanks. Askar

Asrai khn wrote:

We tend to be fairly aggressive on our spam checking when it’s going
into RT, with procmail dumping the mail into a separate mail folder if
it even suspects that the mail is spam.

How you achieve this ? imean would you care to share the process/script of
dumping spam to separate email folder?

Since you’re using Postfix which I don’t really know I’ll do most of
this as an outline, and our procmail script is fairly custom do I’ll
only quote chunks of the procmailrc.

The first line of defense are our front-line mail relays. These reject
on various DNS rbls and add headers for several others. The lists we
use include SpamHaus’ full zen list, SpamCop, MAPS and SORBS. They also
run a copy of spam assassin WITHOUT any of the baysaen stuff turned on,
a high enough spam score causes an out right rejection at the end of the
SMTP DATA time, and leaves a header in the message for lower scores. We
also reject on Virus signatures using the, and have some extra
signatures to pick up certain types of phishing scams
(http://www.sanesecurity.co.uk/). Any rejection at this stage is done
before we’ve actually accepted the mail, so backscatter (which I would
guess is what caught you out) is seen to come from the server sending
you the spam NOT from your servers.

This is how we treat almost all incoming mail, rejecting approximately
90% of mail before it’s even through the front door (with surprisingly
few complaints about mail not getting through given the number of users
we have from “spam friendly” countries).

The second line of defense is procmail. The mail server on our RT
servers don’t deliver directly to RT they deliver to the procmail
command, which eventually delivers to RT. Procmail runs a batch of
extra checks, including; a baysaen filter using “bogofilter”,
“renattach” which catches certain types of attachments, “nodupmail”
which catches certain types of mail loops, “double clicks” and general
duplicated mail, and some custom stuff so that certain queues only see
mail sent via a specific web form. If bogofilter or the earlier lower
spamassassin threshold are triggered then we drop the mail into a “SPAM”
mail file. We have a couple of perl scripts which allow us to identify
“privileged” users out of RT and lets them bypass most of the procmail
filtering.

The privileged check is basically a perl wrapper around some simple RT
code with a little bit of caching built in.

my $user = RT::User->new($RT::SystemUser);

$user->Load($username);

unless ($user->Id) {
exit(1);
}

foreach my $key qw(Name EmailAddress RealName Privileged Disabled) {
my $value = $user->$key;
print($key.“: “.$value.”\n”);
}

Getting procmail to run external commands is straight forward

For example here’s some of our bogofilter and spam assassin stuff

BOGOFILTER=/usr/bin/bogofilter
BOGOFILTER_SPAM_ARGS=“-c /etc/bogofilter-SPAM.cf”
BOGOFILTER_MD_ARGS=“-c /etc/bogofilter-MD.cf”
SPAM_ASSASIN_RESULT=“formail -xX-Spam-Flag: | grep -i yes | awk '{print $2}'

SPAM - bogofilter.

:0fw
| $BOGOFILTER -e -p $BOGOFILTER_SPAM_ARGS

on failure return mail to the queue, the MTA will try delivery later,

75 is the value for EX_TEMPFAIL in /usr/include/sysexits.h

:0e
{ EXITCODE=75 HOST }

record the result.

BOGOFILTER_RESULT=“formail -xX-Bogosity: | grep -i yes

a virus can bombard us with mta bounce messages because we were used in

a forged “From” address, pick these out as they’re valid messages despite

being caused by a virus.

:0fw
| $BOGOFILTER -e -p $BOGOFILTER_MD_ARGS
:0e
{ EXITCODE=75 HOST }
MAILER_DAEMON=“formail -xX-MD-Bogosity: | grep -i yes

SPAM=“$BLACKLISTED$SPAM_ASSASIN_RESULT$BOGOFILTER_RESULT”

:0

  • ? test -n “$SPAM”
    SPAM

Those final 3 lines are what tells procmail to deliver, literally into a
file named “SPAM” if there’s any content in the $SPAM variable

Mark
Mark Chappell
Unix Systems Administrator

We use DSPAM and its Quarantine instead of the bogofilter+SPAM folder
here, but it is basically the same idea and it works very well indeed.

KenOn Fri, Jan 04, 2008 at 03:33:14PM +0000, Mark Chappell wrote:

Asrai khn wrote:

We tend to be fairly aggressive on our spam checking when it’s going
into RT, with procmail dumping the mail into a separate mail folder if
it even suspects that the mail is spam.

How you achieve this ? imean would you care to share the process/script of
dumping spam to separate email folder?

Since you’re using Postfix which I don’t really know I’ll do most of this
as an outline, and our procmail script is fairly custom do I’ll only quote
chunks of the procmailrc.

The first line of defense are our front-line mail relays. These reject on
various DNS rbls and add headers for several others. The lists we use
include SpamHaus’ full zen list, SpamCop, MAPS and SORBS. They also run a
copy of spam assassin WITHOUT any of the baysaen stuff turned on, a high
enough spam score causes an out right rejection at the end of the SMTP DATA
time, and leaves a header in the message for lower scores. We also reject
on Virus signatures using the, and have some extra signatures to pick up
certain types of phishing scams (http://www.sanesecurity.co.uk/). Any
rejection at this stage is done before we’ve actually accepted the mail, so
backscatter (which I would guess is what caught you out) is seen to come
from the server sending you the spam NOT from your servers.

This is how we treat almost all incoming mail, rejecting approximately 90%
of mail before it’s even through the front door (with surprisingly few
complaints about mail not getting through given the number of users we have
from “spam friendly” countries).

The second line of defense is procmail. The mail server on our RT servers
don’t deliver directly to RT they deliver to the procmail command, which
eventually delivers to RT. Procmail runs a batch of extra checks,
including; a baysaen filter using “bogofilter”, “renattach” which catches
certain types of attachments, “nodupmail” which catches certain types of
mail loops, “double clicks” and general duplicated mail, and some custom
stuff so that certain queues only see mail sent via a specific web form.
If bogofilter or the earlier lower spamassassin threshold are triggered
then we drop the mail into a “SPAM” mail file. We have a couple of perl
scripts which allow us to identify “privileged” users out of RT and lets
them bypass most of the procmail filtering.

The privileged check is basically a perl wrapper around some simple RT code
with a little bit of caching built in.

====
my $user = RT::User->new($RT::SystemUser);

$user->Load($username);

unless ($user->Id) {
exit(1);
}

foreach my $key qw(Name EmailAddress RealName Privileged Disabled) {
my $value = $user->$key;
print($key.“: “.$value.”\n”);
}

Getting procmail to run external commands is straight forward

For example here’s some of our bogofilter and spam assassin stuff

====
BOGOFILTER=/usr/bin/bogofilter
BOGOFILTER_SPAM_ARGS=“-c /etc/bogofilter-SPAM.cf”
BOGOFILTER_MD_ARGS=“-c /etc/bogofilter-MD.cf”
SPAM_ASSASIN_RESULT=“formail -xX-Spam-Flag: | grep -i yes | awk '{print $2}'

SPAM - bogofilter.

:0fw
| $BOGOFILTER -e -p $BOGOFILTER_SPAM_ARGS

on failure return mail to the queue, the MTA will try delivery later,

75 is the value for EX_TEMPFAIL in /usr/include/sysexits.h

:0e
{ EXITCODE=75 HOST }

record the result.

BOGOFILTER_RESULT=“formail -xX-Bogosity: | grep -i yes

a virus can bombard us with mta bounce messages because we were used in

a forged “From” address, pick these out as they’re valid messages despite

being caused by a virus.

:0fw
| $BOGOFILTER -e -p $BOGOFILTER_MD_ARGS
:0e
{ EXITCODE=75 HOST }
MAILER_DAEMON=“formail -xX-MD-Bogosity: | grep -i yes

SPAM=“$BLACKLISTED$SPAM_ASSASIN_RESULT$BOGOFILTER_RESULT”

:0

  • ? test -n “$SPAM”
    SPAM
    ====

Those final 3 lines are what tells procmail to deliver, literally into a
file named “SPAM” if there’s any content in the $SPAM variable

Mark

Mark Chappell
Unix Systems Administrator


The rt-users Archives

SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:

If you sign up for a new RT support contract before December 31, we’ll take
up to 20 percent off the price. This sale won’t last long, so get in touch
today. Email us at sales@bestpractical.com or call us at +1 617 812
0745.

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media. Buy a
copy at http://rtbook.bestpractical.com

Since you’re using Postfix which I don’t really know I’ll do most of
this as an outline, and our procmail script is fairly custom do I’ll
only quote chunks of the procmailrc.

The first line of defense are our front-line mail relays. These reject
on various DNS rbls and add headers for several others. The lists we
use include SpamHaus’ full zen list, SpamCop, MAPS and SORBS. They also
run a copy of spam assassin WITHOUT any of the baysaen stuff turned on,
a high enough spam score causes an out right rejection at the end of the
SMTP DATA time, and leaves a header in the message for lower scores. We
also reject on Virus signatures using the, and have some extra
signatures to pick up certain types of phishing scams
(http://www.sanesecurity.co.uk/). Any rejection at this stage is done
before we’ve actually accepted the mail, so backscatter (which I would
guess is what caught you out) is seen to come from the server sending
you the spam NOT from your servers.

(…)

Hi Mark,

thanks for the detail reply, its very much appreciated :slight_smile:

thanks and regards, askar

So what you guys thinks about removing the “Precedence: bulk” from RT
email header if yes then how to remove it?

Still I am interested how to stop rt3 adding the “Precedence:bulk” to
headers of an emails?

Anyone could could guide me?

Thanks. Askar

Asrai;
If you really want to remove it then have have a look in
lib/RT/SendEmail.pm , search for
$self->SetHeader( ‘Precedence’, “bulk” )

Regards;
Roy

Asrai khn wrote:> On Jan 4, 2008 12:48 AM, Asrai khn asraikhn@gmail.com wrote:

So what you guys thinks about removing the “Precedence: bulk” from RT
email header if yes then how to remove it?

Still I am interested how to stop rt3 adding the “Precedence:bulk” to
headers of an emails?

Anyone could could guide me?

Thanks. Askar



The rt-users Archives

SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:

If you sign up for a new RT support contract before December 31, we’ll take
up to 20 percent off the price. This sale won’t last long, so get in touch today.
Email us at sales@bestpractical.com or call us at +1 617 812 0745.

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Asrai;
If you really want to remove it then have have a look in
lib/RT/SendEmail.pm , search for
$self->SetHeader( ‘Precedence’, “bulk” )

Hi Roy,

Thanks for the reply, would you pls tell me what commenting the line "
$self->SetHeader( ‘Precedence’, “bulk” )" will do the the trick of removing
the Precdence: bulk header from rt mails?

Askar.

If you really want to remove it then have have a look in
lib/RT/SendEmail.pm , search for
$self->SetHeader( ‘Precedence’, “bulk” )

Roy,

As we have installed RT3.6 using ‘yum’ on fedora 7 the file path is little
different ie

/usr/lib/perl5/vendor_perl/5.8.8/RT/Action/SendEmail.pm

and after commenting …

$self->SetHeader( ‘Precedence’, “bulk” )

unless ( $self->TemplateObj->MIMEObj->head->get(“Precedence”) );

rt not adding ‘Precedence: bulk’ to the header.

Hope it will not break anything else :slight_smile:

Thanks. Askar

Good luck;
Be warned that the Precedence: bulk was there for a good reason ,
typically to identify mailing lists and maybe useful in detecting loops…
Sorry not sure of the details.
Roy
Asrai khn wrote:> On Jan 8, 2008 4:48 PM, Roy El-Hames rfh@pipex.net wrote:

If you really want to remove it then have have a look in
lib/RT/SendEmail.pm , search for
$self->SetHeader( ‘Precedence’, “bulk” )

Roy,

As we have installed RT3.6 using ‘yum’ on fedora 7 the file path is little
different ie

/usr/lib/perl5/vendor_perl/5.8.8/RT/Action/SendEmail.pm

and after commenting …

$self->SetHeader( ‘Precedence’, “bulk” )

unless ( $self->TemplateObj->MIMEObj->head->get(“Precedence”) );

rt not adding ‘Precedence: bulk’ to the header.

Hope it will not break anything else :slight_smile:

Thanks. Askar



The rt-users Archives

SAVE THOUSANDS OF DOLLARS ON RT SUPPORT:

If you sign up for a new RT support contract before December 31, we’ll take
up to 20 percent off the price. This sale won’t last long, so get in touch today.
Email us at sales@bestpractical.com or call us at +1 617 812 0745.

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com