RT3: mail UTF-8 automatic conversion and GPG verification

Hi.

I’m (re)writing a GPG email authentication plug-in based on Jesse’s
enhanced-mailgate.

For now, this plug-in should only handle RFC3156 compliant messages (see “5.
OpenPGP signed messages”), that is MIME “multipart/signed” messages composed
of exactly two parts : the “signed” data (usualy “text/plain”) and the
armored signature (“application/pgp-signature”).

As I always got “BAD signature”, I looked at what is passed to gpg for
verification :

[start of signed data]
content-type: text/plain; charset=“utf-8”; format="flowed"
Content-Transfer-Encoding: 7bit
X-RT-Original-Encoding: ISO-8859-15

test
[end of signed data]

DOH ! I forgot RT converts all text parts to UTF-8, “corrupting” the data
before verification happens.

Any idea to work around this “feature” ?

Thanks,

Hi.

I’m (re)writing a GPG email authentication plug-in based on Jesse’s
enhanced-mailgate.

For now, this plug-in should only handle RFC3156 compliant messages (see “5.
OpenPGP signed messages”), that is MIME “multipart/signed” messages composed
of exactly two parts : the “signed” data (usualy “text/plain”) and the
armored signature (“application/pgp-signature”).

As I always got “BAD signature”, I looked at what is passed to gpg for
verification :

[start of signed data]
content-type: text/plain; charset=“utf-8”; format=“flowed”
Content-Transfer-Encoding: 7bit
X-RT-Original-Encoding: ISO-8859-15

test
[end of signed data]

DOH ! I forgot RT converts all text parts to UTF-8, “corrupting” the data
before verification happens.

Any idea to work around this “feature” ?

Thanks,


rt-devel mailing list
rt-devel@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-devel

Handle mail decryption with procmail/maildrop before it reaches RT, and
forward the decrypted mail into the mailgate.

/rhb

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

  • Rasmus Brøgger <Rasmus.Brogger at uni-c.dk> [2003-08-14 14:18]:> On Thu, Aug 14, 2003 at 08:09:34PM +0200, Guillaume Perréal wrote:

Any idea to work around this “feature” ?

Handle mail decryption with procmail/maildrop before it reaches RT,
and forward the decrypted mail into the mailgate.

But then you can’t use GPG as part of the authentication mechanism.

(darren)


An idea is not responsible for the people who believe in it.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)
Comment: This message is digitally signed and can be verified for authenticity.

iD8DBQE/O9PczsinjrVhZaoRApPVAJ4h9ukM3I+KvnLQ8PW327q3HKCACgCgnEaR
xfxjrR9tvOAyppkAbg20LO4=
=lV6E
-----END PGP SIGNATURE-----

  • Rasmus Brøgger <Rasmus.Brogger at uni-c.dk> [2003-08-14 14:18]:

Any idea to work around this “feature” ?

Handle mail decryption with procmail/maildrop before it reaches RT,
and forward the decrypted mail into the mailgate.

This could be used for encrypted message, not for signed one, which I want
to authenticate user from (what Darren pointed out :slight_smile:

But then you can’t use GPG as part of the authentication mechanism.

Maybe the signature verification could be done on the MDA side (rt-mailgate)
which then pass information passed to RT via an additionnal parameter. This
solution needs to make modifications in rt-mailgate,
/REST/1.0/NoAuth/mail-gateway, RT::EmailParser and RT::Interface::Email…

Another way is to keep a copy of the original MIME::Entity during the
authentication process, but it can be painful with large attachements…

Yet another solution is to make the UTF-8 conversion after authentication
phase. I don’t know if and how this could make things going bad, if it ever
did.

Anyone ?

Thanks,

Maybe the signature verification could be done on the MDA side (rt-mailgate)
which then pass information passed to RT via an additionnal parameter. This
solution needs to make modifications in rt-mailgate,
/REST/1.0/NoAuth/mail-gateway, RT::EmailParser and RT::Interface::Email…

I believe this would leave you vulernable to third-parties injecting
falsely valid messages between rt-mailgate and the REST API

Another way is to keep a copy of the original MIME::Entity during the
authentication process, but it can be painful with large attachements…

Yet another solution is to make the UTF-8 conversion after authentication
phase. I don’t know if and how this could make things going bad, if it ever
did.

It might result in butchered user-names in creation. But that could be
compensated for. I’d like autrijus input on this potential change. It
sounds the most reasonable

Anyone ?

Thanks,


rt-devel mailing list
rt-devel@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-devel

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

It might result in butchered user-names in creation. But that could be
compensated for. I’d like autrijus input on this potential change. It
sounds the most reasonable

How about seeing if $Transaction->ContentObj->OriginalContent works?

Thanks,
/Autrijus/

It might result in butchered user-names in creation. But that could be
compensated for. I’d like autrijus input on this potential change. It
sounds the most reasonable

How about seeing if $Transaction->ContentObj->OriginalContent works?

Well, the “call stack” looks like this when I make the GPG check (from inner
to outer) :

GetCurrentUser() from RT::Interface::Email::Auth::GnuPG,
Gateway() from RT::Interface::Email,
Mason component /REST/1.O/NoAuth/mail-gateway

At this time, there isn’t any existing Transaction.

But I managed to make it working, by postponing body parts conversion. Here’s
a summary of changes :

  • in RT::EmailParser, I redefined _PostProcessNewEntity() to convert and
    unfold only main headers. I’m hoping this will avoid getting butchered
    user-names from other mail plug-ins, though it will not mess with signature
    verification as main mail headers couldn’t be (and thus aren’t) signed.

  • in RT::Interface::Email, I redefined Gateway (sig!) to do the UTF-8
    conversion just after authentication (does it fail or pass). (I also changed
    a bit the way AuthStat and CurrentUser are assigned from plug-in return
    values).

  • I rewrote the RT::Interface::email::Auth::GnuPG from enhanced-mailgate’s
    GetCurrentUserFromPGPSignature sub. The main change is to use the
    ‘–status-fd’ option of GPG that allows to get an locale-independant, easily
    parseable status output.

I can post an archive with the two *_Local.pm files and the GnuPG.pm one if
someone wants. But I mainly ask for comments about that. After that, I’ll go
on my way to integrate pseudo-headers handling in the Gateway sub.

Best regards,

Le Vendredi 15 Ao�t 2003 08:35, Autrijus Tang a �crit :

It might result in butchered user-names in creation. But that could be
compensated for. I’d like autrijus input on this potential change. It
sounds the most reasonable

How about seeing if $Transaction->ContentObj->OriginalContent works?

Well, the “call stack” looks like this when I make the GPG check (from inner
to outer) :

GetCurrentUser() from RT::Interface::email::Auth::GnuPG,
Gateway() from RT::Interface::Email,
Mason component /REST/1.O/NoAuth/mail-gateway

At this time, there isn’t any existing Transaction.

It also wouldn’t work, as you need to preserve the mime object.

But I managed to make it working, by postponing body parts conversion. Here’s
a summary of changes :

  • in RT::EmailParser, I redefined _PostProcessNewEntity() to convert and
    unfold only main headers. I’m hoping this will avoid getting butchered
    user-names from other mail plug-ins, though it will not mess with signature
    verification as main mail headers couldn’t be (and thus aren’t) signed.

do you then go and clean up headers later on?

  • in RT::Interface::Email, I redefined Gateway (sig!) to do the UTF-8
    conversion just after authentication (does it fail or pass). (I also changed
    a bit the way AuthStat and CurrentUser are assigned from plug-in return
    values).

What did you need to change? I’d love to see.

  • I rewrote the RT::Interface::email::Auth::GnuPG from enhanced-mailgate’s
    GetCurrentUserFromPGPSignature sub. The main change is to use the
    ‘–status-fd’ option of GPG that allows to get an locale-independant, easily
    parseable status output.

Cool. you should also have a look at Mail::GnuPG.

I can post an archive with the two *_Local.pm files and the GnuPG.pm one if
someone wants. But I mainly ask for comments about that. After that, I’ll go
on my way to integrate pseudo-headers handling in the Gateway sub.

I’d love to see your changes. Also, with your changes, does RT still
pass a “make regression”?

I’d like to recommend that you implement the pseudoheader parsing as its
own module, so that it can be reused by other components that might also
want it, such as a CVS or Subversion integration.
Best,

Jesse

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse Vincent a écrit :

But I managed to make it working, by postponing body parts conversion. Here’s
a summary of changes :

  • in RT::EmailParser, I redefined _PostProcessNewEntity() to convert and
    unfold only main headers. I’m hoping this will avoid getting butchered
    user-names from other mail plug-ins, though it will not mess with signature
    verification as main mail headers couldn’t be (and thus aren’t) signed.

do you then go and clean up headers later on?

_PostProcessNewEntity now calls RT:I18N::SetMIMEHeaderToEncoding
instead of SetMIMEEntityToEncoding. From what I read from the code, I
believe it only affect main headers.

I relie on RT::I18N::SetMIMEEntityToUTF8 to clean up the headers later,
am I right ?

  • in RT::Interface::Email, I redefined Gateway (sig!) to do the UTF-8
    conversion just after authentication (does it fail or pass). (I also changed
    a bit the way AuthStat and CurrentUser are assigned from plug-in return
    values).

What did you need to change? I’d love to see.

Firstly, I put an call to RT::I18N::SetMIMEEntityToUTF8 just after the
authentication process.

Secondly, AuthStat and CurrentUser are only overwritten only if
NewAuthStat is greater or equal to AuthStat. My purpose is that
CurrentUser was not erased if a module returned an AuthLevel of 0 with a
undef’ed user, preserving any preexisting authentication. For example, a
filter module (say RT::Interface::email::Filter::SpamAssassin) can now
safely returns an AuthLevel of 0 and an undefined CurrentUser to let the
message pass without erasing the return status of a previous plug-in
(say RT::Interface::email::Auth::MailFrom). In the original
implementation of RT 3-0-4, specifying a @RT::MailPlugins like
qw(Auth::MailFrom Filter::SpamAssassin) will leads in no user identified
in all cases, as the output of Filter::SpamAssassin will erase the one
of Auth::MailFrom. (by the way, I didn’t manage to have
Filter::SpamAssassin working without hacking it a bit).

  • I rewrote the RT::Interface::email::Auth::GnuPG from enhanced-mailgate’s
    GetCurrentUserFromPGPSignature sub. The main change is to use the
    ‘–status-fd’ option of GPG that allows to get an locale-independant, easily
    parseable status output.

Cool. you should also have a look at Mail::GnuPG.

I tried to use it in a first time but it poorly handles non-signed mails
: it dies… So I was forced to catch the exception and parse it to
guess if it was related to a non-signed mail or to any other error…

Plus, I had problem with the Mail::GnuPG in FastCGI mode : could not get
anything from the output filehandles. I don’t know what it is related to
CGI::Fast (or FCGI), Mail::GnuPG or GnuPG::Interface.

I can post an archive with the two *_Local.pm files and the GnuPG.pm one if
someone wants. But I mainly ask for comments about that. After that, I’ll go
on my way to integrate pseudo-headers handling in the Gateway sub.

I’d love to see your changes. Also, with your changes, does RT still
pass a “make regression”?

Didn’t tried yet, didn’t thought to. I fear this will be interesting… :slight_smile:

I’d like to recommend that you implement the pseudoheader parsing as its
own module, so that it can be reused by other components that might also
want it, such as a CVS or Subversion integration.

I’ll remember that, thanks.

Oh, a question : for now, the signed message is passed unchanged (UTF-8
apart) and the signature is archived as an attachement (or at least as a
text part) but I think this is quite unuseful and maybe misleading. As
the UTF8 conversion will corrupt the signing, I wonder if I could strip
of the signature part and replace the main body by the signed parts,
that is changing the structure of the mail to have it looks as an
ordinary mail. But I don’t know if :

  1. this is really a good thing to do,
  2. this is easy to do (I’m not accustomed with MIME::Entity)

Any suggestion ?

Guillaume Perréal.

Responsable informatique,
Cemagref, groupement de Lyon,
France.

Tél: (+33) 4.72.20.87.87.
Fax: (+33) 4.78.47.78.75.
Site: http://www.lyon.cemagref.fr/

I’d love to see your changes. Also, with your changes, does RT still
pass a “make regression”?

Didn’t tried yet, didn’t thought to. I fear this will be interesting… :slight_smile:

Now I saw how I should include my changes to “make regression”, I think I
should add that I integrated them as *_Local.pm files, not in the core files.

I join an archive of those files (in the current quite untested state) to this
mail.

I will try to intregate them in the distrib source to test them.

Best regards,
Guillaume Perreal

Auth-GnuPg.tgz (6.2 KB)

Darn, I forgot to send this mail with a real address, not the bogus one I use
for testing :-)Le Lundi 18 Août 2003 12:03, Guillaume Perréal a écrit :

I’d love to see your changes. Also, with your changes, does RT still
pass a “make regression”?

Didn’t tried yet, didn’t thought to. I fear this will be interesting…
:slight_smile:

Now I saw how I should include my changes to “make regression”, I think I
should add that I integrated them as *_Local.pm files, not in the core
files.

I join an archive of those files (in the current quite untested state) to
this mail.

I will try to intregate them in the distrib source to test them.

Best regards,

Guillaume Perréal

Auth-GnuPg.tgz (6.2 KB)

Jesse Vincent wrote :

I’d love to see your changes. Also, with your changes, does RT still
pass a “make regression”?

Modified rt-3-0-5pre4 fails one test, concerning a nested rfc822 message :

ok 943 - Loaded nested-rfc-822 to test
ok 944 - It is a mixed multipart
ok 945 - found ticket 29
not ok 946

Failed test (/home/rt3-devel/lib/t/04_send_email.pl at line 370)

got: ‘[Jonas Liljegren] Re: [Para] =?iso-8859-1?q?Niv=E5er=3F?=’

expected: ‘[Jonas Liljegren] Re: [Para] Niv?r?’

ok 947 - We recorded the content type right
ok 948 - Has one attachment, presumably a text-plain and a message RFC 822 and another plain

I join a diff of my changes. Maybe someone could tell me if I made an
obvious error…

Thanks.
Guillaume Perréal.

postpone-utf8.patch (3.47 KB)

Jesse Vincent wrote :

I’d love to see your changes. Also, with your changes, does RT still
pass a “make regression”?

Modified rt-3-0-5pre4 fails one test, concerning a nested rfc822 message :

ok 943 - Loaded nested-rfc-822 to test
ok 944 - It is a mixed multipart
ok 945 - found ticket 29
not ok 946

Failed test (/home/rt3-devel/lib/t/04_send_email.pl at line 370)

got: ‘[Jonas Liljegren] Re: [Para] =?iso-8859-1?q?Niv=E5er=3F?=’

expected: ‘[Jonas Liljegren] Re: [Para] Niv?r?’

ok 947 - We recorded the content type right
ok 948 - Has one attachment, presumably a text-plain and a message RFC 822
and another plain

So yeah, that’s sort of what I was afraid of haivng happen. Apparently,
you’re either not encoding the neaders properly before creating the
ticket or you’re double-encoding them.

I join a diff of my changes. Maybe someone could tell me if I made an
obvious error…

Thanks.

Guillaume Perr�al.
Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.