Anyone else? rt3.2.2+apache2+fastcgi - problem with mailgate (double utf8-encoding)

RT Users
1

Hello,

anybody seeing this same problem with apache2 and fastcgi 2.4.2:

When posting an email with german umlauts in Subject into RT, an email
bounce is created with the following error message

Parsing of undecoded UTF-8 will give garbage when decoding entities at
/export/perl-5.8.5/lib/site_perl/5.8.5/LWP/Protocol.pm line 114, <> chunk 1.

The email is correctly accepted by RT (a ticket is created in the DB
without any visible encoding problems) but the annoying bounce is created.

!! The same email (I attach an example) posted to the same RT (really the
same files) running under apache 1.3.33 with fastcgi 2.2.10 will be
accepted without the error bounce.

complete configuration below!

TRACKED DOWN to the following:

I can reproduce the error by piping the test message into rt-mailgate (with
–debug) when apache2 is up and saw the difference when piping the same
email to the same rt-mailgate when apache 1.3 is up:

rt-mailgate dumps the content of the answer of apache to stdout and this
shows:

apache 1.3 (correct!)
Ok…Subject:testticket für RT…

apache 2 (incorrect, parser complains)
Ok…Subject:testticket f??r RT…

where ?? are the characters I use to see if utf8 - “ü” is encoded to utf8
again.

I can send apache2 and apache1 config file snippets if needed, but I am
quite sure apache config is correct (yes, I use AddDefaultCharset UTF-8 in
the location serving rt/REST).

Any help appreciated,
Dirk.

problematic configuration

Debian Linux (sarge)
RT 3.2.2
RTFM 2.0.4
Apache/2.0.52
perl 5.8.5
fastcgi 2.4.2
mysql 4.0.22

unproblematic configuration

Debian Linux (sarge)
same RT instance
same RTFM instance
Apache/1.3.33 Ben-SSL/1.55
same perl
fastcgi 2.2.10
same mysql server and DB

testticket f_r RT (402 Bytes)

2

Hello rt-users,

here is an update to the issue, which I still do not understand.
I expected that writing

AddDefaultCharset UTF-8

into the virtualhost or rt location forces apache2 to always send an answer
marked as charset=utf-8, but that does not happen.

I now have “AddDefaultCharset UTF-8” in my server config, virtual host
section and rt location, but the answer from mailgate is still marked as
“ISO-8859-1” (see packet dump below) yielding in the warning/error
described in the original post.

Server: Apache/2.0.52 (Debian GNU/Linux) mod_auth_kerb/5.0-rc5
mod_fastcgi/2.4.2 PHP/4.3.9-1 mod_ssl/2.0.52 OpenSSL/0.9.7e
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1

72
okTicket: 11711Queue: rt.testOwner: NobodyStatus: newSubject: testticket
f…r rtRequestor: pape-rt@mi.fu-berlin.de

Could it be, that FastCgiServer is ignoring “AddDefaultCharset UTF-8”?

Dirk.

–Am Dienstag, 14. Dezember 2004 9:40 Uhr +0100 schrieb Dirk Pape
pape-rt@inf.fu-berlin.de:

3

Hello rt-users,

–Am Montag, 20. Dezember 2004 9:10 Uhr +0100 schrieb Dirk Pape
pape-rt@inf.fu-berlin.de:

into the virtualhost or rt location forces apache2 to always send an
answer marked as charset=utf-8, but that does not happen.

I patched this:

http://page.mi.fu-berlin.de/~pape/rt3/patches/rt/3.2.2/mail-gateway_charset_utf8.patch

Now rt-mailgate doesn’t complain anymore.

Are those lines:

<%flags>
inherit => undef # inhibit UTF8 conversion done in /autohandler
</%flags>

really necessary in RST/…/mail-gate?
Is this patch expected to break some other thing?

Dirk.

4

Hello rt-users,

I patched this:

http://page.mi.fu-berlin.de/~pape/rt3/patches/rt/3.2.2/mail-gateway_charset_utf8.patch

Now rt-mailgate doesn’t complain anymore.

Are those lines:

<%flags>
inherit => undef # inhibit UTF8 conversion done in /autohandler
</%flags>

really necessary in RST/…/mail-gate?
Is this patch expected to break some other thing?

By default, RT assumes that all incoming http messages are in UTF8 and
acts accordingly. This can (and in the past did) scramble incoming email
if it wasn’t UTF8 to begin with.

Jesse
5

Hello Jesse,

–Am Mittwoch, 22. Dezember 2004 12:59 Uhr -0500 schrieb Jesse Vincent
jesse@bestpractical.com:

By default, RT assumes that all incoming http messages are in UTF8 and
acts accordingly. This can (and in the past did) scramble incoming email
if it wasn’t UTF8 to begin with.

ok, I changed the patch to set content-type charset to utf8 explicitely in
rt-mailgate and REST/1.0/autohandler.

Since this is expected to be done by apache’s “AddDefaultCharset UFT-8”
anyway (which is disfunctional in my situation for any reason I still don’t
know), it should be harmless:

http://page.mi.fu-berlin.de/~pape/rt3/patches/rt/mail-gateway_charset_utf8.patch

Am I really the only one having this problem?

Dirk.