Rt-mailgate breaks when charset isn't UTF-8

Hi,

RT version 3.0.1

We’ve got some tickets being created from mail that is created from a web
interface. The web interface sends the mail with a charset of iso-8859-1.

What’s happening is that the mail gets to rt-mailgate, and the ticket gets
created, but the message body doesn’t get inserted into the database. The
template that gets sent to the queue admincc watchers shows all the ticket
data is there, but there is no body and no subject. This only occurs for a
couple of mails, where the hyphen appears as

So, we tracked down the problem to rt-mailgate; it looks like rt-mailgate is
assuming everything is UTF-8 encoding, and breaking when it sees a character
that shouldn’t exist.

We’ve worked around the problem by creating a shell script wrapper that
assumes the input encoding is iso-8859-1 and converts to UTF-8, also
changing the LANG to C.

Here’s the script:
#!/bin/sh

rt-mailgate expects input in UTF-8 and bombs out if input is not valid

LANG=C
MAILGATE=/opt/rt3/bin/rt-mailgate
ICONV=/usr/bin/iconv

export LANG

exec $ICONV -f iso8859-1 -t UTF-8 | $MAILGATE “$@”

The problem is that again, non-iso-8859-1 emails will get corrupted, eg
UTF-8 mail using non-ASCII characters.

So, I’d suggest that rt-mailgate shouldn’t assume anything about the
encoding, then look at the mail headers to decide. I think you should
default to an octet-stream if nothing can be found, or US-ASCII (which I
believe is the de-facto standard encoding when not explicit).

By the way, I was also going to submit this to the rt3 bugs address, but I
couldn’t find it listed on the rt website.

jaq@spacepants.org http://spacepants.org/jaq.gpg

Jamie Wilkinson wrote:

RT version 3.0.1

So, we tracked down the problem to rt-mailgate; it looks like rt-mailgate is
assuming everything is UTF-8 encoding, and breaking when it sees a character
that shouldn’t exist.

You really really want to try 3.0.2pre6.

By the way, I was also going to submit this to the rt3 bugs address, but I
couldn’t find it listed on the rt website.

rt-3.0-bugs@fsck.com, but only if pain persists with 3.0.2pre6. :slight_smile:
�|� http://www.bestpractical.com/rt – Trouble Ticketing. Free.

This one time, at band camp, Jamie Wilkinson wrote:

This only occurs for a couple of mails, where the hyphen appears as

better finish my sentence: the hyphen in the mail is an iso8859-1 hyphen,
code 0xad. The default ascii hyphen works in both charsets, but the 0xad
hyphen doesn’t exist in UTF-8 and hence something breaks.

jaq@spacepants.org http://spacepants.org/jaq.gpg

This one time, at band camp, Phil Homewood wrote:

Jamie Wilkinson wrote:

RT version 3.0.1

So, we tracked down the problem to rt-mailgate; it looks like rt-mailgate is
assuming everything is UTF-8 encoding, and breaking when it sees a character
that shouldn’t exist.

You really really want to try 3.0.2pre6.

Ok, thanks. Can you point to the change number that fixes this? I’d prefer
to apply just the one patch than upgrade the lot just at this point in time.

jaq@spacepants.org http://spacepants.org/jaq.gpg

This one time, at band camp, Jamie Wilkinson wrote:

This one time, at band camp, Phil Homewood wrote:

Jamie Wilkinson wrote:

RT version 3.0.1

So, we tracked down the problem to rt-mailgate; it looks like rt-mailgate is
assuming everything is UTF-8 encoding, and breaking when it sees a character
that shouldn’t exist.

You really really want to try 3.0.2pre6.

Ok, thanks. Can you point to the change number that fixes this? I’d prefer
to apply just the one patch than upgrade the lot just at this point in time.

Found it, change 80. Thanks :wink:

jaq@spacepants.org http://spacepants.org/jaq.gpg

Ok, thanks. Can you point to the change number that fixes this? I’d prefer
to apply just the one patch than upgrade the lot just at this point in time.

Found it, change 80. Thanks :wink:

As a heads up:

There were a number of different changes that fixed different parts of
this issue. Applying patches piecemeal may get you into trouble.
Jesse


jaq@spacepants.org http://spacepants.org/jaq.gpg


rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

http://www.bestpractical.com/rt – Trouble Ticketing. Free.

This one time, at band camp, Jesse Vincent wrote:

Ok, thanks. Can you point to the change number that fixes this? I’d prefer
to apply just the one patch than upgrade the lot just at this point in time.

Found it, change 80. Thanks :wink:

As a heads up:

There were a number of different changes that fixed different parts of
this issue. Applying patches piecemeal may get you into trouble.

Yeah, I think I’ve found most of them.

Could I ask that you restrict your changes to just one feature or bugfix at
a time, in future?

jaq@spacepants.org http://spacepants.org/jaq.gpg

Yeah, I think I’ve found most of them.

Could I ask that you restrict your changes to just one feature or bugfix at
a time, in future?

Unfortunately, that’s not always an option, especially when we’re
merging changes/fixes from a non-local branch.

http://www.bestpractical.com/rt – Trouble Ticketing. Free.