Norwegian, Swedish and Danish character problems

We have a single server set up for our multi-country rt3 server.
But we have a big problem… I regard it as a showstopper, due to
the amount of these letters beeing used in the affected languages.

The characters we are having trouble with are:
��� ��� ��� ��� (And I do not know the Danish ones)

All of these appear as double-character codes. ex: æ
This does not happen always, but I would say something like 50%
of the new messages that arrive are like this.

Now, it seems to me like the EmailInputEncodings setting in the
config file is somehow fault, because changing the order of the types
does change what messages appear wrong and what are right. It
actually seems to me like rt only sees the first one, then uses that no
matter what the actual format of the mail is. Or something…

From my RT conf file:
@EmailInputEncodings = qw(utf-8 iso-8859-1 us-ascii) unless
(@EmailInputEncodings);
Set($EmailOutputEncoding , ‘utf-8’);

Apache has been set to use UTF-8 as default.

From one of the tickets:
content-type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: iso-8859-1

Ønsker å bli kontaktet angående

This is clearly wrong… And this is really a big problem.

I have checked, and it is not true that this only affects messages
written in Outlook.
I have seen this problem appear with:

-PHP’s mail(); function using the same server as rt is hosted on
content-type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: iso-8859-1

-Apple Mail (2.613):
content-type: text/plain; charset=“utf-8”; format=“flowed”
X-RT-Original-Encoding: ISO-8859-1

-Outlook Express:
content-type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: iso-8859-1

-Microsoft something ( MimeOLE V6.00.2800.1165)
content-type: text/plain; charset=“utf-8”
X-RT-Original-Encoding: us-ascii

So, to me it seems like it somehow converts the message into utf-8,
but forgets to save to the ticket info that it has been converted. Thus
converts it again for output. Because… none of these messages were
sent using utf-8 type.

Sending an original utf-8 message (from Evolution) to rt works just
fine… But I cannot make everyone change to Evolution either…

I hope this helps in finding the cause…

I run:
RT 3.0.10
perl-HTML-Parser-3.26-17
perl-XML-Parser-2.31-15
perl-libxml-enno-1.02-29
perl-DBD-MySQL-2.1021-3
perl-suidperl-5.8.0-88.3
perl-GD-Text-Util-0.86-0.rh90.dag
perl-Filter-1.29-3
perl-HTML-Tagset-3.03-28
perl-Parse-Yapp-1.05-30
perl-libwww-perl-5.65-6
perl-libxml-perl-0.07-28
perl-XML-Encoding-1.01-23
perl-XML-Grove-0.46alpha-25
perl-DBI-1.32-5
perl-GD-1.41-1.rh90.dag
perl-GD-Graph-1.43-1
perl-5.8.0-88.3
perl-CGI-2.81-88.3
perl-DateManip-5.40-30
perl-URI-1.21-7
perl-XML-Dumper-0.4-25
perl-XML-Twig-3.09-3
perl-CPAN-1.61-88.3
Apache 2.0.47 w/FastCGI

-HK

We have a single server set up for our multi-country rt3 server.
But we have a big problem… I regard it as a showstopper, due to
the amount of these letters beeing used in the affected languages.

The characters we are having trouble with are:
??? ??? ??? ??? (And I do not know the Danish ones)

perl-5.8.0-88.3

Perl 5.8.0 is known to be broken in this regard. Please upgrade to
perl 5.8.3 or 5.8.4.

Jesse

The characters we are having trouble with are:
??? ??? ??? ??? (And I do not know the Danish ones)
perl-5.8.0-88.3

Perl 5.8.0 is known to be broken in this regard. Please upgrade to
perl 5.8.3 or 5.8.4.

Wow, it seems like that solved both the character and the
speed problems.

Now I just wonder why the rt-test-dependencies script doesn’t issue
atleast a warning when finding perl 5.8.0. It just says FOUND like
it should if everything is ok. Please add a warning message.

I used Redhat 9.0 with the latest updates, and that means Perl 5.8.0.
For those interested, what I did to upgrade was:
-Download perl-5.8.3-16.src.rpm from a fedora core 1 ftp
-run: rpmbuild --rebuild --target i686 perl-5.8.3-16.src.rpm
-Now, install the 3 perl rpm files located in /usr/src/redhat/RPMS/i686/

-HK

Perl 5.8.0 is known to be broken in this regard. Please upgrade to
perl 5.8.3 or 5.8.4.

Wow, it seems like that solved both the character and the
speed problems.

Glad to hear it.

Now I just wonder why the rt-test-dependencies script doesn’t issue
atleast a warning when finding perl 5.8.0. It just says FOUND like
it should if everything is ok. Please add a warning message.

It was an oversight. And has been fixed for RT 3.0.11. Thanks for the
reminder.