Critical bug with german letter' (Intern)

I had mySQL 3.23, so I upgraded to MySQL 4.0.16.
I also reinstalled rt-3.0.7.
(rm -Rf rt3; make testdeps; make install; make dropdb; make initialize-database)

Created a ticket with the 3 norwegian special characters, and I get the same result as before.
Tested on IE and Opera - same.

MySQL 4.0.16
Apache 2.0.40
Perl 5.8.0
FastCGI 2.4.0
RT 3.0.7_01

Regards,
Ronny Pettersen

btw:
I have an other install running mysql 3.23 and RT-3.0.6 where I can get it working by
making sure not to add a comment to a ticket after httpd restart (2nd comment will be wrong).
If i create a ticket, the comments to this and all other tickets are encoded just fine.From: rt-users-bounces@lists.fsck.com
[mailto:rt-users-bounces@lists.fsck.com]On Behalf Of Jesse Vincent
Sent: 19. november 2003 06:12
To: Andreas Falk
Cc: rt-users@lists.fsck.com
Subject: Re: SV: [rt-users] Re: critical bug with german letter’

We have the same problem with the swedish åäö ÅÄÖ.

Here’s one ticket that have this problem:
http://rt3.fsck.com/Ticket/Display.html?id=4365

So. The reason rt3.fsck.com exhibits this bug is because it’s running on
perl 5.6.1. Perl 5.6.1 is non-supported for RT and known to break on
non-ascii characters. We haven’t been able to reproduce this problem
on perl 5.8.x. If you can do so, the way to get RT fixed is to extend
the test suite to show off the problem. If you can send us a patch for
the test suite that fails due to this bug, then we’ve got something to
work from to fix.

Best,
Jesse

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

MySQL 4.0.16
Apache 2.0.40
Perl 5.8.0

This is RedHat’s build of perl 5.8.0?
Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

MySQL 4.0.16
Apache 2.0.40
Perl 5.8.0

This is RedHat’s build of perl 5.8.0?

Yes… perl-5.8.0-88.3.i386.rpm (13 Aug 2003 build).
Everything except MySQL (from mysql.com) is from RedHat.

btw:
Have the same problem on a box with perl 5.8.1 from Fedora.

This is RedHat’s build of perl 5.8.0?

Yes… perl-5.8.0-88.3.i386.rpm (13 Aug 2003 build).
Everything except MySQL (from mysql.com) is from RedHat.

You’ve seen the writeups about RedHat 9 and how you need to set the
environment’s locale before running RT?

btw:
Have the same problem on a box with perl 5.8.1 from Fedora.

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

You’ve seen the writeups about RedHat 9 and how you need to set the
environment’s locale before running RT?

Yes.
I’ve always been using “LANG=en_US” (not en_US.UTF-8).
I’ve also tried LANG=C. LC_ALL was unset, tried LC_ALL=C as well.
And setting initial-env in httpd.conf, tried both en_US and C:
FastCgiServer /usr/local/rt3/bin/mason_handler.fcgi -init-start-delay 15 -idle-timeout 3600 -initial-env LANG=en_US
(without this, LANG does not seem to be set, according to /proc//environ)

Anything else I’ve missed?

Regards,
Ronny Pettersen

btw:
As I’ve said earlier, I have one rt-3.0.6 install which is working - as long as I don’t restart it.
If I do restart it, I need to create 1 ticket before anyone adds comments.
But this “trick” doesn’t work on 3.0.7, and the resulting encoding contains more characters than in 3.0.6.

I’ve written a test using WWW::Mechanize that generates the same
ticket with the wrong encoding. (Ticket subject is fine - correct encoding, content is not).
I used the lib/t/03web.pl to write something that just creates this ticket,
haven’t done anything to check if it went fine or not.
Maybe a start for a more proper testscript…

Regards,
Ronny Pettersen
#!/usr/bin/perl
use strict;
use WWW::Mechanize;
use HTTP::Request::Common;
use HTTP::Cookies;
use LWP;
use Test::More qw(no_plan);

use lib “/usr/local/rt3/lib”;
use RT;
ok(RT::LoadConfig);

my $cookie_jar = HTTP::Cookies->new;
my $agent = WWW::Mechanize->new();

give the agent a place to stash the cookies

$agent->cookie_jar($cookie_jar);

get the top page

my $url = “http://localhost”.$RT::WebPath.“/”;
$agent->get($url);

is ($agent->{‘status’}, 200, “Loaded a page”);

follow the link marked “Login”

ok($agent->{form}->find_input(‘user’));
ok($agent->{form}->find_input(‘pass’));
ok ($agent->{‘content’} =~ /username:/i);
$agent->field( ‘user’ => ‘root’ );
$agent->field( ‘pass’ => ‘password’ );

the field isn’t named, so we have to click link 0

$agent->click(0);
is($agent->{‘status’}, 200, “Fetched the page ok”);

$agent->get($url.“Ticket/Create.html?Queue=1”);
is ($agent->{‘status’}, 200, “Loaded Create.html”);
$agent->form(3);
$agent->field(‘Subject’ => Encode::encode_utf8(“Testing æøå”));
$agent->field(‘Content’ => Encode::encode_utf8(“Content æøå”));
ok($agent->submit(), “Created new ticket with æøå”);

}}}

1;

I’ve written a test using WWW::Mechanize that generates the same
ticket with the wrong encoding. (Ticket subject is fine - correct encoding, content is not).
I used the lib/t/03web.pl to write something that just creates this ticket,
haven’t done anything to check if it went fine or not.
Maybe a start for a more proper testscript…

And with this, I can replicate the problem you’ve described. RT is
fairly convinced that the message the script is sending is in latin1 and
so doing the “apropriate” conversions.

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

I’ve written a test using WWW::Mechanize that generates the same
ticket with the wrong encoding. (Ticket subject is fine - correct encoding, content is not).
I used the lib/t/03web.pl to write something that just creates this ticket,
haven’t done anything to check if it went fine or not.
Maybe a start for a more proper testscript…

And I have a fix. But I want to test it a bit more and I’m about to be
late for dinner. It should get committed later tonight

Regards,
Ronny Pettersen

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

The following patch should fix this issue. I’d love to hear if it fixes
things for those of you seeing this isse

pallas:% diff /tmp/rt-3-0-7/lib/RT/Interface/Web.pm Web.pm
510a511

        Charset => 'utf8',

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Tnx Jesse.

I’ve been struggeling with this since the start of 3.0x
But now it seems to work…

Specs:
rh9 (beq deb does not like the broadcom nic on our hp dl360g3)
perl-5.8.0-88.3
httpd-2.0.40-21.5
mod_fastcgi-2.4.0
MySQL-4.0.16-0

[root@rt src]# locale
LANG=C
LC_CTYPE=“C”
LC_NUMERIC=“C”
LC_TIME=“C”
LC_COLLATE=“C”
LC_MONETARY=“C”
LC_MESSAGES=“C”
LC_PAPER=“C”
LC_NAME=“C”
LC_ADDRESS=“C”
LC_TELEPHONE=“C”
LC_MEASUREMENT=“C”
LC_IDENTIFICATION=“C”
LC_ALL=

Btw… we are working on a translation to swedish, but it is a long road
ahead…

A lovely scrot. http://www.edu.nykoping.se/buit/rt3/works.png

// falk
// buit

-----Ursprungligt meddelande-----
Från: rt-users-bounces@lists.fsck.com
[mailto:rt-users-bounces@lists.fsck.com] För Jesse Vincent
Skickat: den 21 november 2003 06:33
Till: ronny.pettersen@telenor.com
Kopia: rt-users@lists.fsck.com
Ämne: Re: [rt-users] Re: critical bug with german letter’ (Intern)

The following patch should fix this issue. I’d love to hear if it fixes
things for those of you seeing this isse

pallas:% diff /tmp/rt-3-0-7/lib/RT/Interface/Web.pm Web.pm 510a511

        Charset => 'utf8',

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.
rt-users mailing list
rt-users@lists.fsck.com http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

I finally got around to test this too.
And it works great, tested on both 3.0.6 and 3.0.7_01.
Thanks Jesse!

-----Opprinnelig melding-----Fra: Jesse Vincent [mailto:jesse@bestpractical.com]
Sendt: 21. november 2003 06:33
Til: Pettersen Ronny (Networks)
Kopi: rt-users@lists.fsck.com
Emne: Re: [rt-users] Re: critical bug with german letter’ (Intern)

The following patch should fix this issue. I’d love to hear if it fixes
things for those of you seeing this isse

pallas:% diff /tmp/rt-3-0-7/lib/RT/Interface/Web.pm Web.pm
510a511

        Charset => 'utf8',

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.