RT 3.0.2 I18N patch proposal

Hello everybody,

Some international characters display incorrectly in my RT 3.0.2
setup. I’ve come to use the following patch to correct these
problems.

–> share/html/l, lib/RT/CurrentUser.pm
In short, the localization filter was not always calling "decode_utf8"
when it should, mainly when using strings such as “#[_1]: …”, that
is, strings with arguments.

–> lib/RT/I18N.pm
In method “maketext”, I found that some localized strings were both
using UTF-8 and ISO-8859-1 chars.

Can you please review this patch ?

Thank you.

Remy Chibois

rt-3.0.2-I18N.patch.gz (557 Bytes)

Some international characters display incorrectly in my RT 3.0.2
setup. I’ve come to use the following patch to correct these
problems.

–> share/html/l, lib/RT/CurrentUser.pm
In short, the localization filter was not always calling "decode_utf8"
when it should, mainly when using strings such as “#[_1]: …”, that
is, strings with arguments.

These two looks like cool.

–> lib/RT/I18N.pm
In method “maketext”, I found that some localized strings were both
using UTF-8 and ISO-8859-1 chars.

Can you please elaborate on this a bit? I fail to see the rationale
behind it…

/Autrijus/

Hello,

–Am Freitag, 16. Mai 2003 10:31 Uhr +0200 schrieb Remy Chibois
rchibois@free.fr:

–> share/html/l, lib/RT/CurrentUser.pm
In short, the localization filter was not always calling "decode_utf8"
when it should, mainly when using strings such as “#[_1]: …”, that
is, strings with arguments.

I applied this patch to 3.0.2 and atfer doing this some entities with
german umlauts don not display any more in the web interface:

in the select field for ticket status the entries for deleted (german
"gelöscht") and stalled (german “zurückgestellt”) appear blank in a browser
with german language pref.

Dirk.

Hello,

–Am Montag, 19. Mai 2003 11:55 Uhr +0200 schrieb Dirk Pape
pape-rt@inf.fu-berlin.de:

I applied this patch to 3.0.2 and atfer doing this some entities with
german umlauts don not display any more in the web interface:

in the select field for ticket status the entries for deleted (german
"gelöscht") and stalled (german “zurückgestellt”) appear blank in a
browser with german language pref.

ok, I digged more into this and found that the patch to share/html/l is the
problem. I think it is wrong!

localized strings with german umlauts in it do not display in the web
interface any more. Reinstalling the original version did not work for me
first since Mason cached the patched versions. I had to “touch” the
reinstalled originals to make them active again (cost me some time to
figure it out).

My config:

RT-3.0.2 on debian woody with
perl 5.8.0
modperl 1
apache 1.3.26
sbin/rt-test-dependencies --with-modperl --with-mysql happy with my
installed libs

I will open another thread with what I think are the actual issues with
international character handling in RT. They are the only things, which
stop me to make rt 3 productive.

Dirk.

Quoting Dirk Pape pape-rt@inf.fu-berlin.de:

[…]

ok, I digged more into this and found that the patch to share/html/l is
the
problem. I think it is wrong!

This is why I wanted people using international chars to test it.
What LANG are you using ?
What are your InputEncoding and OutputEncoding in RT_SiteConfig.pm ?
This is strange. I tried numerous things before patching, such as
changing LANG ou using a fresh database, but localized strings were
no longer displaying correctly in the web interfce when I switched from
3.0.1 to 3.0.2.
Do other localized strings display correctly (is it a umlaut specific
issue) ?
Without the patch, are all international chars displayed correctly
(interface and/or emails) ?

My setup is:
RT-3.0.2 on RH 7.3
perl 5.8.0
modperl 1
apache 1.3.27
LANG=en_US.iso885915 (also tried with LANG=C)

Can other people using international chars post their experiences and
setup ?

[…]

I will open another thread with what I think are the actual issues with

international character handling in RT. They are the only things, which

stop me to make rt 3 productive.

rt 3 is used daily here and I’m doing my best to help correct those
I18N problems which poison that great software. Perhaps can we work a
bit on the patch so that it works for you ?

Remy

This is why I wanted people using international chars to test it.
What LANG are you using ?

Not quite sure what you mean, but phpinfo() tells me that an
environment-variable LANG is set to ‘C’.

What are your InputEncoding and OutputEncoding in RT_SiteConfig.pm ?

@EmailInputEncodings = qw(utf-8 iso-8859-1 us-ascii) unless
(@EmailInputEncodings);

Set($EmailOutputEncoding , ‘iso-8859-1’);

Without the patch, are all international chars displayed correctly
(interface and/or emails) ?

My experience is that characters never have been displayed correctly.
After applying this 3.0.2-patch, I see the exact same behaviour as
before (maybe I did something wrong?). As well as norwegian characters
being displayed wrong, some of the other characters that appear (or,
should have appeared) after the norwegian characters, disappears. I gave
an example of this in an earlier thread on the rt-users-list.

I’m using RT 3.0.2 on Debian woody
Perl 5.6.1
modperl 1
apache 1.3.26

Kristian R�nningen krmm@nordkapp.net

Quoting Kristian R�nningen krmm@nordkapp.net:

This is why I wanted people using international chars to test it.
What LANG are you using ?

Not quite sure what you mean, but phpinfo() tells me that an
environment-variable LANG is set to ‘C’.

That’s what I meant. Thanks.

My experience is that characters never have been displayed correctly.
After applying this 3.0.2-patch, I see the exact same behaviour as
before (maybe I did something wrong?). As well as norwegian characters
being displayed wrong, some of the other characters that appear (or,
should have appeared) after the norwegian characters, disappears. I
gave
an example of this in an earlier thread on the rt-users-list.

Ok. I can only test with Perl 5.8.0 (time restrictions mainly).
Perhaps you should try to setup an RT instance with Perl 5.8.0 as some earlier
posts state that this correct some display issues.

Remy Chibois

Hello,

–Am Dienstag, 20. Mai 2003 10:22 Uhr +0200 schrieb Remy Chibois
rchibois@free.fr:

This is why I wanted people using international chars to test it.
What LANG are you using ?

de

What are your InputEncoding and OutputEncoding in RT_SiteConfig.pm ?

@EmailInputEncodings = qw(utf-8 iso-8859-1 iso-8859-15 us-ascii) unless
(@EmailInputEncodings);
Set($EmailOutputEncoding , ‘iso-8859-1’);

This is strange. I tried numerous things before patching, such as
changing LANG ou using a fresh database, but localized strings were
no longer displaying correctly in the web interfce when I switched from
3.0.1 to 3.0.2.
Do other localized strings display correctly (is it a umlaut specific
issue) ?

other localized strings are correct, localized strings with umlauts in
localization appear as empty strings.

Without the patch, are all international chars displayed correctly
(interface and/or emails) ?

yes, both, except the two issues I reported to rt-users last hour.

Thanks,
Dirk.

Hello Kristian,

–Am Dienstag, 20. Mai 2003 10:59 Uhr +0200 schrieb Kristian Rønningen
krmm@nordkapp.net:

I’m using RT 3.0.2 on Debian woody
Perl 5.6.1
modperl 1
apache 1.3.26

I would bet, that it is due to Perl 5.6.1. Our configurations are quite
identical, despite of me using Perl 5.8.0, which has unicode support.

RT 3.0.2 works for me in most cases, but not perfect though (you see my
config and issues elsewhere in this thread).

Dirk.

Hello,

–Am Dienstag, 20. Mai 2003 11:35 Uhr +0200 schrieb Dirk Pape
pape-rt@inf.fu-berlin.de:

de

de is the language I use in the webinterface. If you need to know the
locale, I am not quite sure how to determine.

locale as root gives:

LANG=POSIX
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=

locale -a gives a long list with the install locales (C is among them and
some de*)

Dirk.

Hello everybody,

Some international characters display incorrectly in my RT 3.0.2
setup. I’ve come to use the following patch to correct these
problems.

–> share/html/l, lib/RT/CurrentUser.pm
In short, the localization filter was not always calling "decode_utf8"
when it should, mainly when using strings such as “#[_1]: …”, that
is, strings with arguments.

–> lib/RT/I18N.pm
In method “maketext”, I found that some localized strings were both
using UTF-8 and ISO-8859-1 chars.

That should never, ever be the case. Where were you finding iso-8859-1
characters?

Can you please review this patch ?

This patch doesn’t appear to actually fix anything on my text system.
Non-ascii characters still aren’t displaying for me.

Thank you.


Remy Chibois

http://www.bestpractical.com/rt – Trouble Ticketing. Free.

Quoting Jesse Vincent jesse@bestpractical.com:

Hello everybody,

Some international characters display incorrectly in my RT 3.0.2
setup. I’ve come to use the following patch to correct these
problems.

–> share/html/l, lib/RT/CurrentUser.pm
In short, the localization filter was not always calling
"decode_utf8"
when it should, mainly when using strings such as “#[_1]: …”, that
is, strings with arguments.

When not using this chunk, every translated string with arguments does not
display correctly. Normal UTF-8 chars are “prefixed” by other UTF-8 chars. The
HTML source therefor contains 4 bytes for each international char.
Those problems occur for example in the “Connected as …” header string and in
the "#: " string (Display.html).

–> lib/RT/I18N.pm
In method “maketext”, I found that some localized strings were both
using UTF-8 and ISO-8859-1 chars.

This chunk was “necessary” after applying the first one, so it appears this is a
side effect.

This patch doesn’t appear to actually fix anything on my text system.
Non-ascii characters still aren’t displaying for me.

Allright, let’s ignore this one.
Do you think the LANG environment variable has something to do with these problems ?

Remy Chibois