Modperl2: Scope problem?

RT Developers
1

Hi!

Does someone know of any scope problems with moperl2?

A testcase would be the patch I submitted at:

http://rt3.fsck.com/Ticket/Display.html?id=5286

If I assign a constant value inside LoadOrCreateByEmail(), let’s say,
something like:

my $email=‘Joe Doe joedoe@domain.com’;

Everything goes fine… if I don’t then I get an:

Unrecognised line: Jesse Vincent joedoe@domain.com at /usr/lib/rt/RT/EmailParser.pm line 406

I’ve gone deep until Mail::Address code itself and the parameter value
seems to be correct all the way. I test the perl code inside a script and
it runs ok.

I’m running out of ideas… can someone give me some light…

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

2

Everything goes fine… if I don’t then I get an:

Unrecognised line: Jesse Vincent joedoe@domain.com at /usr/lib/rt/RT/EmailParser.pm line 406

I’ve gone deep until Mail::Address code itself and the parameter value
seems to be correct all the way. I test the perl code inside a script and
it runs ok.

I found the same problem with an entire different setup at:
http://www.nntp.perl.org/group/perl.qpsmtpd/1226

I tested it with Mail::Address 1.58 and 1.62 same behavior. The most
weired thing it that it works running a script from command-line calling
directly and indirectly (through RT::EmailParser::ParseAddressFromHeader),
hardcoding the $email variable…

It does not work even reimplementing the ParseAddressFromHeader locally at
User_Overlay.pm…

Any hints? I’m getting mad with this one…

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

3

Everything goes fine… if I don’t then I get an:

Unrecognised line: Jesse Vincent joedoe@domain.com at /usr/lib/rt/RT/EmailParser.pm line 406

I’ve gone deep until Mail::Address code itself and the parameter value
seems to be correct all the way. I test the perl code inside a script and
it runs ok.

I found the same problem with an entire different setup at:
Mail::Address->Parse croaks... - nntp.perl.org

I tested it with Mail::Address 1.58 and 1.62 same behavior. The most
weired thing it that it works running a script from command-line calling
directly and indirectly (through RT::EmailParser::ParseAddressFromHeader),
hardcoding the $email variable…

It does not work even reimplementing the ParseAddressFromHeader locally at
User_Overlay.pm…

Any hints? I’m getting mad with this one…

Just for the record, the cause and solution:

Mail::Address->parse($email) caused an the error (croaks):

Unrecognised line: ‘foo@mydomain.com’

Well it seems that perl 5.8.0-5.8.X has a problem with utf8/regexps.

The internal representation of perl strings has an UTF-8 flag associated
that can be turned off via Encode::_utf8_off($string), so a way to solve
some issues like this are to use this workaround. (see Encode
documentation for more details).

I saw postings refering that the problem may be cause by using perl with
thread support, which is not the case I tested it under perl with and
without thread support with the same results. (The stock perl shipped with
some distros are threaded enabled: e.g. RedHat 9, RHEL, WBEL…)

A generic workaround to cases like this would be to use:

Encode:_utf8_off($parameter_variable);

before calling the problematic function, which in this case is
Mail::Address->parse().

This has to be used with caution since it messes with perl internals, so
sometimes you’ll have to switch the flag on after calling the function.

If there’s something incorrect above please let me know!!!

Cheers,

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

4

So, this is one of the issues that was fixed in 5.8.3?On Tue, Apr 27, 2004 at 08:13:47PM +0100, Paulo Matos wrote:

On Mon, 26 Apr 2004, Paulo Matos wrote:

On Mon, 26 Apr 2004, Paulo Matos wrote:

Everything goes fine… if I don’t then I get an:

Unrecognised line: Jesse Vincent joedoe@domain.com at /usr/lib/rt/RT/EmailParser.pm line 406

I’ve gone deep until Mail::Address code itself and the parameter value
seems to be correct all the way. I test the perl code inside a script and
it runs ok.

I found the same problem with an entire different setup at:
Mail::Address->Parse croaks... - nntp.perl.org

I tested it with Mail::Address 1.58 and 1.62 same behavior. The most
weired thing it that it works running a script from command-line calling
directly and indirectly (through RT::EmailParser::ParseAddressFromHeader),
hardcoding the $email variable…

It does not work even reimplementing the ParseAddressFromHeader locally at
User_Overlay.pm…

Any hints? I’m getting mad with this one…

Just for the record, the cause and solution:

Mail::Address->parse($email) caused an the error (croaks):

Unrecognised line: ‘foo@mydomain.com’

Well it seems that perl 5.8.0-5.8.X has a problem with utf8/regexps.

The internal representation of perl strings has an UTF-8 flag associated
that can be turned off via Encode::_utf8_off($string), so a way to solve
some issues like this are to use this workaround. (see Encode
documentation for more details).

I saw postings refering that the problem may be cause by using perl with
thread support, which is not the case I tested it under perl with and
without thread support with the same results. (The stock perl shipped with
some distros are threaded enabled: e.g. RedHat 9, RHEL, WBEL…)

A generic workaround to cases like this would be to use:

Encode:_utf8_off($parameter_variable);

before calling the problematic function, which in this case is
Mail::Address->parse().

This has to be used with caution since it messes with perl internals, so
sometimes you’ll have to switch the flag on after calling the function.

If there’s something incorrect above please let me know!!!

Cheers,

–
Paulo Matos

|Sys & Net Admin | Serviço de Informática |
|Faculdade de Ciências e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

Rt-devel mailing list
Rt-devel@lists.bestpractical.com
The rt-devel Archives

5

IS rt3.fsck.com moved somewhere else?

I send mails to rt-bugs to update #5286 and it does not appear there…

I don’t get it returned as well…

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

6

I’ve got a filter in front of it. Let me go poke at the logs.On Tue, Apr 27, 2004 at 08:49:03PM +0100, Paulo Matos wrote:

IS rt3.fsck.com moved somewhere else?

I send mails to rt-bugs to update #5286 and it does not appear there…

I don’t get it returned as well…

–
Paulo Matos

|Sys & Net Admin | Servi?o de Inform?tica |
|Faculdade de Ci?ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

7

FWIW, I’ve tracked the issue to a taint error in RT 3.1. All mail that
hasn’t been delivered has been spooled. I’m working on the bug in the
software, then will redeliver the mail.On Tue, Apr 27, 2004 at 08:49:03PM +0100, Paulo Matos wrote:

IS rt3.fsck.com moved somewhere else?

I send mails to rt-bugs to update #5286 and it does not appear there…

I don’t get it returned as well…

–
Paulo Matos

|Sys & Net Admin | Servi?o de Inform?tica |
|Faculdade de Ci?ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

8

FWIW, I’ve tracked the issue to a taint error in RT 3.1. All mail that
hasn’t been delivered has been spooled. I’m working on the bug in the
software, then will redeliver the mail.

Read my mails before redelivering… there’s one repeated… and one will
originating an already existent ticket… :frowning: thougth they were lost… and
also kind of messy from this last bug I was fighting off…

Almost done with RPM packages for RT…

I’ll keep in touch…

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

9

FWIW, I’ve tracked the issue to a taint error in RT 3.1. All mail that
hasn’t been delivered has been spooled. I’m working on the bug in the
software, then will redeliver the mail.

Read my mails before redelivering… there’s one repeated… and one will
originating an already existent ticket… :frowning: thougth they were lost… and
also kind of messy from this last bug I was fighting off…

nod I see that you’ve gone and fixed bugs for things that aren’t an
issue on 5.8.3. Libraries RT depends on are already known to be badly
broken on 5.8.0 due to bugs in perl. I’m feeling rather hesitant to
drop in explicit workarounds there. :confused: Thoughts?

V

Best,
Jesse

10

FWIW, I’ve tracked the issue to a taint error in RT 3.1. All mail that
hasn’t been delivered has been spooled. I’m working on the bug in the
software, then will redeliver the mail.

Read my mails before redelivering… there’s one repeated… and one will
originating an already existent ticket… :frowning: thougth they were lost… and
also kind of messy from this last bug I was fighting off…

nod I see that you’ve gone and fixed bugs for things that aren’t an
issue on 5.8.3. Libraries RT depends on are already known to be badly
broken on 5.8.0 due to bugs in perl. I’m feeling rather hesitant to
drop in explicit workarounds there. :confused: Thoughts?

Well, IMHO the easier RT can be installed most widely it will be used.
Since 5.8.3 isn’t shipped with a lot of distros right now… it will be
nice to have it working… without the hell that can be an upgrade of
perl…

Humm… a way of doing things might be to use version specif conditions…
might not be the most elegant… but… it works…

I noticed that Encode::utf8_off() is used on other places… why?

Are perl 5.8.3 with a good UTF8 support? If so, it might be worth trying
to do a perl upgrade…

what do you say?

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

11

Well, IMHO the easier RT can be installed most widely it will be used.

The problem is that we run into people griping about how RT is broken
when the real issue is that they’re using it on a different perl.

Humm… a way of doing things might be to use version specif conditions…
might not be the most elegant… but… it works…

nod having a “compatibility” module that wraps things that need it
would be very cool. The idea in my head is that it would preload the
module we need to wrap and perform a bit of symbol table frobbery if
necessary for the running perl.

I noticed that Encode::utf8_off() is used on other places… why?

A lot of it predates 5.8.3. Some of it is for using other modules and
systems that aren’t utf8-friendly.

Are perl 5.8.3 with a good UTF8 support? If so, it might be worth trying
to do a perl upgrade…

Yeah, I’m much happier with 5.8.3. I don’t object to doing things to
support older perls, but I worry about going to lengths to do so.

12

Paulo Matos wrote:

FWIW, I’ve tracked the issue to a taint error in RT 3.1. All mail that
hasn’t been delivered has been spooled. I’m working on the bug in the
software, then will redeliver the mail.

Read my mails before redelivering… there’s one repeated… and one will
originating an already existent ticket… :frowning: thougth they were lost… and
also kind of messy from this last bug I was fighting off…

nod I see that you’ve gone and fixed bugs for things that aren’t an
issue on 5.8.3. Libraries RT depends on are already known to be badly
broken on 5.8.0 due to bugs in perl. I’m feeling rather hesitant to
drop in explicit workarounds there. :confused: Thoughts?

Well, IMHO the easier RT can be installed most widely it will be used.
Since 5.8.3 isn’t shipped with a lot of distros right now… it will be
nice to have it working… without the hell that can be an upgrade of
perl…

Humm… a way of doing things might be to use version specif conditions…
might not be the most elegant… but… it works…

I noticed that Encode::utf8_off() is used on other places… why?

Are perl 5.8.3 with a good UTF8 support? If so, it might be worth trying
to do a perl upgrade…
Reseantly released perl 5.8.4 has UTF speed up also.
Stable, but still there is may be errors with UTF.
perl has error with join+UTF until 5.8.3, I don’t think that you can
find all places where Encode utf8_off must be used.
Best regards. Ruslan.

13

Are perl 5.8.3 with a good UTF8 support? If so, it might be worth trying
to do a perl upgrade…

Recently released perl 5.8.4 has UTF speed up also.
Stable, but still there may be errors with UTF.
perl has error with join+UTF until 5.8.3, I don’t think that you can
find all places where Encode utf8_off must be used.

Well, for a start all modules used in RT should be reviewed for the buggy
cases… (it mean we had to be positively sure when perl breaks) and then
yes we could (with a reasonable amount of precision) cirurgically activate
or deactivate the UTF8 flag.

Where you have a strong point is if we think of the time to do it… If
we (non-English people) doesn’t do that, others won’t have neither the
need nor a great motivation to do it.

I already had a fight years ago with iso-8859-* support… now is the
“encoding world war II” with UTF8… god protect us all there’s rummors
that UTF16 will be the one… :slight_smile:

Paulo Matos

|Sys & Net Admin | Servi�o de Inform�tica |
|Faculdade de Ci�ncias e Tecnologia | Tel: +351-21-2948596 |
|Universidade Nova de Lisboa | Fax: +351-21-2948548 |
|P-2829-516 Caparica | e-Mail: pjsm@fct.unl.pt |

14

Paulo Matos wrote:

Are perl 5.8.3 with a good UTF8 support? If so, it might be worth trying
to do a perl upgrade…

Recently released perl 5.8.4 has UTF speed up also.
Stable, but still there may be errors with UTF.
perl has error with join+UTF until 5.8.3, I don’t think that you can
find all places where Encode utf8_off must be used.

Well, for a start all modules used in RT should be reviewed for the buggy
cases… (it mean we had to be positively sure when perl breaks) and then
yes we could (with a reasonable amount of precision) cirurgically activate
or deactivate the UTF8 flag.

Where you have a strong point is if we think of the time to do it… If
we (non-English people) doesn’t do that, others won’t have neither the
need nor a great motivation to do it.

I already had a fight years ago with iso-8859-* support… now is the
“encoding world war II” with UTF8… god protect us all there’s rummors
that UTF16 will be the one… :slight_smile:

I’m not going to help bugs counter growing with such changes. It’s wrong
way. RT requires 5.8.x perl now. 5.8.4 is update release for this branch
and if you think that bug in perl not in RT just update perl and check.
It’s not an upgrade.
If your distro developers don’t fix errors then take refund.

I’m going to complain on p5p ML and follow fixes. May be someday I’ll do
fix myself, now I can only send small bug report.

		Best regards. Ruslan.
15

nod I see that you’ve gone and fixed bugs for things that aren’t an
issue on 5.8.3. Libraries RT depends on are already known to be badly
broken on 5.8.0 due to bugs in perl. I’m feeling rather hesitant to
drop in explicit workarounds there. :confused: Thoughts?

RT is already pretty big and hard to wrap ones head around. I’d rather
it not gain a bunch of fixes for an outdated and buggy perl version.

seph