RT 3.0.3rc2

Jesse_Vincent · June 19, 2003, 7:13am

The only changes here were header parsing and generation fixes.
We backed out the encode_mimewords patch from Dirk, as that was causing
more problems than it was solving and replaced it with a more clever
encode_mimeword that avoids encoding the email address.

We also fixed header-decoding on incoming mail.

We took mitya’s failure cases and dropped them into the test suite, to
make sure it never happens again.

-j

 160    117	  Another shot at the header encoding fix
 161    118	  Mitya's failing processing of html email
 162    119	  Bumping to 3.0.3rc2

Request Tracker — Best Practical Solutions – Trouble Ticketing. Free.

Dmitry_Sivachenko · June 19, 2003, 8:11am

The only changes here were header parsing and generation fixes.
We backed out the encode_mimewords patch from Dirk, as that was causing
more problems than it was solving and replaced it with a more clever
encode_mimeword that avoids encoding the email address.

We also fixed header-decoding on incoming mail.

We took mitya’s failure cases and dropped them into the test suite, to
make sure it never happens again.

-j
 160    117	  Another shot at the header encoding fix
 161    118	  Mitya's failing processing of html email
 162    119	  Bumping to 3.0.3rc2

Jesse,
thank you very much, many problems were fixes in the past time.

I just tried 3.0.3rc2.

I found a couple of problems:

When I am sending http://mitya.pp.ru/rt-bad-email1.dat, the following
appears in the maillog:

Jun 19 12:01:01 vh postfix/local[35201]: 1ACFE16E84C: to=info@vh.demos.su, rel
ay=local, delay=1, status=bounced (Command died with status 32: “/usr/local/rt3/
bin/rt-mailgate --queue info --action correspond --url http://vh.demos.su”. Comm
and output: RT server error. The RT server which handled your email did not beh
ave as expected. It said:

System error

<td align="lef

The following e-mail http://mitya.pp.ru/rt-bad-email2.dat looks OK in the
WebUI, but when RT sends it via e-mail, all non-ascii characters in it’s body
become ‘???’ (Subject is OK).

Could you please investigate?

Thanks!

error:< /b>	Unknown encoding ‘UNKNOWN’ at /usr/local/rt3/ lib/RT/I18N.pm line 275
context:

Autrijus_Tang · June 19, 2003, 8:44am

When I am sending http://mitya.pp.ru/rt-bad-email1.dat, the following
appears in the maillog:

That mail has the extremely evil header:

and RT simply does not know how to handle this encoding.

I have committed the patch below into my branch which takes care of
this case by applying the same _GuessCharset routine to MIME words.

As for your second problem, I can’t duplicate it – but maybe you’d
like to check your @EmailOutputEncoding?

Thanks,
/Autrijus/

— //depot/RT/rt/lib/RT/I18N.pm 2003/06/19 08:03:42
+++ //depot/RT/rt/lib/RT/I18N.pm 2003/06/19 08:34:19
@@ -273,7 +273,11 @@
# now we have got a decoded subject, try to convert into
# utf-8 encoding
unless ($charset =~ m/utf-8/i) {

  Encode::from_to($enc_str, $charset, "utf8");

  eval { Encode::from_to($enc_str, $charset, "utf8") };

```
  if ($@) {
```

  $charset = _GuessCharset( $enc_str );

  Encode::from_to($enc_str, $charset, "utf8");

```
  }
```
}

$str .= $prefix . $enc_str . $trailing;
@@ -296,17 +300,33 @@
sub _FindOrGuessCharset {
my $entity = shift;
my $head = $entity->head;

my $charset;
my $fallback = ‘iso-8859-1’;
return $charset if ($charset = $head->mime_attr(“content-type.charset”));

if ($head->mime_attr(“content-type.charset”)) {
return $head->mime_attr(“content-type.charset”);
}

my $body = $entity->bodyhandle or return;

return ‘utf-8’ if Encode::is_utf8( $body->as_string );

return _GuessCharset( $head->as_string . $body->as_string );
+}
+# }}}
+# {{{ _GuessCharset
+=head2 _GuessCharset STRING
+use Encode::Guess to try to figure it out the string’s encoding.
+=cut
+sub _GuessCharset {
my $fallback = ‘iso-8859-1’;
my $charset;

if ( @RT::EmailInputEncodings and eval { require Encode::Guess; 1 } ) {
Encode::Guess->set_suspects(@RT::EmailInputEncodings);

my $decoder = Encode::Guess->guess( $head->as_string . $body->as_string );

my $decoder = Encode::Guess->guess( $_[0] );

if ( ref $decoder ) {
$charset = $decoder->name;

Dmitry_Sivachenko · June 19, 2003, 8:59am

When I am sending http://mitya.pp.ru/rt-bad-email1.dat, the following
appears in the maillog:

That mail has the extremely evil header:
From: =?UNKNOWN?Q?=C8=ED=F2=E5=F0=ED=E5=F2_=EC=E0=F0=EA=E5=F2=E8=ED=E3?=
and RT simply does not know how to handle this encoding.

Yes, you are right.
But RT should not crash regardless what it’s input is, obviously…

I have committed the patch below into my branch which takes care of
this case by applying the same _GuessCharset routine to MIME words.

Thanks, I’ll test it.

As for your second problem, I can’t duplicate it – but maybe you’d
like to check your @EmailOutputEncoding?

In my RT_SiteConfig.pm:

Set($EmailOutputEncoding , ‘koi8-r’);

Dmitry_Sivachenko · June 19, 2003, 9:18am

When I am sending http://mitya.pp.ru/rt-bad-email1.dat, the following
appears in the maillog:

That mail has the extremely evil header:
From: =?UNKNOWN?Q?=C8=ED=F2=E5=F0=ED=E5=F2_=EC=E0=F0=EA=E5=F2=E8=ED=E3?=
and RT simply does not know how to handle this encoding.

Your patch did solve the problem to rt-bad-email1.dat.

Thank you!

The problem with rt-bad-email2.dat remains.

Dirk_Pape · June 19, 2003, 1:35pm

Hey Jesse,

–Am Donnerstag, 19. Juni 2003 3:13 Uhr -0400 schrieb Jesse Vincent
jesse@bestpractical.com:

The only changes here were header parsing and generation fixes.
We backed out the encode_mimewords patch from Dirk, as that was causing
more problems than it was solving and replaced it with a more clever
encode_mimeword that avoids encoding the email address.

that seems to resolve my subject encoding issue and encouraged me to
reapply my threading patch and voila, for now (few testing only) my problem
is gone.

I beg your pardon for having introduced this bug into RT.

But this might be compensated by contruting my (hopefully) soon
rehabilitated threading patch and some more contributions we developped her
to support our workflow (e.g. a slightly changed handling of signatures in
Web-Replies, ability to toggle ticket history display order on the fly or
for the session, “Notify AdminCC only if Ticket is unowned”, public Tickets
in SelfService, some Callbacks for Ticket Actions: “stall until”,
“dispatch”

BTW: Does a policy exist how to pack contributions for RT3?

Dirk.

Jesse_Vincent · June 19, 2003, 6:01pm

Hey Jesse,

–Am Donnerstag, 19. Juni 2003 3:13 Uhr -0400 schrieb Jesse Vincent
jesse@bestpractical.com:

The only changes here were header parsing and generation fixes.
We backed out the encode_mimewords patch from Dirk, as that was causing
more problems than it was solving and replaced it with a more clever
encode_mimeword that avoids encoding the email address.

that seems to resolve my subject encoding issue and encouraged me to
reapply my threading patch and voila, for now (few testing only) my problem
is gone.

I beg your pardon for having introduced this bug into RT.

No worries. We didn’t have the right tests to catch that it would cause
a problem (and really, it’s a deficiency in MIME-Tools’ implementation).
But the bug never made it into a shipping release, so it’s no big deal

But this might be compensated by contruting my (hopefully) soon
rehabilitated threading patch and some more contributions we developped her
to support our workflow (e.g. a slightly changed handling of signatures in
Web-Replies, ability to toggle ticket history display order on the fly or
for the session, “Notify AdminCC only if Ticket is unowned”, public Tickets
in SelfService, some Callbacks for Ticket Actions: “stall until”,
“dispatch”

Neat. I’m sure people will be thrilled to see them. Pack them oen by one
and bounce them to me and I’ll drop them in /contrib

BTW: Does a policy exist how to pack contributions for RT3?

No proper policy exists yet. Off the top of my head:

I’d like to see and
<contribution_name>.README
<contribution_name>.tar.gz

The .tar.gz should contain a structure something like the following

another copy of the readme [required]
a Makefile 
etc/initialdata [optional]
html/
lib/
doc/
po/ (localization)

what else?

Autrijus indicated that he might have something clever for packaging
addons for us soon, too.

Dirk.

rt-devel mailing list
rt-devel@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-devel

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Jesse_Vincent · June 19, 2003, 6:56pm

Jesse,
thank you very much, many problems were fixes in the past time.

I just tried 3.0.3rc2.

I found a couple of problems:

When I am sending http://mitya.pp.ru/rt-bad-email1.dat, the following
appears in the maillog:

Autrijus patch is in 3.0.3rc3 (out now) and fixes this.

The following e-mail http://mitya.pp.ru/rt-bad-email2.dat looks OK in the
WebUI, but when RT sends it via e-mail, all non-ascii characters in it’s body
become ‘???’ (Subject is OK).

It should be fine in attached copies of the message (on
correspondence), but not displayed inline. As this is not a regression
from RT 3.0.2, it will not be fixed for 3.0.3. (3.0.3 is SO MUCH better
about unicode than 3.0.2 was, that I really want to get it out into the
world for folks.

Could you please investigate?

Could you open a bug by sending mail to rt-bugs@fsck.com with examples
and details attached, so we have it recorded?

Thanks.

Jesse

Thanks!

rt-devel mailing list
rt-devel@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-devel

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Dmitry_Sivachenko · June 19, 2003, 7:15pm

The following e-mail http://mitya.pp.ru/rt-bad-email2.dat looks OK in the
WebUI, but when RT sends it via e-mail, all non-ascii characters in it’s body
become ‘???’ (Subject is OK).

It should be fine in attached copies of the message (on
correspondence), but not displayed inline. As this is not a regression

Could you please explain this in more detail, please.
I don’t quite understand why it is OK.

Thanks!

Jesse_Vincent · June 19, 2003, 7:17pm

The following e-mail http://mitya.pp.ru/rt-bad-email2.dat looks OK in the
WebUI, but when RT sends it via e-mail, all non-ascii characters in it’s body
become ‘???’ (Subject is OK).

It should be fine in attached copies of the message (on
correspondence), but not displayed inline. As this is not a regression

Could you please explain this in more detail, please.
I don’t quite understand why it is OK.

So. When the text/html part is attached, it should have the correct
content type. When it’s displayed “inline” as the “Content” of the
message, it’s not automatically converted to plain text in utf8. This
is a bug. But it’s not a bug that will be fixed before 3.0.3.

Thanks!

from RT 3.0.2, it will not be fixed for 3.0.3. (3.0.3 is SO MUCH better
about unicode than 3.0.2 was, that I really want to get it out into the
world for folks.

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

Dmitry_Sivachenko · June 19, 2003, 8:17pm

The following e-mail http://mitya.pp.ru/rt-bad-email2.dat looks OK in the
WebUI, but when RT sends it via e-mail, all non-ascii characters in it’s body
become ‘???’ (Subject is OK).

It should be fine in attached copies of the message (on
correspondence), but not displayed inline. As this is not a regression

Could you please explain this in more detail, please.
I don’t quite understand why it is OK.

So. When the text/html part is attached, it should have the correct
content type. When it’s displayed “inline” as the “Content” of the
message, it’s not automatically converted to plain text in utf8. This
is a bug. But it’s not a bug that will be fixed before 3.0.3.

Ah, I see, that’s OK.
I’ll remind you about it just after 3.0.3 release, if you don’t mind.
Thanks god release should be out RSN.

Thanks you!

Jesse_Vincent · June 19, 2003, 8:59pm

Ah, I see, that’s OK.
I’ll remind you about it just after 3.0.3 release, if you don’t mind.
Thanks god release should be out RSN.

Cool, though feel free to report to the bug tracking system now, so
everyone knows it’s an issue

Thanks you!

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.