Converting text/html e-mail to UTF

RT Developers
1

Hello!

It seems that SetMIMEEntityToEncoding converts text to UTF8 only for
text/plain type. At least there is the following comment here:

the entity is not text; convert at least MIME word encoded attachment filename

TODO: should we be converting ANY text/ type? autrijus?

We faced the following problem: there are e-mails that are NOT multipart,
but their content-type is ‘text/html; charset=…’.

Such e-mails are not converted to UTF8 and consequently RT sends such
tickets with ‘???’ instead of non-ascii characters.

Is it possible to fix that by converting text/* types into UTF?

Thanks!

2

We faced the following problem: there are e-mails that are NOT multipart,
but their content-type is ‘text/html; charset=…’.

Ahh, HTML emails.

Such e-mails are not converted to UTF8 and consequently RT sends such
tickets with ‘???’ instead of non-ascii characters.

You mean, when it’s sent via email to other correspondent?
What about the WebUI?

Is it possible to fix that by converting text/* types into UTF?

Maybe. But I don’t particularly like that approach since
technically, XHTML and other text/* files may contain CDATA,
which does not belong to any encoding and in any case cannot
be transparently utf8ized.

I think the proper fix is to make unutf8ized (raw) text/html
work, instead of force-utf8ize it.

Thanks,
/Autrijus/

3

We faced the following problem: there are e-mails that are NOT multipart,
but their content-type is ‘text/html; charset=…’.

Ahh, HTML emails.

Such e-mails are not converted to UTF8 and consequently RT sends such
tickets with ‘???’ instead of non-ascii characters.

You mean, when it’s sent via email to other correspondent?

Yes.

What about the WebUI?

On WebUI it is OK.

Is it possible to fix that by converting text/* types into UTF?

Maybe. But I don’t particularly like that approach since
technically, XHTML and other text/* files may contain CDATA,
which does not belong to any encoding and in any case cannot
be transparently utf8ized.

I think the proper fix is to make unutf8ized (raw) text/html
work, instead of force-utf8ize it.

I don’t quite understand the last paragraph…

4

I think the proper fix is to make unutf8ized (raw) text/html
work, instead of force-utf8ize it.

I don’t quite understand the last paragraph…

(I think
(the proper fix is (
(preserve the raw octets of text/html entities)
and
(make re-sending out those entities work correctly)
) instead of
(forcefully turn all text/html entities into utf8)
)
)

Thanks,
/Autrijus/

5

I think the proper fix is to make unutf8ized (raw) text/html
work, instead of force-utf8ize it.

I don’t quite understand the last paragraph…

(I think
(the proper fix is (
(preserve the raw octets of text/html entities)
and
(make re-sending out those entities work correctly)
) instead of
(forcefully turn all text/html entities into utf8)
)
)

Got it. That makes sense. Thanks for the explanation!

I am more than willing to test the patch…

Thanks!

6

I think the proper fix is to make unutf8ized (raw) text/html
work, instead of force-utf8ize it.

I don’t quite understand the last paragraph…

(I think
(the proper fix is (
(preserve the raw octets of text/html entities)
and
(make re-sending out those entities work correctly)
) instead of
(forcefully turn all text/html entities into utf8)
)
)

3.0.3rc1 still has this problem. Please take a look at
http://mitya.pp.ru/rt-bad-email2.dat

Thank you!