UTF8 email and webmail providers

Hi all,

We’re in the midst of upgrading from 3.2 to 3.8 (yay!) and have run
into an interesting issue. Back in mid 2004 when RT was first
introduced at the company, it was discovered there were issues with
quite a few of the large webmail providers in properly processing &
displaying utf8-encoded emails. Because of those issues we created
“the scrubber” which, when used via $SendmailPath as the email
handler, processed outgoing emails and converted them from utf8 to the
native/most common character set of the language. It’s still doing
it’s job to this day.

Fast forward to 2009 and our pending upgrading. I really want to get
rid of this post-processing of email and let everything go out as
UTF-8. I’m trying to gather evidence to bolster my case to get rid of
our “scrubber”. Does anyone know of current issues with webmail
providers not properly handling utf-8? IIRC, some of the big
problems were with Hebrew and Japanese. I’ve done some googling, but
the most recent result was yahoo webmail from 2006. Any evidence,
anecdotal or otherwise, would be most appreciated.

Thanks,
Drew
Drew Taylor * Web development & consulting
Email: drew@drewtaylor.com * Site implementation & hosting
Web : www.drewtaylor.com * perl/mod_perl/DBI/mysql/postgres