Problems with rt 3.8.7/postgresql 8.4.2 encoding

Guys:

I found myself with the following error (i live in Venezuela, so we expect
certain characters like á é í ó ú ñ in mail):

[Sun Jan 10 23:51:58 2010] [info]: <
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com> #8/129 - Scrip 3 On
Create Autoreply To Requestors
(/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:300)
[Sun Jan 10 23:51:59 2010] [info]: <
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com> sent To:
echavez@yv-consulting.com.ve (/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:331)
[Sun Jan 10 23:51:59 2010] [warning]: DBD::Pg::st execute failed: ERROR:
invalid byte sequence for encoding “UTF8”: 0xc361
[Sun Jan 10 23:51:59 2010] [warning]: RT::Handle=HASH(0xb89f54c) couldn’t
execute the query ‘INSERT INTO Attachments (Subject, Filename, ContentType,
Headers, Creator, MessageId, Parent, Created, Content, ContentEncoding,
TransactionId) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)’ at
/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line 522
[Sun Jan 10 23:51:59 2010] [crit]: Attachment insert failed: ERROR: invalid
byte sequence for encoding “UTF8”: 0xc361
[Sun Jan 10 23:51:59 2010] [crit]: Attachment insert failed: ERROR: invalid
byte sequence for encoding “UTF8”: 0xc361
[Sun Jan 10 23:51:59 2010] [info]: <
rt-3.8.7-17334-1263167518-968.8-4-0@yv-consulting.com> #8/129 - Scrip 4 On
Create Notify AdminCcs (/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:300)
[Sun Jan 10 23:51:59 2010] [info]: <
rt-3.8.7-17334-1263167518-968.8-4-0@yv-consulting.com> No recipients found.
Not sending. (/opt/rt/bin/…/lib/RT/Interface/Email.pm:342)
[Sun Jan 10 23:51:59 2010] [info]: Ticket 8 created in queue ‘Soporte
[App/Web Server]’ by echavez (/opt/rt/bin/…/lib/RT/Ticket_Overlay.pm:667)
[Sun Jan 10 23:51:59 2010] [crit]: HasRight called with no valid object
(/opt/rt/bin/…/lib/RT/Principal_Overlay.pm:322)

Somebody could pls help me?

Best Regards!!!

EC

P.D. Sorry my losey english!

Hi Luis:

Disregarding the fact that you’re selling me support in this list, i mention
you the following:

I already know that the problem is that i’ve created an autoresponse
template in spanish with ascii characters that are not recognized in unicode
format.

What i want to know is how i solve this issue in PostgreSQL, cuz in Oracle
10gR2 dosn’t happen…

Best Regards!

EC2010/1/11 Luis E. lem@itverx.com.ve

On Mon, 2010-01-11 at 23:04 +0000, Eliezer E Chávez wrote:

Guys:

I found myself with the following error (i live in Venezuela, so we
expect certain characters like á é í ó ú ñ in mail):
[…]
[Sun Jan 10 23:51:59 2010] [warning]: DBD::Pg::st execute failed:
ERROR: invalid byte sequence for encoding “UTF8”: 0xc361
[…]
Somebody could pls help me?

Hola Eliezer.

El problema que estás viendo ocurre porque en algún punto, estás
manipulando un bloque de texto que está marcado como UTF-8 pero que
contiene secuencias binarias ilegales en esa codificación. Esto puede
ocurrir por una variedad de razones – desde un problema de
configuración de los MTA que tocaron e inyectaron el mensaje, hasta una
plantilla mal hecha.

Sugiero hagas pruebas graduales para aislar el componente que tiene el
problema. Comienza usando plantillas mínimas, compuestas sólo con
ASCII. Por el punto donde se genera el error, el problema puede estar
causado por la plantilla de auto-respuesta o por la plantilla de
bienvenida.

En Itverx, mi empresa, ofrecemos varios servicios conexos que pueden
resultarte de interés:

  • Planes de capacitación y soporte para Request Tracker / Grok®
    Ticketing Solution (nuestra solución de ticketing basada en RT).

  • Sistemas Grok® TS hospedados.

Hay más información en http://itverx.com.ve/grok/ticket/

Cordiales saludos

-lem

Hi Luis:

Disregarding the fact that you’re selling me support in this list,
[…]

Sorry, you’re wrong. I answered in a private message, in spanish BTW,
because you mentioned in your message that you’re from Venezuela and
that your English had room for improvement.

In that private message, I pointed some things to look at and yes, let
you know that we offer professional services that might be of interest.
Again, this was outside the regular list traffic.

As an exercise, think about who violated etiquette by posting a private
message on a public forum now.

I already know that the problem is that i’ve created an autoresponse
template in spanish with ascii characters that are not recognized in
unicode format.

Good. That paragraph does not mean what you think it does. A text
composed entirely of ASCII characters, is valid UTF-8 (UTF-8 and Unicode
are not synonyms, ASCII is just one of many possible encodings).

Your problem seems to be related to a chain of encoding/decoding
operations in which at least one of them is interpreting a string
assuming the wrong encoding.

What i want to know is how i solve this issue in PostgreSQL, cuz in
Oracle 10gR2 dosn’t happen…

Your problem is not with PostgreSQL, which is doing the right thing.
You’re feeding it a string of badly encoded UTF-8. Check your
environment and make sure you’re doing everything with UTF-8 so that the
encodings are consistent. IOW, follow the advice you Cc-ed to the list.

Best regards.

-lem

Hi Luis:

Disregarding the fact that you’re selling me support in this list, i
mention you the following:

I think that was an off list message - at least I didn’t get it.

I already know that the problem is that i’ve created an autoresponse
template in spanish with ascii characters that are not recognized in
unicode format.

What i want to know is how i solve this issue in PostgreSQL, cuz in
Oracle 10gR2 dosn’t happen…

You have to have created the database with a utf8 encoding (what does
\l from a psql terminal show)? Next step would be making sure that the
data you’re stuffing into the DB is utf-8. If you dump an email
verbatim the sender may have set the encoding so something else. I
don’t know enough about RT to say if it’ll handle this translation for
you.

patrick

Sorry for the misunderstanding, but i’m a support consultant too, so, i
dislike others selling me… :slight_smile:

Ok, as a clarification, and in spanish:

Creé una plantilla de autorespuesta en español, pero cuando intento crear un
nuevo ticket y RT intenta guardar el mensaje en la base de datos se queja de
los caracteres latinos (á, é, ñ, etc…)

Cómo hago para corregir eso, defino la base de datos en ISO-8859-1 (LATIN1)
ó como hago para decirle a RT como enviar la codificación a PostgreSQL.

Saludos y mis disculpas de nuevo.

EC2010/1/11 Luis E. lem@itverx.com.ve

On Mon, 2010-01-11 at 07:24 -0430, Eliezer E Chávez wrote:

Hi Luis:

Disregarding the fact that you’re selling me support in this list,
[…]

Sorry, you’re wrong. I answered in a private message, in spanish BTW,
because you mentioned in your message that you’re from Venezuela and
that your English had room for improvement.

In that private message, I pointed some things to look at and yes, let
you know that we offer professional services that might be of interest.
Again, this was outside the regular list traffic.

As an exercise, think about who violated etiquette by posting a private
message on a public forum now.

I already know that the problem is that i’ve created an autoresponse
template in spanish with ascii characters that are not recognized in
unicode format.

Good. That paragraph does not mean what you think it does. A text
composed entirely of ASCII characters, is valid UTF-8 (UTF-8 and Unicode
are not synonyms, ASCII is just one of many possible encodings).

Your problem seems to be related to a chain of encoding/decoding
operations in which at least one of them is interpreting a string
assuming the wrong encoding.

What i want to know is how i solve this issue in PostgreSQL, cuz in
Oracle 10gR2 dosn’t happen…

Your problem is not with PostgreSQL, which is doing the right thing.
You’re feeding it a string of badly encoded UTF-8. Check your
environment and make sure you’re doing everything with UTF-8 so that the
encodings are consistent. IOW, follow the advice you Cc-ed to the list.

Best regards.

-lem

This is the full trace…

[Sun Jan 10 23:51:58 2010] [info]: <
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com> #8/129 - Scrip 3 On
Create Autoreply To Requestors
(/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:300)
[Sun Jan 10 23:51:59 2010] [info]: <
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com> sent To:
echavez@yv-consulting.com.ve (/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:331)
[Sun Jan 10 23:51:59 2010] [warning]: DBD::Pg::st execute failed: ERROR:
invalid byte sequence for encoding “UTF8”: 0xc361
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by “client_encoding”.
at /usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line 509.
(/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm:509)
[Sun Jan 10 23:51:59 2010] [warning]: RT::Handle=HASH(0xb89f54c) couldn’t
execute the query ‘INSERT INTO Attachments (Subject, Filename, ContentType,
Headers, Creator, MessageId, Parent, Created, Content, ContentEncoding,
TransactionId) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)’ at
/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line 522

DBIx::SearchBuilder::Handle::SimpleQuery(‘RT::Handle=HASH(0xb89f54c)’,
‘INSERT INTO Attachments (Subject, Filename, ContentType, Head…’,
‘Respuesta Autom\x{c3}\x{a1}tica: Prueba’, ‘undef’, ‘text/plain’,
'Content-Type: text/plain; charset=“utf-8”
/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line 357
DBIx::SearchBuilder::Handle::Insert(‘RT::Handle=HASH(0xb89f54c)’,
‘Attachments’, ‘Subject’, ‘Respuesta Autom\x{c3}\x{a1}tica: Prueba’,
‘Filename’, ‘undef’, ‘ContentType’, ‘text/plain’, ‘Headers’, …) called at
/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle/Pg.pm line 66

DBIx::SearchBuilder::Handle::Pg::Insert(‘RT::Handle=HASH(0xb89f54c)’,
‘Attachments’, ‘Subject’, ‘Respuesta Autom\x{c3}\x{a1}tica: Prueba’,
‘ContentType’, ‘text/plain’, ‘Filename’, ‘undef’, ‘Headers’, …) called at
/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Record.pm line 1293

DBIx::SearchBuilder::Record::Create(‘RT::Attachment=HASH(0xc8c6d5c)’,
‘Subject’, ‘Respuesta Autom\x{c3}\x{a1}tica: Prueba’, ‘Filename’, ‘undef’,
‘ContentType’, ‘text/plain’, ‘Headers’, 'Content-Type: text/plain;
charset="utf-8"
289
RT::Record::Create(‘RT::Attachment=HASH(0xc8c6d5c)’,
‘TransactionId’, 130, ‘ContentType’, ‘text/plain’, ‘ContentEncoding’,
‘none’, ‘Parent’, 35, …) called at
/opt/rt/bin/…/lib/RT/Attachment_Overlay.pm line 178
RT::Attachment::Create(‘RT::Attachment=HASH(0xc8c6d5c)’,
‘TransactionId’, 130, ‘Parent’, 35, ‘Attachment’,
‘MIME::Entity=HASH(0xc8bbaa8)’) called at
/opt/rt/bin/…/lib/RT/Attachment_Overlay.pm line 158
RT::Attachment::Create(‘RT::Attachment=HASH(0xc8b5a34)’,
‘TransactionId’, 130, ‘Attachment’, ‘MIME::Entity=HASH(0xc8b9238)’) called
at /opt/rt/bin/…/lib/RT/Transaction_Overlay.pm line 514
RT::Transaction::_Attach(‘RT::Transaction=HASH(0xc8bba30)’,
‘MIME::Entity=HASH(0xc8b9238)’) called at
/opt/rt/bin/…/lib/RT/Transaction_Overlay.pm line 154
RT::Transaction::Create(‘RT::Transaction=HASH(0xc8bba30)’, ‘Ticket’,
8, ‘Type’, ‘EmailRecord’, ‘Data’, ‘<
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com>’, ‘MIMEObj’,
‘MIME::Entity=HASH(0xc8b9238)’, …) called at
/opt/rt/bin/…/lib/RT/Action/SendEmail.pm line 543

RT::Action::SendEmail::RecordOutgoingMailTransaction(‘RT::Action::Autoreply=HASH(0xc8b0fe4)’,
‘MIME::Entity=HASH(0xc8b9238)’) called at
/opt/rt/bin/…/lib/RT/Action/SendEmail.pm line 138

RT::Action::SendEmail::Commit(‘RT::Action::Autoreply=HASH(0xc8b0fe4)’)
called at /opt/rt/bin/…/lib/RT/ScripAction_Overlay.pm line 238
RT::ScripAction::Commit(‘RT::ScripAction=HASH(0xc89f030)’) called at
/opt/rt/bin/…/lib/RT/Scrip_Overlay.pm line 464
eval {…} called at /opt/rt/bin/…/lib/RT/Scrip_Overlay.pm line 463
RT::Scrip::Commit(‘RT::Scrip=HASH(0xc89e658)’, ‘TicketObj’,
‘RT::Ticket=HASH(0xc89e8b0)’, ‘TransactionObj’,
‘RT::Transaction=HASH(0xc89e820)’) called at
/opt/rt/bin/…/lib/RT/Scrips_Overlay.pm line 196
RT::Scrips::Commit(‘RT::Scrips=HASH(0xc88cb28)’) called at
/opt/rt/bin/…/lib/RT/Transaction_Overlay.pm line 188
RT::Transaction::Create(‘RT::Transaction=HASH(0xc86a624)’,
‘ObjectId’, 8, ‘ObjectType’, ‘RT::Ticket’, ‘TimeTaken’, 0, ‘Type’, ‘Create’,
…) called at /opt/rt/bin/…/lib/RT/Record.pm line 1457
RT::Record::_NewTransaction(‘RT::Ticket=HASH(0xbcbb388)’, ‘Type’,
‘Create’, ‘TimeTaken’, 0, ‘MIMEObj’, ‘MIME::Entity=HASH(0xbcb8e80)’,
‘CommitScrips’, 1, …) called at /opt/rt/bin/…/lib/RT/Ticket_Overlay.pm
line 656
RT::ticket::Create(‘RT::Ticket=HASH(0xbcbb388)’, ‘Queue’, 4,
‘Subject’, ‘Prueba’, ‘Requestor’, ‘ARRAY(0xa8c65dc)’, ‘Cc’,
‘ARRAY(0xa8c65c4)’, …) called at /opt/rt/bin/…/lib/RT/Interface/Email.pm
line 1444
RT::Interface::email::Gateway(‘HASH(0xc532500)’) called at
/opt/rt/share/html/REST/1.0/NoAuth/mail-gateway line 61
HTML::Mason::Commands::ANON(‘SessionType’, ‘REST’, ‘action’,
‘correspond’, ‘queue’, ‘Soporte [App/Web Server]’, ‘message’, ‘Delivered-To:
soporte.as@yv-consulting.com.ve\x{a}Received: from …’) called at
/usr/lib/perl5/site_perl/5.8.8/HTML/Mason/Component.pm line 135

HTML::Mason::Component::run(‘HTML::Mason::Component::FileBased=HASH(0xbcb7b6c)’,
‘SessionType’, ‘REST’, ‘action’, ‘correspond’, ‘queue’, ‘Soporte [App/Web
Server]’, ‘message’, ‘Delivered-To:
soporte.as@yv-consulting.com.ve\x{a}Received:
from …’, …) called at
/usr/lib/perl5/site_perl/5.8.8/HTML/Mason/Request.pm line 1279
eval {…} called at
/usr/lib/perl5/site_perl/5.8.8/HTML/Mason/Request.pm line 1274
HTML::Mason::Request::comp(‘undef’, ‘undef’, ‘undef’, ‘SessionType’,
‘REST’, ‘action’, ‘correspond’, ‘queue’, ‘Soporte [App/Web Server]’, …)
called at /usr/lib/perl5/site_perl/5.8.8/HTML/Mason/Request.pm line 473
eval {…} called at
/usr/lib/perl5/site_perl/5.8.8/HTML/Mason/Request.pm line 473
eval {…} called at
/usr/lib/perl5/site_perl/5.8.8/HTML/Mason/Request.pm line 425

HTML::Mason::Request::exec(‘RT::Interface::Web::Request=HASH(0xbe7efe8)’)
called at /usr/lib/perl5/site_perl/5.8.8/HTML/Mason/ApacheHandler.pm line
168

HTML::Mason::Request::ApacheHandler::exec(‘RT::Interface::Web::Request=HASH(0xbe7efe8)’)
called at /usr/lib/perl5/site_perl/5.8.8/HTML/Mason/ApacheHandler.pm line
825

HTML::Mason::ApacheHandler::handle_request(‘HTML::Mason::ApacheHandler=HASH(0xc510d20)’,
‘Apache2::RequestRec=SCALAR(0xbcc0ff8)’) called at /opt/rt/bin/webmux.plline 166
eval {…} called at /opt/rt/bin/webmux.pl line 166
RT::Mason::handler(‘Apache2::RequestRec=SCALAR(0xbcc0ff8)’) called
at -e line 0
eval {…} called at -e line 0 (/usr/lib/perl5/5.8.8/Carp.pm:272)
[Sun Jan 10 23:51:59 2010] [crit]: Attachment insert failed: ERROR: invalid
byte sequence for encoding “UTF8”: 0xc361
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by “client_encoding”.
(/opt/rt/bin/…/lib/RT/Attachment_Overlay.pm:191)
[Sun Jan 10 23:51:59 2010] [crit]: Attachment insert failed: ERROR: invalid
byte sequence for encoding “UTF8”: 0xc361
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by “client_encoding”.
(/opt/rt/bin/…/lib/RT/Attachment_Overlay.pm:164)
[Sun Jan 10 23:51:59 2010] [info]: <
rt-3.8.7-17334-1263167518-968.8-4-0@yv-consulting.com> #8/129 - Scrip 4 On
Create Notify AdminCcs (/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:300)
[Sun Jan 10 23:51:59 2010] [info]: <
rt-3.8.7-17334-1263167518-968.8-4-0@yv-consulting.com> No recipients found.
Not sending. (/opt/rt/bin/…/lib/RT/Interface/Email.pm:342)
[Sun Jan 10 23:51:59 2010] [info]: Ticket 8 created in queue ‘Soporte
[App/Web Server]’ by echavez (/opt/rt/bin/…/lib/RT/Ticket_Overlay.pm:667)
[Sun Jan 10 23:51:59 2010] [crit]: HasRight called with no valid object
(/opt/rt/bin/…/lib/RT/Principal_Overlay.pm:322)

Regards,

EC2010/1/11 Eliezer E Chávez echavez@yv-consulting.com.ve

Sorry for the misunderstanding, but i’m a support consultant too, so, i
dislike others selling me… :slight_smile:

Ok, as a clarification, and in spanish:

Creé una plantilla de autorespuesta en español, pero cuando intento crear
un nuevo ticket y RT intenta guardar el mensaje en la base de datos se queja
de los caracteres latinos (á, é, ñ, etc…)

Cómo hago para corregir eso, defino la base de datos en ISO-8859-1 (LATIN1)
ó como hago para decirle a RT como enviar la codificación a PostgreSQL.

Saludos y mis disculpas de nuevo.

EC

2010/1/11 Luis E. lem@itverx.com.ve

On Mon, 2010-01-11 at 07:24 -0430, Eliezer E Chávez wrote:

Hi Luis:

Disregarding the fact that you’re selling me support in this list,
[…]

Sorry, you’re wrong. I answered in a private message, in spanish BTW,
because you mentioned in your message that you’re from Venezuela and
that your English had room for improvement.

In that private message, I pointed some things to look at and yes, let
you know that we offer professional services that might be of interest.
Again, this was outside the regular list traffic.

As an exercise, think about who violated etiquette by posting a private
message on a public forum now.

I already know that the problem is that i’ve created an autoresponse
template in spanish with ascii characters that are not recognized in
unicode format.

Good. That paragraph does not mean what you think it does. A text
composed entirely of ASCII characters, is valid UTF-8 (UTF-8 and Unicode
are not synonyms, ASCII is just one of many possible encodings).

Your problem seems to be related to a chain of encoding/decoding
operations in which at least one of them is interpreting a string
assuming the wrong encoding.

What i want to know is how i solve this issue in PostgreSQL, cuz in
Oracle 10gR2 dosn’t happen…

Your problem is not with PostgreSQL, which is doing the right thing.
You’re feeding it a string of badly encoded UTF-8. Check your
environment and make sure you’re doing everything with UTF-8 so that the
encodings are consistent. IOW, follow the advice you Cc-ed to the list.

Best regards.

-lem

Have a look at a thread from last month in rt-devel mailing list:
http://www.gossamer-threads.com/lists/rt/devel/90355

Sounds similar, though different.

EynatFrom: Eliezer E Chávez [mailto:eliezer.chavez@gmail.com]
Sent: Monday, 11 January 2010 5:34 AM
To: RT Users
Subject: [rt-users] Problems with rt 3.8.7/postgresql 8.4.2 encoding

Guys:

I found myself with the following error (i live in Venezuela, so we expect
certain characters like á é í ó ú ñ in mail):

[Sun Jan 10 23:51:58 2010] [info]:
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com #8/129 - Scrip 3 On
Create Autoreply To Requestors
(/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:300)
[Sun Jan 10 23:51:59 2010] [info]:
rt-3.8.7-17334-1263167518-937.8-3-0@yv-consulting.com sent To:
echavez@yv-consulting.com.ve (/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:331)
[Sun Jan 10 23:51:59 2010] [warning]: DBD::Pg::st execute failed: ERROR:
invalid byte sequence for encoding “UTF8”: 0xc361
[Sun Jan 10 23:51:59 2010] [warning]: RT::Handle=HASH(0xb89f54c) couldn’t
execute the query ‘INSERT INTO Attachments (Subject, Filename, ContentType,
Headers, Creator, MessageId, Parent, Created, Content, ContentEncoding,
TransactionId) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)’ at
/usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line 522
[Sun Jan 10 23:51:59 2010] [crit]: Attachment insert failed: ERROR: invalid
byte sequence for encoding “UTF8”: 0xc361
[Sun Jan 10 23:51:59 2010] [crit]: Attachment insert failed: ERROR: invalid
byte sequence for encoding “UTF8”: 0xc361
[Sun Jan 10 23:51:59 2010] [info]:
rt-3.8.7-17334-1263167518-968.8-4-0@yv-consulting.com #8/129 - Scrip 4 On
Create Notify AdminCcs (/opt/rt/bin/…/lib/RT/Action/SendEmail.pm:300)
[Sun Jan 10 23:51:59 2010] [info]:
rt-3.8.7-17334-1263167518-968.8-4-0@yv-consulting.com No recipients found.
Not sending. (/opt/rt/bin/…/lib/RT/Interface/Email.pm:342)
[Sun Jan 10 23:51:59 2010] [info]: Ticket 8 created in queue ‘Soporte
[App/Web Server]’ by echavez (/opt/rt/bin/…/lib/RT/Ticket_Overlay.pm:667)
[Sun Jan 10 23:51:59 2010] [crit]: HasRight called with no valid object
(/opt/rt/bin/…/lib/RT/Principal_Overlay.pm:322)

Somebody could pls help me?

Best Regards!!!

EC

P.D. Sorry my losey english!

Sorry for the misunderstanding, but i’m a support consultant too, so,
i dislike others selling me… :slight_smile:

Ok, as a clarification, and in spanish:

I believe this list requests messages to be written in english.

Creé una plantilla de autorespuesta en español, pero cuando intento
crear un nuevo ticket y RT intenta guardar el mensaje en la base de
datos se queja de los caracteres latinos (á, é, ñ, etc…)

Translation and edition by me, for the non spanish reading readers:

"I created an autoresponse template in spanish. If I try to create a new
ticket, when RT tries to store the message in the database, it complains
on the latin characters (a acute, e acute, n tilde, etc.).

How do I fix that? Shall I define the database as ISO-8859-1 (LATIN1)?
How do I get RT to tell PostgreSQL the encoding?

Regards and apologies."

Looking at the couple of traces you’ve sent, you have a typical
re-encoding problem others have hinted about. Since 0xc361 starts with
character 0xC3 (Ã) and that one is the first one in the two-byte
sequences for many UTF-8 encodings, finding out that 0xC361 was meant to
be an ‘á’ (a acute) is trivial. Therefore, it’s clear to me that
something went from proper UTF-8 into ISO-8559-X but then was
incorrectly interpreted back as UTF-8. And by incorrectly I don’t mean
the software made a mistake, but that it’s improperly configured
(encoding detection isn’t automatic, and it’s hard even for most alert
humans).

You should verify that you’re working with UTF-8 end-to-end. This means
checking that Apache2 is serving UTF-8 and accepting UTF-8, and also
keep PostgreSQL using UTF-8 as database encoding. It also means that the
data YOU input is also in UTF-8, meaning your browser has a sane
configuration and the operating system it runs on can work with UTF-8.

I’m guessing you wrote the template using a browser that was working on
UTF-8, but Apache was expecting ISO-8859 either because the browser said
it was going to provide ISO-8859 or because Apache has a (wrongly)
forced default charset. That caused the properly formed ‘á’ (one char,
two bytes, UTF-8) coming from the browser to be transformed by Apache
into ‘Ãa’ (two chars, two bytes, ISO-8859), and then when that was fed
to PostgreSQL turned out as an error because it’s not proper UTF-8.

BTW, you mentioned that Oracle did not complain. It doesn’t complain
because it’s dangerously permissive. It just gobbled whatever you gave
it without checking. Been there, done that, it’s very very sad.

So, don’t change Pg to ISO-8859-1. Make sure the browser, the OS it’s
running on and Apache are working in UTF-8 all the way.
Ernesto Hernández-Novich - Linux 2.6.28 i686 - Unix: Live free or die!
Geek by nature, Linux by choice, Debian of course.
If you can’t aptitude it, it isn’t useful or doesn’t exist.
GPG Key Fingerprint = 438C 49A2 A8C7 E7D7 1500 C507 96D6 A3D6 2F4C 85E3

Ernesto:

Thanks for a really good clarification, and sorry for my spanish text
(hehe), now i’m sending my config so you can check if i have something
wrong…

1.- Apache config:
<VirtualHost *:80>
ServerName rt.yv-consulting.com
ErrorLog …
CustomLog …
DocumentRoot …

AddDefaultCharset UTF-8
PerlRequire .../webmux.pl

<Location ^/NoAuth/*>
    SetHandler default-handler
</Location>

<Directory ...>
    Order allow,deny
    Allow from all

    SetHandler perl-script
    PerlResponseHandler RT::Mason
</Directory>

2.- PostgreSQL DB
List of databases
Name | Owner | Encoding | Collation | Ctype | Access
privileges

rt | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |

If you need any other file, pls let me know and THANK YOU ALL FOR YOUR
HELP!!!

Best Regards,

EC2010/1/11 Ernesto Hernández-Novich emhnemhn@gmail.com

On Mon, 2010-01-11 at 08:07 -0430, Eliezer E Chávez wrote:

Sorry for the misunderstanding, but i’m a support consultant too, so,
i dislike others selling me… :slight_smile:

Ok, as a clarification, and in spanish:

I believe this list requests messages to be written in english.

Creé una plantilla de autorespuesta en español, pero cuando intento
crear un nuevo ticket y RT intenta guardar el mensaje en la base de
datos se queja de los caracteres latinos (á, é, ñ, etc…)

Translation and edition by me, for the non spanish reading readers:

"I created an autoresponse template in spanish. If I try to create a new
ticket, when RT tries to store the message in the database, it complains
on the latin characters (a acute, e acute, n tilde, etc.).

How do I fix that? Shall I define the database as ISO-8859-1 (LATIN1)?
How do I get RT to tell PostgreSQL the encoding?

Regards and apologies."

Looking at the couple of traces you’ve sent, you have a typical
re-encoding problem others have hinted about. Since 0xc361 starts with
character 0xC3 (Ã) and that one is the first one in the two-byte
sequences for many UTF-8 encodings, finding out that 0xC361 was meant to
be an ‘á’ (a acute) is trivial. Therefore, it’s clear to me that
something went from proper UTF-8 into ISO-8559-X but then was
incorrectly interpreted back as UTF-8. And by incorrectly I don’t mean
the software made a mistake, but that it’s improperly configured
(encoding detection isn’t automatic, and it’s hard even for most alert
humans).

You should verify that you’re working with UTF-8 end-to-end. This means
checking that Apache2 is serving UTF-8 and accepting UTF-8, and also
keep PostgreSQL using UTF-8 as database encoding. It also means that the
data YOU input is also in UTF-8, meaning your browser has a sane
configuration and the operating system it runs on can work with UTF-8.

I’m guessing you wrote the template using a browser that was working on
UTF-8, but Apache was expecting ISO-8859 either because the browser said
it was going to provide ISO-8859 or because Apache has a (wrongly)
forced default charset. That caused the properly formed ‘á’ (one char,
two bytes, UTF-8) coming from the browser to be transformed by Apache
into ‘Ãa’ (two chars, two bytes, ISO-8859), and then when that was fed
to PostgreSQL turned out as an error because it’s not proper UTF-8.

BTW, you mentioned that Oracle did not complain. It doesn’t complain
because it’s dangerously permissive. It just gobbled whatever you gave
it without checking. Been there, done that, it’s very very sad.

So, don’t change Pg to ISO-8859-1. Make sure the browser, the OS it’s
running on and Apache are working in UTF-8 all the way.

Ernesto Hernández-Novich - Linux 2.6.28 i686 - Unix: Live free or die!
Geek by nature, Linux by choice, Debian of course.
If you can’t aptitude it, it isn’t useful or doesn’t exist.
GPG Key Fingerprint = 438C 49A2 A8C7 E7D7 1500 C507 96D6 A3D6 2F4C 85E3


http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Hello, Eliezer.

From error message it’s not clear which column has problems, even
stack trace doesn’t help. Your apache config looks good, but it’s
unrelated in this case. Pg’s configuration is not bad, but probably it
would be better for you to change ctype/collation to something more
spanish for better sorting and case insensetive searches (quite
important for RT).

Regarding original error. I think it’s related to the following ticket:
http://rt3.fsck.com/Ticket/Display.html?id=142142010/1/11 Eliezer E Chávez eliezer.chavez@gmail.com:

Ernesto:

Thanks for a really good clarification, and sorry for my spanish text
(hehe), now i’m sending my config so you can check if i have something
wrong…

1.- Apache config:
<VirtualHost *:80>
ServerName rt.yv-consulting.com
ErrorLog …
CustomLog …
DocumentRoot …

AddDefaultCharset UTF-8
PerlRequire .../webmux.pl

<Location ^/NoAuth/*>
    SetHandler default-handler
</Location>

<Directory ...>
    Order allow,deny
    Allow from all

    SetHandler perl-script
    PerlResponseHandler RT::Mason
</Directory>

2.- PostgreSQL DB
List of databases
Name | Owner | Encoding | Collation | Ctype | Access
privileges
-----------±---------±---------±------------±------------±----------------------

rt | postgres | UTF8 | en_US.UTF-8 | en_US.UTF-8 |

If you need any other file, pls let me know and THANK YOU ALL FOR YOUR
HELP!!!

Best Regards,

EC

2010/1/11 Ernesto Hernández-Novich emhnemhn@gmail.com

On Mon, 2010-01-11 at 08:07 -0430, Eliezer E Chávez wrote:

Sorry for the misunderstanding, but i’m a support consultant too, so,
i dislike others selling me… :slight_smile:

Ok, as a clarification, and in spanish:

I believe this list requests messages to be written in english.

Creé una plantilla de autorespuesta en español, pero cuando intento
crear un nuevo ticket y RT intenta guardar el mensaje en la base de
datos se queja de los caracteres latinos (á, é, ñ, etc…)

Translation and edition by me, for the non spanish reading readers:

"I created an autoresponse template in spanish. If I try to create a new
ticket, when RT tries to store the message in the database, it complains
on the latin characters (a acute, e acute, n tilde, etc.).

How do I fix that? Shall I define the database as ISO-8859-1 (LATIN1)?
How do I get RT to tell PostgreSQL the encoding?

Regards and apologies."

Looking at the couple of traces you’ve sent, you have a typical
re-encoding problem others have hinted about. Since 0xc361 starts with
character 0xC3 (Ã) and that one is the first one in the two-byte
sequences for many UTF-8 encodings, finding out that 0xC361 was meant to
be an ‘á’ (a acute) is trivial. Therefore, it’s clear to me that
something went from proper UTF-8 into ISO-8559-X but then was
incorrectly interpreted back as UTF-8. And by incorrectly I don’t mean
the software made a mistake, but that it’s improperly configured
(encoding detection isn’t automatic, and it’s hard even for most alert
humans).

You should verify that you’re working with UTF-8 end-to-end. This means
checking that Apache2 is serving UTF-8 and accepting UTF-8, and also
keep PostgreSQL using UTF-8 as database encoding. It also means that the
data YOU input is also in UTF-8, meaning your browser has a sane
configuration and the operating system it runs on can work with UTF-8.

I’m guessing you wrote the template using a browser that was working on
UTF-8, but Apache was expecting ISO-8859 either because the browser said
it was going to provide ISO-8859 or because Apache has a (wrongly)
forced default charset. That caused the properly formed ‘á’ (one char,
two bytes, UTF-8) coming from the browser to be transformed by Apache
into ‘Ãa’ (two chars, two bytes, ISO-8859), and then when that was fed
to PostgreSQL turned out as an error because it’s not proper UTF-8.

BTW, you mentioned that Oracle did not complain. It doesn’t complain
because it’s dangerously permissive. It just gobbled whatever you gave
it without checking. Been there, done that, it’s very very sad.

So, don’t change Pg to ISO-8859-1. Make sure the browser, the OS it’s
running on and Apache are working in UTF-8 all the way.

Ernesto Hernández-Novich - Linux 2.6.28 i686 - Unix: Live free or die!
Geek by nature, Linux by choice, Debian of course.
If you can’t aptitude it, it isn’t useful or doesn’t exist.
GPG Key Fingerprint = 438C 49A2 A8C7 E7D7 1500 C507 96D6 A3D6 2F4C 85E3


http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com


http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Best regards, Ruslan.