Use utf8 in scrip action

Maybe something I’m doing badly. I have a scrip action Foo.pm which will
work on some CF with non-ascii names.

Here is part of the code:

--------------------snip-----------------------
package RT::Action::StartNextAction;

use strict;
use warnings;
use base qw(RT::Action);
use utf8;

sub Commit {
my $self = shift;

...

my $next_phone = ...;

my $cf_clones;
my $ActionCFs = $Ticket->CustomFields;
while ( my $CustomField = $ActionCFs->Next ) {
    my $values = $Ticket->CustomFieldValues($CustomField->id);
    next unless ( $values && $values->Count );
    $cf_clones->{$CustomField->Name} = $values->First->Content;
}

if ($next_phone && $next_phone ne $cf_clones->{'Numéro de

télephone’} ) {
$cf_clones->{‘Numéro de télephone’} = $next_phone;
$cf_clones->{‘Nom’} = ‘’;
$cf_clones->{‘Adresse email’} = ‘’;
}

my $NewAction = RT::Ticket->new( $self->CurrentUser );
$NewAction->Create ( …);

foreach my $field_name (keys %{$cf_clones} ) {
$NewAction->AddCustomFieldValue( Field => $field_name, Value =>
$cf_clones->{$field_name} );
}
-----------------------snip--------------

The ->AddCustomFieldValue breaks when $field_name = ‘Numéro de
télephone’; (PostgreSQL does not see here UTF-8 in the produced select)

If I replace this foreach by:

foreach my $field_name (keys %{$cf_clones} ) {
    next if ( $field_name eq 'Numéro de télephone' );
    $NewAction->AddCustomFieldValue( Field => $field_name, Value =>

$cf_clones->{$field_name} );
}
$NewAction->AddCustomFieldValue( Field => ‘Numéro de télephone’,
Value => $cf_clones->{‘Numéro de télephone’} );

it works !!!

any perl utf8 lover can explain the right way to do this?

If of any use (a bug?) here is the PG error:

DBD::Pg::st execute failed: ERROR: invalid byte sequence for encoding
"UTF8": 0xe9726f at /usr/share/perl5/DBIx/SearchBuilder/Handle.pm line 586.
RT::Handle=HASH(0x793ee88) couldn’t execute the query ‘SELECT main.*
FROM CustomFields main JOIN ObjectCustomFields ObjectCustomFields_1 ON
( ObjectCustomFields_1.CustomField = main.id ) WHERE
(ObjectCustomFields_1.ObjectId IN (‘6’, ‘0’)) AND (main.LookupType IN
(‘RT::Queue-RT::Ticket’)) AND (LOWER(main.Name) = ‘numéro de télephone’)
ORDER BY main.Disabled ASC, ObjectCustomFields_1.ObjectId DESC,
main.SortOrder ASC, main.id ASC LIMIT 1’ at
/usr/share/perl5/DBIx/SearchBuilder/Handle.pm line
599#012#011DBIx::SearchBuilder::Handle::SimpleQuery(‘RT::Handle=HASH(0x793ee88)’,
‘SELECT main.* FROM CustomFields main JOIN ObjectCustomFields …’)
called at /usr/share/perl5/DBIx/SearchBuilder.pm line
239#012#011DBIx::SearchBuilder::_DoSearch(‘RT::CustomFields=HASH(0x7f64c68)’)
called at /srv/rt/rt/sbin/…/lib/RT/SearchBuilder.pm line
985#012#011RT::SearchBuilder::_DoSearch(‘RT::CustomFields=HASH(0x7f64c68)’)
called at /usr/share/perl5/DBIx/SearchBuilder.pm line
507#012#011DBIx::SearchBuilder::Next(‘RT::CustomFields=HASH(0x7f64c68)’)
called at /srv/rt/rt/sbin/…/lib/RT/CustomFields.pm line
393#012#011RT::CustomFields::Next(‘RT::CustomFields=HASH(0x7f64c68)’)
called at /usr/share/perl5/DBIx/SearchBuilder.pm line
568#012#011DBIx::SearchBuilder::First(‘RT::CustomFields=HASH(0x7f64c68)’) called
at /srv/rt/rt/sbin/…/lib/RT/CustomField.pm line
579#012#011RT::CustomField::LoadByName(‘RT::CustomField=HASH(0x7b35160)’, ‘Name’,
‘Num\x{e9}ro de t\x{e9}lephone’, ‘LookupType’, ‘RT::Queue-RT::Ticket’,
‘ObjectId’, 6, ‘IncludeGlobal’, 1, …) called at
/srv/rt/rt/sbin/…/lib/RT/Ticket.pm line
2986#012#011RT::ticket::LoadCustomFieldByIdentifier(‘RT::Ticket=HASH(0x7b9cf30)’,
‘Num\x{e9}ro de t\x{e9}lephone’) called at
/srv/rt/rt/sbin/…/lib/RT/Record.pm line
1966#012#011RT::Record::_AddCustomFieldValue(‘RT::Ticket=HASH(0x7b9cf30)’,
‘Field’, ‘Num\x{e9}ro de t\x{e9}lephone’, ‘Value’, 0613260760) called at
/srv/rt/rt/sbin/…/lib/RT/Record.pm line
1952#012#011RT::Record::AddCustomFieldValue('RT::Ticket=HASH

Easter-eggs Spécialiste GNU/Linux
44-46 rue de l’Ouest - 75014 Paris - France - Métro Gaité
Phone: +33 (0) 1 43 35 00 37 - Fax: +33 (0) 1 43 35 00 76
mailto:elacour@easter-eggs.com - http://www.easter-eggs.com

One way to avoid this would be using custom field id’s instead of names.

ChrisAm 15.01.2015 um 16:40 schrieb Emmanuel Lacour:

Maybe something I’m doing badly. I have a scrip action Foo.pm which will
work on some CF with non-ascii names.

Here is part of the code:

--------------------snip-----------------------
package RT::Action::StartNextAction;

use strict;
use warnings;
use base qw(RT::Action);
use utf8;

sub Commit {
my $self = shift;

...

my $next_phone = ...;

my $cf_clones;
my $ActionCFs = $Ticket->CustomFields;
while ( my $CustomField = $ActionCFs->Next ) {
    my $values = $Ticket->CustomFieldValues($CustomField->id);
    next unless ( $values && $values->Count );
    $cf_clones->{$CustomField->Name} = $values->First->Content;
}

if ($next_phone && $next_phone ne $cf_clones->{'Numéro de

télephone’} ) {
$cf_clones->{‘Numéro de télephone’} = $next_phone;
$cf_clones->{‘Nom’} = ‘’;
$cf_clones->{‘Adresse email’} = ‘’;
}

my $NewAction = RT::Ticket->new( $self->CurrentUser );
$NewAction->Create ( …);

foreach my $field_name (keys %{$cf_clones} ) {
$NewAction->AddCustomFieldValue( Field => $field_name, Value =>
$cf_clones->{$field_name} );
}
-----------------------snip--------------

One way to avoid this would be using custom field id’s instead of names.

no, sorry using id here is a bad practice imho. What will happens if I
re-init the DB and add a CF before this one ? Id change…

Easter-eggs Spécialiste GNU/Linux
44-46 rue de l’Ouest - 75014 Paris - France - Métro Gaité
Phone: +33 (0) 1 43 35 00 37 - Fax: +33 (0) 1 43 35 00 76
mailto:elacour@easter-eggs.com - http://www.easter-eggs.com

I put CF names in English (so no weird characters are present) and then I translate them via .po files.
I’ve changed a couple of files to allow RT (RTIR) to show the translated name instead of the original one.

Iñaki de Eguía Díez
Desarrollador - Entornos Abiertos
inaki.deeguia@csa.esDe: rt-devel rt-devel-bounces@lists.bestpractical.com en nombre de Emmanuel Lacour elacour@easter-eggs.com
Enviado: viernes, 16 de enero de 2015 15:42
Para: Christian Loos; rt-devel@lists.bestpractical.com
Asunto: Re: [rt-devel] use utf8 in scrip action

One way to avoid this would be using custom field id’s instead of names.

no, sorry using id here is a bad practice imho. What will happens if I
re-init the DB and add a CF before this one ? Id change…

Easter-eggs Spécialiste GNU/Linux
44-46 rue de l’Ouest - 75014 Paris - France - Métro Gaité
Phone: +33 (0) 1 43 35 00 37 - Fax: +33 (0) 1 43 35 00 76
mailto:elacour@easter-eggs.com - http://www.easter-eggs.com

En cumplimiento de la Ley Orgánica 15/1999, de 13 de diciembre, de Protección de Datos de Carácter Personal, los datos personales reflejados en este documento se incluirán en una base de datos informatizada de titularidad de Centro Regional de Servicios Avanzados, S.A. Dicha base de datos se encuentra inscrita en el Registro General de Protección de Datos conforme a lo dispuesto en la legislación vigente y cumpliendo todas las medidas de seguridad del Real Decreto 994/1999, de 11 de junio, de medidas de seguridad de los ficheros automatizados que contengan datos de carácter personal. Asimismo, el cliente dispone en todo momento de los derechos de información, acceso, rectificación, cancelación y oposición, que podrá ejercitar por escrito mediante carta dirigida al Responsable de Seguridad, en la siguiente dirección: C/ López Bravo, 1. 09001 Burgos (España).

Le 19/01/2015 08:42, I�aki de Eguia Diez a �crit :

I put CF names in English (so no weird characters are present) and then I translate them via .po files.
I’ve changed a couple of files to allow RT (RTIR) to show the translated name instead of the original one.

I already did this for some RT project, thought it’s a work around, not
the proper fix imho :wink:

@BPS: no idea on this? Could this be a bug in RT/SearchBuilder? Or just
an error on my knowledge of perl utf8?

Easter-eggs Sp�cialiste GNU/Linux
44-46 rue de l’Ouest - 75014 Paris - France - M�tro Gait�
Phone: +33 (0) 1 43 35 00 37 - Fax: +33 (0) 1 43 35 00 76
mailto:elacour@easter-eggs.com - http://www.easter-eggs.com

@BPS: no idea on this? Could this be a bug in RT/SearchBuilder? Or just an
error on my knowledge of perl utf8?

There have been a bunch of UTF8 fixes around RT/Pg/DBD::Pg, can you
provide versions of those 3 items, and it would be useful for me to
have a smaller code sample.

Can you give us a tiny initialdata file and a script that does
use RT -init;

calls AddCustomFieldValue and fails for you?

Also, are you actually producing utf8 code points in your text file,
or latin1 since they’re french accents?

-kevin