"Corrupted customfieldvalue" error in Apache error_log with migrated RT4

Guadagnino_Cristiano · March 4, 2013, 3:13pm

Hi all,
I have been experimenting with upgrading our RT 3.8.x installation to rt 4.0.x.

I’m nearly done with it, as I have a nicely working RT 4.0.8 on a test server. We have populated the database with a backup of our production system (RT 3.8.10) and we followed all upgrade steps.

While RT4 is apparently working well, I am a little concerned about errors I get in Apache’s error_log whenever I access RTFM articles.

This is an example of the messages I get:

[Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at /opt/rt4/share/html/Elements/ShowCustomFieldBinary line 48. (/opt/rt4/share/html/Elements/ShowCustomFieldBinary:48)
[Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at /opt/rt4/share/html/Elements/ShowCustomFieldBinary line 48. (/opt/rt4/share/html/Elements/ShowCustomFieldBinary:48)
[Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in concatenation (.) or string at /opt/rt4/share/html/Elements/ShowCustomFieldImage line 48. (/opt/rt4/share/html/Elements/ShowCustomFieldImage:48)
[Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at /opt/rt4/share/html/Elements/ShowCustomFieldImage line 49. (/opt/rt4/share/html/Elements/ShowCustomFieldImage:49)
[Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at /opt/rt4/share/html/Elements/ShowCustomFieldImage line 50. (/opt/rt4/share/html/Elements/ShowCustomFieldImage:50)
[Mon Mar 4 14:55:38 2013] [error]: WebRT: Corrupted customfieldvalue URL. (/opt/rt4/share/html/Elements/Error:82)

There is apparently nothing wrong in the article itself, but that “Corrupted customfieldvalue” makes me nervous. Moreover, I tried to create a brand new article in RT4 (with text, a file attachment and an image attachment) and when I access that article there’s no error in the log.

Can someone help me debug this issue?

T.I.A.

Bye
Cris

Cristiano Guadagnino
Servizio Data Administration
Bankadati S.I.
Gruppo Credito Valtellinese
Tel. +39-0342-522172

Kevin_Falcone · March 4, 2013, 4:29pm

While RT4 is apparently working well, I am a little concerned about errors I get in Apache’s
error_log whenever I access RTFM articles.

For reference, in RT 4 and greater, they’re just Articles.

This is an example of the messages I get:

 [Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at
 /opt/rt4/share/html/Elements/ShowCustomFieldBinary line 48.
 (/opt/rt4/share/html/Elements/ShowCustomFieldBinary:48)
 [Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at
 /opt/rt4/share/html/Elements/ShowCustomFieldBinary line 48.
 (/opt/rt4/share/html/Elements/ShowCustomFieldBinary:48)
 [Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in concatenation (.) or
 string at /opt/rt4/share/html/Elements/ShowCustomFieldImage line 48.
 (/opt/rt4/share/html/Elements/ShowCustomFieldImage:48)
 [Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at
 /opt/rt4/share/html/Elements/ShowCustomFieldImage line 49.
 (/opt/rt4/share/html/Elements/ShowCustomFieldImage:49)
 [Mon Mar 4 14:55:38 2013] [warning]: Use of uninitialized value in join or string at
 /opt/rt4/share/html/Elements/ShowCustomFieldImage line 50.
 (/opt/rt4/share/html/Elements/ShowCustomFieldImage:50)
 [Mon Mar 4 14:55:38 2013] [error]: WebRT: Corrupted customfieldvalue URL.
 (/opt/rt4/share/html/Elements/Error:82)

There is apparently nothing wrong in the article itself, but that “Corrupted customfieldvalue”
makes me nervous. Moreover, I tried to create a brand new article in RT4 (with text, a file
attachment and an image attachment) and when I access that article there’s no error in the
log.

This implies that the links being built don’t look like this:
/Download/CustomFieldValue/id/string they look like something else.

Figure out what kinds of links are being generated, check that your
ObjectCustomFieldValues table has been upgraded properly. Check that
rt-validator passes on your system. etc.

-kevin

Guadagnino_Cristiano · March 5, 2013, 5:06pm

Hi Kevin!On 03/04/2013 05:29 PM, Kevin Falcone wrote:

This implies that the links being built don’t look like this:
/Download/CustomFieldValue/id/string they look like something else.

Figure out what kinds of links are being generated, check that your
ObjectCustomFieldValues table has been upgraded properly. Check that
rt-validator passes on your system. etc.

-kevin

Kevin, I tried to do my homework but I’m a bit confused:

how do I check what kinds of links are generated? I tried to look into the “Links” table but links there don’t look like you said; they’re not even similar
for what I can say, the ObjectCustomFieldValues has been properly upgraded; is there something specific I can check?
rt-validator blows up with the following error:
install_driver(mysql) failed: Can’t load ‘/usr/perl5/site_perl/5.10.0/i86pc-solaris-64int/auto/DBD/mysql/mysql.so’ for module DBD::mysql: ld.so.1: perl: fatal: libmysqlclient.so.16: open failed: No such file or directory at /usr/perl5/5.10.0/lib/i86pc-solaris-64int/DynaLoader.pm line 203. at (eval 85) line 3. Compilation failed in require at (eval 85) line 3. Perhaps a required shared library or dll isn’t installed where expected at /usr/perl5/site_perl/5.10.0/DBIx/SearchBuilder/Handle.pm line 103.

Apparently it is missing libmysqlclient.so.16 and in fact there’s no such file on my system, but there’s a libmysqlclient.so.18 in /usr/mysql/lib. As a desperate try I symlinked libmysqlclient.so.16 to point to libmysqlclient.so.18, but I get the very same error.

I have to add that, on this box, I changed from Mysql to MariaDB, but I had the “Corrupted customfieldvalue” error with Mysql too.

Bye
Cris
Cristiano Guadagnino
Servizio Data Administration
Bankadati S.I.
Gruppo Credito Valtellinese
Tel. +39-0342-522172

Kevin_Falcone · March 6, 2013, 2:59pm

how do I check what kinds of links are generated? I tried to look into the “Links” table but links there don’t look like you said; they’re not even similar

Go to one of the pages showing the error, click ‘View Source’ in your
browser and look at the links it generates to the images or to the
binary downloads.

for what I can say, the ObjectCustomFieldValues has been properly upgraded; is there something specific I can check?

rt-validator blows up with the following error:
install_driver(mysql) failed: Can’t load ‘/usr/perl5/site_perl/5.10.0/i86pc-solaris-64int/auto/DBD/mysql/mysql.so’ for module DBD::mysql: ld.so.1: perl: fatal: libmysqlclient.so.16: open failed: No such file or directory at /usr/perl5/5.10.0/lib/i86pc-solaris-64int/DynaLoader.pm line 203. at (eval 85) line 3. Compilation failed in require at (eval 85) line 3. Perhaps a required shared library or dll isn’t installed where expected at /usr/perl5/site_perl/5.10.0/DBIx/SearchBuilder/Handle.pm line 103.

Apparently it is missing libmysqlclient.so.16 and in fact there’s no such file on my system, but there’s a libmysqlclient.so.18 in /usr/mysql/lib. As a desperate try I symlinked libmysqlclient.so.16 to point to libmysqlclient.so.18, but I get the very same error.

If DBD::mysql isn’t installed properly, how is RT running?

I have to add that, on this box, I changed from Mysql to MariaDB, but I had the “Corrupted customfieldvalue” error with Mysql too.

This has probably caused you all sorts of problems with the libraries.

-kevin

Guadagnino_Cristiano · March 12, 2013, 8:53am

Hi KevinOn 03/06/2013 03:59 PM, Kevin Falcone wrote:

On Tue, Mar 05, 2013 at 05:06:27PM +0000, Guadagnino Cristiano wrote:

how do I check what kinds of links are generated? I tried to look into the “Links” table but links there don’t look like you said; they’re not even similar

Go to one of the pages showing the error, click ‘View Source’ in your
browser and look at the links it generates to the images or to the
binary downloads.

Kevin,
I think I have found out where the problem lies.
On our production system (RT 3.8.10) attachments that have no value assigned show as “(no value)”. On our test system (RT 4.0.8) attachments that have no value simply show a blank space, but in the source you find a link pointing to an invalid url (e.g. “/Download/CustomFieldValue/12194/”).

I tried to look hard at the way the article custom fields are defined in both systems, but I see no difference.
Can you explain the different behavior based on what I told you?

T.I.A.
Cris

Cristiano Guadagnino
Servizio Data Administration
Bankadati S.I.
Gruppo Credito Valtellinese
Tel. +39-0342-522172

Kevin_Falcone · March 13, 2013, 5:50pm

I think I have found out where the problem lies.
On our production system (RT 3.8.10) attachments that have no value assigned show as “(no
value)”. On our test system (RT 4.0.8) attachments that have no value simply show a blank
space, but in the source you find a link pointing to an invalid url (e.g.
“/Download/CustomFieldValue/12194/”).

If you select * from ObjectCustomFieldValues where id = 12194 on your
old and new systems, what’s different?

Keep in mind you might need to examine Content carefully to see if
it’s truly empty or if it contains spaces.

-kevin

Guadagnino_Cristiano · March 15, 2013, 4:35pm

Hi Kevin!On 03/13/2013 06:50 PM, Kevin Falcone wrote:

On Tue, Mar 12, 2013 at 08:53:51AM +0000, Guadagnino Cristiano wrote:

I think I have found out where the problem lies.
On our production system (RT 3.8.10) attachments that have no value assigned show as “(no
value)”. On our test system (RT 4.0.8) attachments that have no value simply show a blank
space, but in the source you find a link pointing to an invalid url (e.g.
"/Download/CustomFieldValue/12194/").

If you select * from ObjectCustomFieldValues where id = 12194 on your
old and new systems, what’s different?

Keep in mind you might need to examine Content carefully to see if
it’s truly empty or if it contains spaces.

-kevin

Kevin, now I am REALLY confused. By looking at mysql tables as you suggested, I discovered a few things:

I told you that new articles created in RT4 did not trigger errors. That’s not true, they DO trigger the same kind of error. They do not trigger errors only if I do attach something both in the “attachments” and in the “images” CFs I defined. If they are left empty, I have errors.
I told you that our production system generated different links. That’s not true either. I was looking at very old articles, before I defined the “attachments” and “images” CFs. The newer articles generate the same kind of links.
BOTH our production system and our test system are configured to log at “warning” level.
Our production system DO NOT show errors in the log when accessing articles (both old and new), while our test system DO show errors.

I decided to look at the source lines indicated in the error log.

This is the source line from out production system (RT3):

<% $Object->Content %>

While this is the source line from our test system (RT4):

<% $Object->Content %>

As you can see they’re nearly identical, but for a slight difference in the part I highlighted.

Now I think we have two possibilities:

that slight difference in code is causing the errors (I’m not fluent enough in perl to understand it)
I have done something wrong defining the CFs in our test system. I am attaching two screenshots, could your be so kind to tell me if you see anything wrong?

Bye
Cris

Cristiano Guadagnino
Servizio Data Administration
Bankadati S.I.
Gruppo Credito Valtellinese
Tel. +39-0342-522172