RT Down & DBIx SearchBuilder errors

My RT installation is very unhappy as of this morning. Most data in
tickets shows, but correspondence and comments are not there. So I can
see all the tickets, but if I click on one, I see all the actions, but
none of the data.

The logs are filling up with errors like this:

Jun 27 10:42:58 rt RT: RT::Handle=HASH(0xa89f334) couldn’t execute the
query ‘SELECT main., MAX(ObjectCustomFields_1.ObjectId)
__special_sort_0, MIN(ObjectCustomFields_1.SortOrder) __special_sort_1
FROM CustomFields main JOIN ObjectCustomFields ObjectCustomFields_1
ON ( ObjectCustomFields_1.CustomField = main.id ) WHERE
(main.Disabled = ‘0’) AND (main.Name = NULL) AND
(ObjectCustomFields_1.ObjectId = ‘11’ OR ObjectCustomFields_1.ObjectId
= ‘0’) AND (main.LookupType = ‘RT::Queue-RT::Ticket-RT::Transaction’)
GROUP BY main.id ORDER BY __special_sort_0 DESC, __special_sort_1 ASC
’ at /usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line
522
DBIx::SearchBuilder::Handle::SimpleQuery(‘RT::Handle=HASH(0xa89f334)’,
'SELECT main.
, MAX(ObjectCustomFields_1.ObjectId) __special_s…’)
called at /usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder.pm line
236
DBIx::SearchBuilder::_DoSearch(‘RT::CustomFields=HASH(0xafa35a4)’)
called at /opt/rt3/bin/…/lib/RT/CustomFields_Overlay.pm line 168
RT::CustomFie

Any idea what this might be? In the past when RT lost contact with the
MySQL database (which is on a different host), I’d have to restart
Apache to get the two talking again. Restarting Apache in this case
hasn’t helped.

I’m trying to determine if this is a problem with database access, or
something wrong with my RT host.

Nobody has made any changes to anything that would have caused this.

Just replying to my own message in case anyone ever runs into this
problem.

I also noticed a line like this in /var/log/messages

Jun 27 12:58:09 rt RT: DBD::mysql::st execute failed: Got error 28
from storage engine at /usr/lib/perl5/site_perl/5.8.8/DBIx/
SearchBuilder/Handle.pm line 509. (/usr/lib/perl5/site_perl/5.8.8/DBIx/
SearchBuilder/Handle.pm:509)

It turns out “Got error 28 from storage engine” is a disk space error.
I asked our DBA to check the MySQL server, and he said it was
operating normally and the partition with MySQL data was not full. I
asked about the other partitions, and he noticed /tmp was full.
Freeing up space on /tmp made MySQL happy again.On Jun 27, 2009, at 10:48 AM, John Arends wrote:

My RT installation is very unhappy as of this morning. Most data in
tickets shows, but correspondence and comments are not there. So I can
see all the tickets, but if I click on one, I see all the actions, but
none of the data.

The logs are filling up with errors like this:

Jun 27 10:42:58 rt RT: RT::Handle=HASH(0xa89f334) couldn’t execute the
query ‘SELECT main., MAX(ObjectCustomFields_1.ObjectId)
__special_sort_0, MIN(ObjectCustomFields_1.SortOrder) __special_sort_1
FROM CustomFields main JOIN ObjectCustomFields ObjectCustomFields_1
ON ( ObjectCustomFields_1.CustomField = main.id ) WHERE
(main.Disabled = ‘0’) AND (main.Name = NULL) AND
(ObjectCustomFields_1.ObjectId = ‘11’ OR ObjectCustomFields_1.ObjectId
= ‘0’) AND (main.LookupType = ‘RT::Queue-RT::Ticket-RT::Transaction’)
GROUP BY main.id ORDER BY __special_sort_0 DESC, __special_sort_1 ASC
’ at /usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder/Handle.pm line
522
DBIx::SearchBuilder::Handle::SimpleQuery(‘RT::Handle=HASH(0xa89f334)’,
'SELECT main.
, MAX(ObjectCustomFields_1.ObjectId) __special_s…’)
called at /usr/lib/perl5/site_perl/5.8.8/DBIx/SearchBuilder.pm line
236
DBIx::SearchBuilder::_DoSearch(‘RT::CustomFields=HASH(0xafa35a4)’)
called at /opt/rt3/bin/…/lib/RT/CustomFields_Overlay.pm line 168
RT::CustomFie

Any idea what this might be? In the past when RT lost contact with the
MySQL database (which is on a different host), I’d have to restart
Apache to get the two talking again. Restarting Apache in this case
hasn’t helped.

I’m trying to determine if this is a problem with database access, or
something wrong with my RT host.

Nobody has made any changes to anything that would have caused this.


The rt-users Archives

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com