Charts displaying only "sometimes" (GD problem?)

Hi all,
I’m having problems with charts, both bar and pie charts. Little context: created new dashboard and new saved search (privately at group-level); trying to display a nice report in the form of a bar chart.
It happens that under certain conditions RT doesn’t show any chart. Actually apache does not generate&return the image, as if I point Chrome to the link of the image [1], then it returns “Error 324 (net::ERR_EMPTY_RESPONSE) The server closed the connection without sending any data”.
Up to now I found the following conditions (not excluding each other) that influence the displaying or not of a chart:

  • type of chart (bar vs. pie)
  • domain on X axis (queue/owner name/owner email address/etc.)
  • language set in the user preferences
  • user logged in

Eg. given chart A:

  • if I login with user1 it shows, but with user2 it does not;
  • given a user, if I select <bar,queue> it doesn’t show, but <bar, CreatorNickname> works
  • given a user and a certain pair <type_of_chart, domain_on_x_axis>, it works if I turn to English from user preferencies; it doesn’t if I select Italian
  • and so on…

I’m using RT 4.0.2 (never upgraded, it’s a fresh install) on Red Hat Enterprise Linux AS release 4 (Nahant Update 2) with Apache/2.0.52, mod_perl ??, GD::Graph 1.44, glibc-2.3.4-2.13

CLUES:

1a. as soon as my browser asks for the image and RT doesn’t generate it, RT prints in /opt/rt4/var/log/apache2.error the following 2 lines:
*** glibc detected *** free(): invalid pointer: 0x0dd16764 *** [Wed Nov 23 16:38:23 2011] [notice] child pid 10963 exit signal Aborted (6), possible coredump in /tmp/apache_core_dumps

1b. There’s something more. Here’s what apache’s core dump says (resulting from the commands gdb /usr/sbin/httpd /tmp/apache_core_dumps/core.22613 + bt full):

#0 0x0096c7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 No symbol table info available.
#1 0x006a07d5 in raise () from /lib/tls/libc.so.6 No symbol table info available.
#2 0x006a2149 in abort () from /lib/tls/libc.so.6 No symbol table info available.
#3 0x006d440a in __libc_message () from /lib/tls/libc.so.6 No symbol table info available.
#4 0x006dab3f in _int_free () from /lib/tls/libc.so.6 No symbol table info available.
#5 0x006daeba in free () from /lib/tls/libc.so.6 No symbol table info available.
#6 0x00cb2a57 in gdFree () from /usr/lib/libgd.so.2 No symbol table info available.
#7 0x00be9c36 in XS_GD__Image_png (my_perl=0x9b85a98, cv=0xa875550)
at GD.xs:944
data = (void *) 0xcec9a44
size = 3545
level = Variable “level” is not available.

  1. after enabling logging by adding Set($LogToFile, “debug”) to RT_Site_Config.pm, I noticed the following message in rt.log:
    [Wed Nov 23 10:56:18 2011] [debug]: You’ve enabled GraphViz, but we couldn’t load the module: Can’t locate GraphViz.pm in @INC (@INC contains: […] at /opt/rt4/sbin/…/lib/RT/Config.pm line 542. (/opt/rt4/sbin/…/lib/RT/Config.pm:543)

Clue #2 doesn’t sound a good one to me, though. In fact I am trying to display charts not graphs! I read about showing relationships between tickets [2], but I’m not interested in that! Just wanted to extract some nice monthly reports as bar charts :slight_smile: Then, I should not need to install Graphviz, should I?

Here’s what I tried to do:

  • clean mason cache (more than once)
  • clean browser’s cache (more than once)
  • reinstalled GD::Image (with cpan ‘force install’)
  • added Set($DisableGD, undef) to RT_Site_Config.pm, as well as Set($DisableGD, 0)

According to clue#1b it’s GD that breaks everything (“at GD.xs:944”). Can you confirm my interpretation is correct? If so, maybe I could go for a reinstall of GD…?
Otherwise I could upgrade glibc, to what version?
Nevertheless, I feel like it’s a cache problem, server-side. Not client-side because cleaning browser’s cache doesn’t help. I don’t know if there exists a server-side cache in RT other than mason cache(which I already cleaned), however it’s my own best explanation. In fact, if it were glibc or GD, I should never ever get any charts then! While “sometimes” I do!

Can anyone point me in the right direction?
Thank you very much in advance.

Alberto

[1] http://mydomain.com/Search/Chart?Class=RT%3A%3ATickets&Format=&Query=(%20Queue%20%3D%20’EstesaP’%20OR%20Queue%20%3D%20’EstesaI’%20)%20AND%20Created%20>%20’-30%20days’&Rows=10&SavedSearchId=RT%3A%3AGroup-126-SavedSearch-33&SearchType=Chart&ShowNavigation=0&hideable=1

[2] Carbon60: Cloud Consulting - Services and Solutions

PS: If you need more examples/tests/information for debugging, please ask me.

The information transmitted is intended for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

Eg. given chart A:

  • if I login with user1 it shows, but with user2 it does not;
  • given a user, if I select <bar,queue> it doesn’t show, but <bar, CreatorNickname> works
  • given a user and a certain pair <type_of_chart, domain_on_x_axis>, it works if I turn to English from user preferencies; it doesn’t if I select Italian
  • and so on…

Are these all repeatable, or does it sometimes work and sometimes
coredump?

I’m using RT 4.0.2 (never upgraded, it’s a fresh install) on Red Hat Enterprise Linux AS release 4 (Nahant Update 2) with Apache/2.0.52, mod_perl ??, GD::Graph 1.44, glibc-2.3.4-2.13

CLUES:

1a. as soon as my browser asks for the image and RT doesn’t generate it, RT prints in /opt/rt4/var/log/apache2.error the following 2 lines:
*** glibc detected *** free(): invalid pointer: 0x0dd16764 *** [Wed Nov 23 16:38:23 2011] [notice] child pid 10963 exit signal Aborted (6), possible coredump in /tmp/apache_core_dumps

1b. There’s something more. Here’s what apache’s core dump says (resulting from the commands gdb /usr/sbin/httpd /tmp/apache_core_dumps/core.22613 + bt full):

#0 0x0096c7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 No symbol table info available.
#1 0x006a07d5 in raise () from /lib/tls/libc.so.6 No symbol table info available.
#2 0x006a2149 in abort () from /lib/tls/libc.so.6 No symbol table info available.
#3 0x006d440a in __libc_message () from /lib/tls/libc.so.6 No symbol table info available.
#4 0x006dab3f in _int_free () from /lib/tls/libc.so.6 No symbol table info available.
#5 0x006daeba in free () from /lib/tls/libc.so.6 No symbol table info available.
#6 0x00cb2a57 in gdFree () from /usr/lib/libgd.so.2 No symbol table info available.
#7 0x00be9c36 in XS_GD__Image_png (my_perl=0x9b85a98, cv=0xa875550)
at GD.xs:944
data = (void *) 0xcec9a44
size = 3545
level = Variable “level” is not available.

  1. after enabling logging by adding Set($LogToFile, “debug”) to RT_Site_Config.pm, I noticed the following message in rt.log:
    [Wed Nov 23 10:56:18 2011] [debug]: You’ve enabled GraphViz, but we couldn’t load the module: Can’t locate GraphViz.pm in @INC (@INC contains: […] at /opt/rt4/sbin/…/lib/RT/Config.pm line 542. (/opt/rt4/sbin/…/lib/RT/Config.pm:543)

This log is unrelated to what you’re seeing.
You’re much more likely to get something useful from apache’s logs
than RT’s logs.

  • clean mason cache (more than once)
  • clean browser’s cache (more than once)

These shouldn’t be relevant.

  • reinstalled GD::Image (with cpan ‘force install’)

Why did you need to force install?
Does GD::Image pass tests under the cpan shell?
The top-level package should actually be GD, RT requires
that GD, GD::Graph and GD::Text are all installed and working properly
to generate charts.

  • added Set($DisableGD, undef) to RT_Site_Config.pm, as well as Set($DisableGD, 0)

These are just the defaults so shouldn’t matter.

According to clue#1b it’s GD that breaks everything (“at GD.xs:944”).
Can you confirm my interpretation is correct? If so, maybe I could go
for a reinstall of GD…? Otherwise I could upgrade glibc, to what
version?

Nevertheless, I feel like it’s a cache problem, server-side. Not client-
side because cleaning browser’s cache doesn’t help. I don’t know if
there exists a server-side cache in RT other than mason cache(which I
already cleaned), however it’s my own best explanation. In fact, if it
were glibc or GD, I should never ever get any charts then! While
“sometimes” I do!

It’s much more likely that you’re exposing some sort of bug in GD than
having a cache problem. RT doesn’t cache charts, it generates them on
the fly.

Are you getting other segfaults on the machine? I’d be looking at the
version of libgd itself and the GD perl modules.

If you can provide a replication recipe that one of us could reproduce
in a debugging environment, that would be great to see as a bug
report.

-kevin