Rt2.0.9 times out/segfaults

RT Users
1

Hi all,

happy new year, and thanks for an unbelievable product.

we’re running rt 2.0.9 on redhat 7.2, with postgresql-7.1.3-2,
and sendmail-8.11.6-3.

I’m getting

[Wed Jan 2 10:07:19 2002] [notice] child pid 17354 exit signal Segmentation fault (11)

under a weird circumstance, and I’m hoping someone will have some clues as to
where I can look.

The above occurs only for one user (myself :slight_smile: ) when trying to reply to
tickets or create new tickets. I get nothing but blank files in /tmp/*.
I think it started to occur when I had replied to another (totally unrelated)
ticket, and then interrupted the browser by pressing the stop key.
Any ideas? Clues as to where I might look? debugging I can turn on?

(BTW, we’ve been running RT2 for about 3 months now, and have over
7000 tickets logged - this is the first hint of a problem we’ve
had).

–Calvin

------------------------* My opinions are mine *------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

2

Hi all,

happy new year, and thanks for an unbelievable product.

we’re running rt 2.0.9 on redhat 7.2, with postgresql-7.1.3-2,
and sendmail-8.11.6-3.

I’m getting

[Wed Jan 2 10:07:19 2002] [notice] child pid 17354 exit signal Segmentation fault (11)

under a weird circumstance, and I’m hoping someone will have some clues as to
where I can look.

Ok,

I’ve recompiled apache without DSO support. Same thing, and I still
get the segmentation fault :frowning:

Apache is apache_1.3.22, mod perl is mod_perl-1.24_01.

The above occurs only for one user (myself :slight_smile: ) when trying to reply to
tickets or create new tickets. I get nothing but blank files in /tmp/*.
I think it started to occur when I had replied to another (totally unrelated)
ticket, and then interrupted the browser by pressing the stop key.
Any ideas? Clues as to where I might look? debugging I can turn on?

(BTW, we’ve been running RT2 for about 3 months now, and have over
7000 tickets logged - this is the first hint of a problem we’ve
had).

–Calvin

Anyone have any ideas, debugging I can turn on? I’m pretty sure
that its not the DSO segfault any more.

My next step is to dump the database, but I’d appreciate any ideas
before I go down that route.

–Calvin

------------------------* My opinions are mine *------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

3

Hi,

You might try running Apache under ‘strace’ (‘truss’ under Solaris)
and repeating whatever causes the segfaults. I’ve needed to do this
to resolve mysterious Apache problems before.

– BobOn 5 Jan 2002, at 16:32, Calvin Browne wrote:

Hi all,

happy new year, and thanks for an unbelievable product.

we’re running rt 2.0.9 on redhat 7.2, with postgresql-7.1.3-2,
and sendmail-8.11.6-3.

I’m getting

[Wed Jan 2 10:07:19 2002] [notice] child pid 17354 exit signal
Segmentation fault (11)

under a weird circumstance, and I’m hoping someone will have some
clues as to where I can look.

Ok,

I’ve recompiled apache without DSO support. Same thing, and I still
get the segmentation fault :frowning:

Apache is apache_1.3.22, mod perl is mod_perl-1.24_01.

The above occurs only for one user (myself :slight_smile: ) when trying to reply
to tickets or create new tickets. I get nothing but blank files in
/tmp/*. I think it started to occur when I had replied to another
(totally unrelated) ticket, and then interrupted the browser by
pressing the stop key. Any ideas? Clues as to where I might look?
debugging I can turn on?

(BTW, we’ve been running RT2 for about 3 months now, and have over
7000 tickets logged - this is the first hint of a problem we’ve
had).

–Calvin

Anyone have any ideas, debugging I can turn on? I’m pretty sure
that its not the DSO segfault any more.

My next step is to dump the database, but I’d appreciate any ideas
before I go down that route.

–Calvin

------------------------* My opinions are mine
*------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

4

Hi,

You might try running Apache under ‘strace’ (‘truss’ under Solaris)
and repeating whatever causes the segfaults. I’ve needed to do this
to resolve mysterious Apache problems before.

– Bob

Thanks for the info.

my current Apache binary:
[root@rt root]# /usr/sbin/httpd -l
Compiled-in modules:
http_core.c
mod_env.c
mod_log_config.c
mod_mime.c
mod_negotiation.c
mod_status.c
mod_include.c
mod_autoindex.c
mod_dir.c
mod_cgi.c
mod_asis.c
mod_imap.c
mod_actions.c
mod_userdir.c
mod_alias.c
mod_access.c
mod_auth.c
mod_auth_dbm.c
mod_setenvif.c
mod_ssl.c
mod_perl.c
suexec: enabled; valid wrapper /usr/sbin/suexec

and the strace output for

[Sun Jan 6 11:21:27 2002] [notice] child pid 6187 exit signal Segmentation fault (11)

(ummm - its pretty long).

Any ideas from anyone out there?

–Calvin

[pid 6179] <… fork resumed> ) = 6187
[pid 6187] rt_sigaction(SIGHUP, {0x80c1c80, , SA_INTERRUPT|0x4000000}, <unfinished …>
[pid 6187] <… rt_sigaction resumed> {0x80c1d28, [HUP USR1], 0x4000000}, 8) = 0
[pid 6187] rt_sigaction(SIGUSR1, {0x80c1c80, , SA_INTERRUPT|0x4000000}, <unfinished …>
[pid 6187] <… rt_sigaction resumed> {0x80c1d28, [HUP USR1], 0x4000000}, 8) = 0
[pid 6187] rt_sigaction(SIGTERM, {0x80c1c80, , SA_INTERRUPT|0x4000000}, <unfinished …>
[pid 6187] <… rt_sigaction resumed> {0x80c1d18, , 0x4000000}, 8) = 0
[pid 6187] getpid( <unfinished …>
[pid 6187] <… getpid resumed> ) = 6187
[pid 6187] geteuid32( <unfinished …>
[pid 6187] <… geteuid32 resumed> ) = 0
[pid 6187] setgid32(0x1f4 <unfinished …>
[pid 6187] <… setgid32 resumed> ) = 0
[pid 6187] open(“/etc/group”, O_RDONLY <unfinished …>
[pid 6187] <… open resumed> ) = 3
[pid 6187] fcntl64(0x3, 0x1, 0, 0x1 <unfinished …>
[pid 6187] <… fcntl64 resumed> ) = 0
[pid 6187] fcntl64(0x3, 0x2, 0x1, 0x1 <unfinished …>
[pid 6187] <… fcntl64 resumed> ) = 0
[pid 6187] fstat64(3, <unfinished …>
[pid 6187] <… fstat64 resumed> {st_mode=S_IFREG|0644, st_size=534, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0 <unfinished …>
[pid 6187] <… mmap2 resumed> ) = 0x4037e000
[pid 6187] _llseek(3, 0, <unfinished …>
[pid 6187] <… llseek resumed> [0], SEEK_CUR) = 0
[pid 6187] read(3, <unfinished …>
[pid 6187] <… read resumed> “root:x:0:root\nbin:x:1:root,bin,d”…, 4096) = 534
[pid 6187] read(3, <unfinished …>
[pid 6187] <… read resumed> “”, 4096) = 0
[pid 6187] close(3 <unfinished …>
[pid 6187] <… close resumed> ) = 0
[pid 6187] munmap(0x4037e000, 4096 <unfinished …>
[pid 6187] <… munmap resumed> ) = 0
[pid 6187] open(“/etc/ld.so.cache”, O_RDONLY <unfinished …>
[pid 6187] <… open resumed> ) = 3
[pid 6187] fstat64(3, <unfinished …>
[pid 6187] <… fstat64 resumed> {st_mode=S_IFREG|0644, st_size=27186, …}) = 0
[pid 6187] old_mmap(NULL, 27186, PROT_READ, MAP_PRIVATE, 3, 0 <unfinished …>
[pid 6187] <… old_mmap resumed> ) = 0x4042e000
[pid 6187] close(3 <unfinished …>
[pid 6187] <… close resumed> ) = 0
[pid 6187] open(“/lib/libnss_nisplus.so.2”, O_RDONLY <unfinished …>
[pid 6187] <… open resumed> ) = 3
[pid 6187] read(3, <unfinished …>
[pid 6187] <… read resumed> “\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\260\36”…, 1024) = 1024
[pid 6187] fstat64(3, <unfinished …>
[pid 6187] <… fstat64 resumed> {st_mode=S_IFREG|0755, st_size=355236, …}) = 0
[pid 6187] old_mmap(NULL, 43304, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0 <unfinished …>
[pid 6187] <… old_mmap resumed> ) = 0x40435000
[pid 6187] mprotect(0x4043f000, 2344, PROT_NONE <unfinished …>
[pid 6187] <… mprotect resumed> ) = 0
[pid 6187] old_mmap(0x4043f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x9000 <unfinished …>
[pid 6187] <… old_mmap resumed> ) = 0x4043f000
[pid 6187] close(3 <unfinished …>
[pid 6187] <… close resumed> ) = 0
[pid 6187] munmap(0x4042e000, 27186 <unfinished …>
[pid 6187] <… munmap resumed> ) = 0
[pid 6187] uname( <unfinished …>
[pid 6187] <… uname resumed> {sys=“Linux”, node=“rt.ourdomainname.co.za”, …}) = 0
[pid 6187] open(“/var/nis/NIS_COLD_START”, O_RDONLY <unfinished …>
[pid 6187] <… open resumed> ) = -1 ENOENT (No such file or directory)
[pid 6187] setgroups32(0x1, 0x8260b50 <unfinished …>
[pid 6187] <… setgroups32 resumed> ) = 0
[pid 6187] geteuid32( <unfinished …>
[pid 6187] <… geteuid32 resumed> ) = 0
[pid 6187] setuid32(0x30 <unfinished …>
[pid 6187] <… setuid32 resumed> ) = 0
[pid 6187] getpid( <unfinished …>
[pid 6187] <… getpid resumed> ) = 6187
[pid 6187] getuid32( <unfinished …>
[pid 6187] <… getuid32 resumed> ) = 48
[pid 6187] geteuid32( <unfinished …>
[pid 6187] <… geteuid32 resumed> ) = 48
[pid 6187] getgid32( <unfinished …>
[pid 6187] <… getgid32 resumed> ) = 500
[pid 6187] getegid32( <unfinished …>
[pid 6187] <… getegid32 resumed> ) = 500
[pid 6187] open(“/etc/httpd/logs/ssl_mutex.6178”, O_WRONLY <unfinished …>
[pid 6187] <… open resumed> ) = 3
[pid 6187] fcntl64(0x3, 0, 0xf, 0x400168e4 <unfinished …>
[pid 6187] <… fcntl64 resumed> ) = 25
[pid 6187] close(3 <unfinished …>
[pid 6187] <… close resumed> ) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
[pid 6187] rt_sigaction(SIGURG, {0x80c0738, [], SA_INTERRUPT|0x4000000}, <unfinished …>
[pid 6187] <… rt_sigaction resumed> {SIG_DFL}, 8) = 0
[pid 6187] rt_sigaction(SIGALRM, {0x80c09d0, [], SA_INTERRUPT|0x4000000}, <unfinished …>
[pid 6187] <… rt_sigaction resumed> {SIG_DFL}, 8) = 0
[pid 6187] rt_sigaction(SIGUSR1, {0x80c1cac, [], SA_INTERRUPT|0x4000000}, <unfinished …>
[pid 6187] <… rt_sigaction resumed> {0x80c1c80, [], SA_INTERRUPT|0x4000000}, 8) = 0
[pid 6187] semop(1540098, 0x819fb50, 1 <unfinished …>
[pid 6187] <… semop resumed> ) = 0
[pid 6187] select(19, [17 18], NULL, NULL, NULL <unfinished …>
[pid 6187] <… select resumed> ) = 1 (in [17])
[pid 6187] accept(17, {sin_family=AF_INET, sin_port=htons(32909), sin_addr=inet_addr(“192.16192.168.34”)}}, [16]) = 3
[pid 6187] semop(1540098, 0x819fb56, 1) = 0
[pid 6187] rt_sigaction(SIGUSR1, {SIG_IGN}, {0x80c1cac, [], SA_INTERRUPT|0x4000000}, 8) = 0
[pid 6187] getsockname(3, {sin_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr(“192.168.247.6”)}}, [16]) = 0
[pid 6187] setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
[pid 6187] brk(0x8cba000) = 0x8cba000
[pid 6187] brk(0x8cbd000) = 0x8cbd000
[pid 6187] time(NULL) = 1010308583
[pid 6187] getpid() = 6187
[pid 6187] time(NULL) = 1010308583
[pid 6187] time(NULL) = 1010308583
[pid 6187] getpid() = 6187
[pid 6187] brk(0x8cc0000) = 0x8cc0000
[pid 6187] time(NULL) = 1010308583
[pid 6187] getpid() = 6187
[pid 6187] brk(0x8cc6000) = 0x8cc6000
[pid 6187] brk(0x8ccf000) = 0x8ccf000
[pid 6187] brk(0x8cd4000) = 0x8cd4000
[pid 6187] read(3, “\200F\1\3\0\0-\0\0\0\20”, 11) = 11
[pid 6187] read(3, “\1\0\200\7\0\300\3\0\200\6\0@\2\0\200\4\0\200\0\0\4\0\376”…, 61) = 61
[pid 6187] time(NULL) = 1010308583
[pid 6187] getpid() = 6187
[pid 6187] time(NULL) = 1010308583
[pid 6187] getpid() = 6187
[pid 6187] getpid() = 6187
[pid 6187] time(NULL) = 1010308583
[pid 6187] getpid() = 6187
[pid 6187] getpid() = 6187
[pid 6187] write(3, “\26\3\0\0*\2\0\0&\3\0<8\25\347j\314p"\2\31\31:\177\366”…, 1029) = 1029
[pid 6187] read(3, <unfinished …>
[pid 6187] <… read resumed> “\26\3\0\0\204”, 5) = 5
[pid 6187] read(3, “\20\0\0\200vf<CJ\363#\n^5P%p\322\35\4\17a\21\330\215\315”…, 132) = 132
[pid 6187] read(3, “\24\3\0\0\1”, 5) = 5
[pid 6187] read(3, “\1”, 1) = 1
[pid 6187] read(3, “\26\3\0\0008”, 5) = 5
[pid 6187] read(3, “\232\302\356\261\f0/\24\274\35\333\37\247P\276r\32\360”…, 56) = 56
[pid 6187] brk(0x8cd6000) = 0x8cd6000
[pid 6187] write(3, “\24\3\0\0\1\1\26\3\0\0008\216zE \10\240d+>\321v\331\21”…, 67) = 67
[pid 6187] read(3, “\27\3\0\2jyJo\325\251\21|.\302C.3\236\25_)9\356\1\246"..., 18437) = 623 [pid 6187] rt_sigaction(SIGUSR1, {SIG_IGN}, {SIG_IGN}, 8) = 0 [pid 6187] time(NULL) = 1010308583 [pid 6187] read(3, "\27\3\0\6\D\336\35\37<\271\223.\336X\n\343'U\267\247”…, 18437) = 1448
[pid 6187] read(3, "\24\36?\353\255\36*kHV \210\312\373\7\375\242\35g\276"…, 189) = 189
[pid 6187] stat64(“/opt/rt2/WebRT/html/Ticket/Display.html”, {st_mode=S_IFREG|0755, st_size=4226, …}) = 0
[pid 6187] rt_sigaction(SIGALRM, NULL, {0x80c09d0, , SA_INTERRUPT|0x4000000}, 8) = 0
[pid 6187] dup2(16, 2) = 2
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] open(“/tmp/debugfile”, O_WRONLY|O_APPEND|O_CREAT, 0666) = 4
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=4901072, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4037e000
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=4901072, …}) = 0
[pid 6187] _llseek(4, 4901072, [4901072], SEEK_SET) = 0
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=4901072, …}) = 0
[pid 6187] fcntl64(0x4, 0x2, 0x1, 0) = 0
[pid 6187] write(4, “DBIx::SearchBuilder – \n”, 24) = 24
[pid 6187] close(4) = 0
[pid 6187] munmap(0x4037e000, 4096) = 0
[pid 6187] open(“/tmp/debugfile”, O_WRONLY|O_APPEND|O_CREAT, 0666) = 4
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=4901096, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4037e000
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=4901096, …}) = 0
[pid 6187] _llseek(4, 4901096, [4901096], SEEK_SET) = 0
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=4901096, …}) = 0
[pid 6187] fcntl64(0x4, 0x2, 0x1, 0) = 0
[pid 6187] stat64(“/opt/rt2/etc/DBD/Pg.pmc”, 0xbffff270) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/opt/rt2/etc/DBD/Pg.pm”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/opt/rt2/lib/DBD/Pg.pmc”, 0xbffff270) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/opt/rt2/lib/DBD/Pg.pm”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/i386-linux/DBD/Pg.pmc”, 0xbffff270) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/perl5/5.6.0/i386-linux/DBD/Pg.pm”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/DBD/Pg.pmc”, 0xbffff270) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/perl5/5.6.0/DBD/Pg.pm”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/DBD/Pg.pmc”, 0xbffff270) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/DBD/Pg.pm”, O_RDONLY) = 5
[pid 6187] fstat64(5, {st_mode=S_IFREG|0444, st_size=35704, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4037f000
[pid 6187] read(5, “\n# $Id: Pg.pm,v 1.44 2001/06/27”…, 4096) = 4096
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] brk(0x8cd7000) = 0x8cd7000
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] brk(0x8cd8000) = 0x8cd8000
[pid 6187] stat64(“/opt/rt2/etc/POSIX.pmc”, 0xbfffee80) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/opt/rt2/etc/POSIX.pm”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/opt/rt2/lib/POSIX.pmc”, 0xbfffee80) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/opt/rt2/lib/POSIX.pm”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/i386-linux/POSIX.pmc”, 0xbfffee80) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/perl5/5.6.0/i386-linux/POSIX.pm”, O_RDONLY) = 6
[pid 6187] fstat64(6, {st_mode=S_IFREG|0444, st_size=18784, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x403d2000
[pid 6187] read(6, “package POSIX;\n\nour(@ISA, %EXPOR”…, 4096) = 4096
[pid 6187] brk(0x8cd9000) = 0x8cd9000
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] open(“/usr/lib/perl5/5.6.0/i386-linux/auto/POSIX/autosplit.ix”, O_RDONLY) = 7
[pid 6187] brk(0x8cda000) = 0x8cda000
[pid 6187] fstat64(7, {st_mode=S_IFREG|0444, st_size=2168, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x403d4000
[pid 6187] read(7, “# Index created by AutoSplit for”…, 4096) = 2168
[pid 6187] brk(0x8cdb000) = 0x8cdb000
[pid 6187] brk(0x8cdc000) = 0x8cdc000
[pid 6187] read(7, “”, 4096) = 0
[pid 6187] close(7) = 0
[pid 6187] munmap(0x403d4000, 4096) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] brk(0x8cdd000) = 0x8cdd000
[pid 6187] brk(0x8cde000) = 0x8cde000
[pid 6187] brk(0x8cdf000) = 0x8cdf000
[pid 6187] brk(0x8ce0000) = 0x8ce0000
[pid 6187] brk(0x8ce1000) = 0x8ce1000
[pid 6187] close(6) = 0
[pid 6187] munmap(0x403d2000, 4096) = 0
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/i386-linux/auto/POSIX/POSIX.so”, {st_mode=S_IFREG|0555, st_size=95207, …}) = 0
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/i386-linux/auto/POSIX/POSIX.bs”, {st_mode=S_IFREG|0444, st_size=0, …}) = 0
[pid 6187] open(“/usr/lib/perl5/5.6.0/i386-linux/auto/POSIX/POSIX.so”, O_RDONLY) = 6
[pid 6187] read(6, “\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20;\0\000”…, 1024) = 1024
[pid 6187] fstat64(6, {st_mode=S_IFREG|0555, st_size=95207, …}) = 0
[pid 6187] brk(0x8ce2000) = 0x8ce2000
[pid 6187] old_mmap(NULL, 87788, PROT_READ|PROT_EXEC, MAP_PRIVATE, 6, 0) = 0x40440000
[pid 6187] mprotect(0x40455000, 1772, PROT_NONE) = 0
[pid 6187] old_mmap(0x40455000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 6, 0x14000) = 0x40455000
[pid 6187] close(6) = 0
[pid 6187] brk(0x8ce3000) = 0x8ce3000
[pid 6187] brk(0x8ce4000) = 0x8ce4000
[pid 6187] brk(0x8ce5000) = 0x8ce5000
[pid 6187] brk(0x8ce6000) = 0x8ce6000
[pid 6187] brk(0x8ce7000) = 0x8ce7000
[pid 6187] brk(0x8ce8000) = 0x8ce8000
[pid 6187] brk(0x8ce9000) = 0x8ce9000
[pid 6187] brk(0x8cea000) = 0x8cea000
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/i386-linux/auto/POSIX/load_imports.al”, {st_mode=S_IFREG|0444, st_size=6522, …}) = 0
[pid 6187] getgroups32(0x20, 0xbfffefc0) = 1
[pid 6187] open(“/usr/lib/perl5/5.6.0/i386-linux/auto/POSIX/load_imports.al”, O_RDONLY) = 6
[pid 6187] fstat64(6, {st_mode=S_IFREG|0444, st_size=6522, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x403d2000
[pid 6187] read(6, “# NOTE: Derived from …/…/lib/P”…, 4096) = 4096
[pid 6187] brk(0x8ceb000) = 0x8ceb000
[pid 6187] brk(0x8cec000) = 0x8cec000
[pid 6187] brk(0x8ced000) = 0x8ced000
[pid 6187] brk(0x8cee000) = 0x8cee000
[pid 6187] brk(0x8cef000) = 0x8cef000
[pid 6187] brk(0x8cf0000) = 0x8cf0000
[pid 6187] read(6, “sys_stat_h => [qw(S_IRGRP S_IROT”…, 4096) = 2426
[pid 6187] brk(0x8cf1000) = 0x8cf1000
[pid 6187] brk(0x8cf2000) = 0x8cf2000
[pid 6187] brk(0x8cf3000) = 0x8cf3000
[pid 6187] brk(0x8cf4000) = 0x8cf4000
[pid 6187] brk(0x8cf5000) = 0x8cf5000
[pid 6187] read(6, “”, 4096) = 0
[pid 6187] close(6) = 0
[pid 6187] munmap(0x403d2000, 4096) = 0
[pid 6187] brk(0x8cf6000) = 0x8cf6000
[pid 6187] brk(0x8cf7000) = 0x8cf7000
[pid 6187] brk(0x8cf8000) = 0x8cf8000
[pid 6187] brk(0x8cf9000) = 0x8cf9000
[pid 6187] brk(0x8cfa000) = 0x8cfa000
[pid 6187] brk(0x8cfb000) = 0x8cfb000
[pid 6187] brk(0x8cfc000) = 0x8cfc000
[pid 6187] brk(0x8cfd000) = 0x8cfd000
[pid 6187] brk(0x8cfe000) = 0x8cfe000
[pid 6187] brk(0x8cff000) = 0x8cff000
[pid 6187] brk(0x8d01000) = 0x8d01000
[pid 6187] brk(0x8d03000) = 0x8d03000
[pid 6187] brk(0x8d04000) = 0x8d04000
[pid 6187] brk(0x8d05000) = 0x8d05000
[pid 6187] brk(0x8d07000) = 0x8d07000
[pid 6187] brk(0x8d08000) = 0x8d08000
[pid 6187] brk(0x8d0b000) = 0x8d0b000
[pid 6187] brk(0x8d0c000) = 0x8d0c000
[pid 6187] brk(0x8d0d000) = 0x8d0d000
[pid 6187] read(5, " where relkind = 'r' \n"…, 4096) = 4096
[pid 6187] brk(0x8d0e000) = 0x8d0e000
[pid 6187] brk(0x8d0f000) = 0x8d0f000
[pid 6187] brk(0x8d10000) = 0x8d10000
[pid 6187] brk(0x8d11000) = 0x8d11000
[pid 6187] read(5, " | 2| 5| SM"…, 4096) = 4096
[pid 6187] brk(0x8d12000) = 0x8d12000
[pid 6187] brk(0x8d13000) = 0x8d13000
[pid 6187] brk(0x8d14000) = 0x8d14000
[pid 6187] brk(0x8d15000) = 0x8d15000
[pid 6187] brk(0x8d16000) = 0x8d16000
[pid 6187] brk(0x8d17000) = 0x8d17000
[pid 6187] read(5, “$data_type) {\n $str =”…, 4096) = 4096
[pid 6187] brk(0x8d18000) = 0x8d18000
[pid 6187] brk(0x8d19000) = 0x8d19000
[pid 6187] close(5) = 0
[pid 6187] munmap(0x4037f000, 4096) = 0
[pid 6187] stat64(“/opt/rt2/etc/auto/DBD/Pg”, 0xbffff3d0) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/opt/rt2/lib/auto/DBD/Pg”, 0xbffff3d0) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/i386-linux/auto/DBD/Pg”, 0xbffff3d0) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/5.6.0/auto/DBD/Pg”, 0xbffff3d0) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/DBD/Pg”, {st_mode=S_IFDIR|0755, st_size=4096, …}) = 0
[pid 6187] stat64(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/DBD/Pg/Pg.so”, {st_mode=S_IFREG|0555, st_size=55106, …}) = 0
[pid 6187] stat64(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/DBD/Pg/Pg.bs”, 0xbffff3d0) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/DBD/Pg/Pg.so”, O_RDONLY) = 5
[pid 6187] read(5, “\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0 *\0\000”…, 1024) = 1024
[pid 6187] fstat64(5, {st_mode=S_IFREG|0555, st_size=55106, …}) = 0
[pid 6187] brk(0x8d1a000) = 0x8d1a000
[pid 6187] old_mmap(NULL, 50700, PROT_READ|PROT_EXEC, MAP_PRIVATE, 5, 0) = 0x40456000
[pid 6187] mprotect(0x40462000, 1548, PROT_NONE) = 0
[pid 6187] old_mmap(0x40462000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0xb000) = 0x40462000
[pid 6187] close(5) = 0
[pid 6187] open(“/usr/lib/i686/mmx/libpq.so.2”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/i686/mmx”, 0xbfffe9b0) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/i686/libpq.so.2”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/i686”, 0xbfffe9b0) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/mmx/libpq.so.2”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] stat64(“/usr/lib/mmx”, 0xbfffe9b0) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/libpq.so.2”, O_RDONLY) = 5
[pid 6187] read(5, “\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\08\0\000"..., 1024) = 1024 [pid 6187] fstat64(5, {st_mode=S_IFREG|0755, st_size=76352, ...}) = 0 [pid 6187] old_mmap(NULL, 67356, PROT_READ|PROT_EXEC, MAP_PRIVATE, 5, 0) = 0x40463000 [pid 6187] mprotect(0x40472000, 5916, PROT_NONE) = 0 [pid 6187] old_mmap(0x40472000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0xe000) = 0x40472000 [pid 6187] close(5) = 0 [pid 6187] open("/usr/lib/libkrb5.so.3", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 6187] open("/usr/lib/libkrb5.so.3", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 6187] open("/etc/ld.so.cache", O_RDONLY) = 5 [pid 6187] fstat64(5, {st_mode=S_IFREG|0644, st_size=27186, ...}) = 0 [pid 6187] old_mmap(NULL, 27186, PROT_READ, MAP_PRIVATE, 5, 0) = 0x4042e000 [pid 6187] close(5) = 0 [pid 6187] open("/usr/kerberos/lib/libkrb5.so.3", O_RDONLY) = 5 [pid 6187] read(5, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\260\360"..., 1024) = 1024 [pid 6187] fstat64(5, {st_mode=S_IFREG|0755, st_size=425493, ...}) = 0 [pid 6187] old_mmap(NULL, 358208, PROT_READ|PROT_EXEC, MAP_PRIVATE, 5, 0) = 0x40474000 [pid 6187] mprotect(0x404c9000, 10048, PROT_NONE) = 0 [pid 6187] old_mmap(0x404c9000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0x54000) = 0x404c9000 [pid 6187] close(5) = 0 [pid 6187] open("/usr/lib/libk5crypto.so.3", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 6187] open("/usr/lib/libk5crypto.so.3", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 6187] open("/usr/kerberos/lib/libk5crypto.so.3", O_RDONLY) = 5 [pid 6187] read(5, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0!\0\000"..., 1024) = 1024 [pid 6187] fstat64(5, {st_mode=S_IFREG|0755, st_size=78193, ...}) = 0 [pid 6187] old_mmap(NULL, 67828, PROT_READ|PROT_EXEC, MAP_PRIVATE, 5, 0) = 0x404cc000 [pid 6187] mprotect(0x404db000, 6388, PROT_NONE) = 0 [pid 6187] old_mmap(0x404db000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0xe000) = 0x404db000 [pid 6187] close(5) = 0 [pid 6187] open("/usr/lib/libcom_err.so.3", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 6187] open("/usr/lib/libcom_err.so.3", O_RDONLY) = -1 ENOENT (No such file or directory) [pid 6187] open("/usr/kerberos/lib/libcom_err.so.3", O_RDONLY) = 5 [pid 6187] read(5, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\t\0\000”…, 1024) = 1024
[pid 6187] fstat64(5, {st_mode=S_IFREG|0755, st_size=8713, …}) = 0
[pid 6187] old_mmap(NULL, 8616, PROT_READ|PROT_EXEC, MAP_PRIVATE, 5, 0) = 0x404dd000
[pid 6187] mprotect(0x404de000, 4520, PROT_NONE) = 0
[pid 6187] old_mmap(0x404de000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0) = 0x404de000
[pid 6187] close(5) = 0
[pid 6187] open(“/usr/lib/libresolv.so.2”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/usr/lib/libresolv.so.2”, O_RDONLY) = -1 ENOENT (No such file or directory)
[pid 6187] open(“/lib/libresolv.so.2”, O_RDONLY) = 5
[pid 6187] read(5, “\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220'\0”…, 1024) = 1024
[pid 6187] fstat64(5, {st_mode=S_IFREG|0755, st_size=261196, …}) = 0
[pid 6187] old_mmap(NULL, 72512, PROT_READ|PROT_EXEC, MAP_PRIVATE, 5, 0) = 0x404e0000
[pid 6187] mprotect(0x404ee000, 15168, PROT_NONE) = 0
[pid 6187] old_mmap(0x404ee000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 5, 0xd000) = 0x404ee000
[pid 6187] old_mmap(0x404f0000, 6976, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x404f0000
[pid 6187] close(5) = 0
[pid 6187] brk(0x8d1b000) = 0x8d1b000
[pid 6187] munmap(0x4042e000, 27186) = 0
[pid 6187] brk(0x8d1c000) = 0x8d1c000
[pid 6187] brk(0x8d1d000) = 0x8d1d000
[pid 6187] brk(0x8d1e000) = 0x8d1e000
[pid 6187] time(NULL) = 1010308584
[pid 6187] brk(0x8d23000) = 0x8d23000
[pid 6187] brk(0x8d26000) = 0x8d26000
[pid 6187] socket(PF_UNIX, SOCK_STREAM, 0) = 5
[pid 6187] connect(5, {sin_family=AF_UNIX, path=“/var/run/.nscd_socket”}, 110) = -1 ENOENT (No such file or directory)
[pid 6187] close(5) = 0
[pid 6187] open(“/etc/hosts”, O_RDONLY) = 5
[pid 6187] fcntl64(0x5, 0x1, 0, 0x1) = 0
[pid 6187] fcntl64(0x5, 0x2, 0x1, 0x1) = 0
[pid 6187] fstat64(5, {st_mode=S_IFREG|0644, st_size=180, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4037f000
[pid 6187] read(5, “# Do not remove the following li”…, 4096) = 180
[pid 6187] close(5) = 0
[pid 6187] munmap(0x4037f000, 4096) = 0
[pid 6187] socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 5
[pid 6187] setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0
[pid 6187] connect(5, {sin_family=AF_INET, sin_port=htons(5432), sin_addr=inet_addr(“127.0.0.1”)}}, 16) = 0
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “\0\0\1(\4\322\26/\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0”…, 296, 0) = 296
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] recv(5, “N”, 1, 0) = 1
[pid 6187] fcntl64(0x5, 0x4, 0x800, 0x8cc0630) = 0
[pid 6187] select(6, , [5], [5], NULL) = 1 (out [5])
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “\0\0\1(\0\2\0\0rt2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0”…, 296, 0) = 296
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] select(6, [5], , [5], NULL) = 1 (in [5])
[pid 6187] recv(5, “R\0\0\0\0”, 16384, 0) = 5
[pid 6187] select(6, [5], , [5], NULL) = 1 (in [5])
[pid 6187] recv(5, “K\0\0\30M\353\266\365\255Z”, 16384, 0) = 10
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “Qselect getdatabaseencoding()\0”, 30, 0) = 30
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] select(6, [5], , [5], NULL <unfinished …>
[pid 6187] <… select resumed> ) = 1 (in [5])
[pid 6187] recv(5, “Pblank\0T\0\1getdatabaseencoding\0\0\0”…, 16384, 0) = 64
[pid 6187] write(4, “DBIx::SearchBuilder::Handle Just”…, 152) = 152
[pid 6187] close(4) = 0
[pid 6187] munmap(0x4037e000, 4096) = 0
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “QSET TIME ZONE 'GMT'\0”, 21, 0) = 21
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] select(6, [5], , [5], NULL) = 1 (in [5])
[pid 6187] recv(5, “CSET VARIABLE\0Z”, 16384, 0) = 15
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “QSET DATESTYLE TO 'ISO'\0”, 24, 0) = 24
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] select(6, [5], , [5], NULL) = 1 (in [5])
[pid 6187] recv(5, “CSET VARIABLE\0Z”, 16384, 0) = 15
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] brk(0x8d27000) = 0x8d27000
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “QSELECT * FROM Users WHERE Name”…, 47, 0) = 47
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] select(6, [5], , [5], NULL) = 1 (in [5])
[pid 6187] recv(5, “Pblank\0T\0"id\0\0\0\0\27\0\4\377\377\377\377name\0\0”…, 16384, 0) = 908
[pid 6187] brk(0x8d28000) = 0x8d28000
[pid 6187] brk(0x8d29000) = 0x8d29000
[pid 6187] brk(0x8d2a000) = 0x8d2a000
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] time(NULL) = 1010308584
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] send(5, “QSELECT * FROM Users WHERE Name”…, 44, 0) = 44
[pid 6187] rt_sigaction(SIGPIPE, {SIG_IGN}, {SIG_IGN}, 8) = 0
[pid 6187] select(6, [5], , [5], NULL) = 1 (in [5])
[pid 6187] recv(5, “Pblank\0T\0"id\0\0\0\0\27\0\4\377\377\377\377name\0\0”…, 16384, 0) = 948
[pid 6187] brk(0x8d2b000) = 0x8d2b000
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] time(NULL) = 1010308584
[pid 6187] open(“/tmp/rt.log.6178.0”, O_WRONLY|O_APPEND|O_CREAT, 0666) = 4
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=0, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4037e000
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=0, …}) = 0
[pid 6187] _llseek(4, 0, [0], SEEK_SET) = 0
[pid 6187] fstat64(4, {st_mode=S_IFREG|0600, st_size=0, …}) = 0
[pid 6187] fcntl64(0x4, 0x2, 0x1, 0x2) = 0
[pid 6187] brk(0x8d2c000) = 0x8d2c000
[pid 6187] open(“/opt/rt2/WebRT/sessiondata/Apache-Session-d1466663eb5a952e7f9498b1c551b6ed.lock”, O_RDWR|O_CREAT|O_TRUNC, 0666) = 6
[pid 6187] fstat64(6, {st_mode=S_IFREG|0600, st_size=0, …}) = 0
[pid 6187] fcntl64(0x6, 0x2, 0x1, 0x2) = 0
[pid 6187] flock(6, LOCK_SH) = 0
[pid 6187] stat64(“/opt/rt2/WebRT/sessiondata/d1466663eb5a952e7f9498b1c551b6ed”, {st_mode=S_IFREG|0600, st_size=12523, …}) = 0
[pid 6187] open(“/opt/rt2/WebRT/sessiondata/d1466663eb5a952e7f9498b1c551b6ed”, O_RDWR|O_CREAT, 0666) = 7
[pid 6187] fcntl64(0x7, 0x3, 0xb, 0) = 2
[pid 6187] fstat64(7, {st_mode=S_IFREG|0600, st_size=12523, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4037f000
[pid 6187] llseek(7, 0, [0], SEEK_CUR) = 0
[pid 6187] fstat64(7, {st_mode=S_IFREG|0600, st_size=12523, …}) = 0
[pid 6187] fcntl64(0x7, 0x2, 0x1, 0) = 0
[pid 6187] read(7, "\3\3\0\0\0\5\n\00250X\0\0\0\25tickets_rows_per"…, 4096) = 4096
[pid 6187] brk(0x8d2d000) = 0x8d2d000
[pid 6187] brk(0x8d2e000) = 0x8d2e000
[pid 6187] read(7, “X\0\0\0\5write\10\201X\0\0\0\6publicXX\0\0\0\fEma”…, 4096) = 4096
[pid 6187] read(7, “ty\4\3\0\0\0\2\10\201X\0\0\0\4read\10\201X\0\0\0\4auto”…, 4096) = 4096
[pid 6187] brk(0x8d31000) = 0x8d31000
[pid 6187] read(7, “= 'new')) AND ((lower(main.Queue”…, 4096) = 235
[pid 6187] read(7, “”, 4096) = 0
[pid 6187] read(7, “”, 4096) = 0
[pid 6187] stat64(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/Storable/thaw.al”, {st_mode=S_IFREG|0444, st_size=640, …}) = 0
[pid 6187] getgroups32(0x20, 0xbffff230) = 1
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] open(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/Storable/thaw.al”, O_RDONLY) = 9
[pid 6187] fstat64(9, {st_mode=S_IFREG|0444, st_size=640, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x403d2000
[pid 6187] read(9, “# NOTE: Derived from blib/lib/St”…, 4096) = 640
[pid 6187] read(9, “”, 4096) = 0
[pid 6187] close(9) = 0
[pid 6187] munmap(0x403d2000, 4096) = 0
[pid 6187] brk(0x8d35000) = 0x8d35000
[pid 6187] brk(0x8d36000) = 0x8d36000
[pid 6187] brk(0x8d37000) = 0x8d37000
[pid 6187] brk(0x8d38000) = 0x8d38000
[pid 6187] brk(0x8d39000) = 0x8d39000
[pid 6187] brk(0x8d3a000) = 0x8d3a000
[pid 6187] brk(0x8d3b000) = 0x8d3b000
[pid 6187] brk(0x8d3c000) = 0x8d3c000
[pid 6187] brk(0x8d3e000) = 0x8d3e000
[pid 6187] brk(0x8d3f000) = 0x8d3f000
[pid 6187] stat64(“/opt/rt2/WebRT/html/Ticket/Display.html”, {st_mode=S_IFREG|0755, st_size=4226, …}) = 0
[pid 6187] brk(0x8d50000) = 0x8d50000
[pid 6187] brk(0x8d61000) = 0x8d61000
[pid 6187] brk(0x8d72000) = 0x8d72000
[pid 6187] brk(0x8d73000) = 0x8d73000
[pid 6187] brk(0x8d74000) = 0x8d74000
[pid 6187] brk(0x8d75000) = 0x8d75000
[pid 6187] brk(0x8d76000) = 0x8d76000
[pid 6187] brk(0x8d77000) = 0x8d77000
[pid 6187] brk(0x8d78000) = 0x8d78000
[pid 6187] brk(0x8d79000) = 0x8d79000
[pid 6187] brk(0x8d7a000) = 0x8d7a000
[pid 6187] brk(0x8d7b000) = 0x8d7b000
[pid 6187] brk(0x8d7c000) = 0x8d7c000
[pid 6187] brk(0x8d7d000) = 0x8d7d000
[pid 6187] read(3, <unfinished …>
[pid 6179] kill(6187, SIGALRM) = 0
[pid 6187] <… read resumed> 0x8cc3388, 18437) = ? ERESTARTSYS (To be restarted)
[pid 6187] — SIGALRM (Alarm clock) —
[pid 6187] write(3, “\25\3\0\0\22nJ\261z$\223:A!\316\370L\2=\234\31\242\0”, 23) = 23
[pid 6187] close(3) = 0
[pid 6187] sigreturn() = ? (mask now )
[pid 6187] — SIGSEGV (Segmentation fault) —
[pid 6187] chdir(“/etc/httpd”) = 0
[pid 6187] rt_sigaction(SIGSEGV, {SIG_DFL}, {SIG_DFL}, 8) = 0
[pid 6187] getpid() = 6187
[pid 6187] kill(6187, SIGSEGV) = 0
[pid 6187] sigreturn() = ? (mask now )
[pid 6187] — SIGSEGV (Segmentation fault) —
[pid 6179] <… wait4 resumed> [WIFSIGNALED(s) && WTERMSIG(s) == SIGSEGV], WNOHANG, NULL) = 6187

5

Hi,

This is the part where I sheepishly admit that this is where I start guessing how C and/or unix works. The process reminds me
of reconstructing an airplane crash by spreading out all the pieces in big warehouse and trying to figure out why the plane
suddenly decided to not be airworthy anymore.

Hi,

You might try running Apache under ‘strace’ (‘truss’ under Solaris)
and repeating whatever causes the segfaults. I’ve needed to do this
to resolve mysterious Apache problems before.

– Bob

Thanks for the info.
[snip]
and the strace output for

[Sun Jan 6 11:21:27 2002] [notice] child pid 6187 exit signal Segmentation fault (11)

(ummm - its pretty long).

Any ideas from anyone out there?

–Calvin

I’ve clipped some of the output and annotated it as best I can. I’m still not sure what’s going on, but I think I can see a timeout
on a socket read. I’m not sure what’s causing the segfault; it looks like process 6187 is just handling the signals it gets. (Note:
Astute readers will note my lingering FORTRAN mindset.)

[pid 6179] <… fork resumed> ) = 6187
[… watch as fork is resumed … ]
[pid 6187] open(“/etc/httpd/logs/ssl_mutex.6178”, O_WRONLY <unfinished …>
[pid 6187] <… open resumed> ) = 3
[pid 6187] fcntl64(0x3, 0, 0xf, 0x400168e4 <unfinished …>
[pid 6187] <… fcntl64 resumed> ) = 25
[pid 6187] close(3 <unfinished …>
[pid 6187] <… close resumed> ) = 0
[ … resume opening ssl_mutex.6178 on unit 3; does this really close or is it still hanging aroung? ]
[pid 6187] open(“/etc/httpd/logs/ssl_mutex.6178”, O_WRONLY <unfinished …>
[pid 6187] <… open resumed> ) = 3
[pid 6187] fcntl64(0x3, 0, 0xf, 0x400168e4 <unfinished …>
[pid 6187] <… fcntl64 resumed> ) = 25
[pid 6187] close(3 <unfinished …>
[pid 6187] <… close resumed> ) = 0
[ … resume some more … ]
[ resume a socket connection (this is now on unit 3, I think). ]
[pid 6187] select(19, [17 18], NULL, NULL, NULL <unfinished …>
[pid 6187] <… select resumed> ) = 1 (in [17])
[ yuck! what’s this “192.16192.168.34”? Typo? ]
[pid 6187] accept(17, {sin_family=AF_INET, sin_port=htons(32909), sin_addr=inet_addr(“192.16192.168.34”)}}, [16]) = 3
[pid 6187] semop(1540098, 0x819fb56, 1) = 0
[pid 6187] rt_sigaction(SIGUSR1, {SIG_IGN}, {0x80c1cac, , SA_INTERRUPT|0x4000000}, 8) = 0
[pid 6187] getsockname(3, {sin_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr(“192.168.247.6”)}}, [16]) = 0
[pid 6187] setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0
[ … DBD has trouble loading Postgres libraries (probably not a big deal); same for non-existant Kerberos libs ]
[ Apache::Session does its magic and some SQL queries occur … ]
[ Here we see the Storable module getting autoloaded ]
[pid 6187] stat64(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/Storable/thaw.al”, {st_mode=S_IFREG|0444, st_size=640, …}) = 0
[pid 6187] getgroups32(0x20, 0xbffff230) = 1
[pid 6187] rt_sigprocmask(SIG_BLOCK, NULL, , 8) = 0
[pid 6187] open(“/usr/lib/perl5/site_perl/5.6.0/i386-linux/auto/Storable/thaw.al”, O_RDONLY) = 9
[pid 6187] fstat64(9, {st_mode=S_IFREG|0444, st_size=640, …}) = 0
[pid 6187] mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x403d2000
[pid 6187] read(9, “# NOTE: Derived from blib/lib/St”…, 4096) = 640
[pid 6187] read(9, “”, 4096) = 0
[pid 6187] close(9) = 0
[ memory is freed? ]
[pid 6187] munmap(0x403d2000, 4096) = 0
[ what is all this brk() stuff? results of munmap()? ]
[pid 6187] brk(0x8d35000) = 0x8d35000
[pid 6187] brk(0x8d36000) = 0x8d36000
[pid 6187] brk(0x8d37000) = 0x8d37000
[pid 6187] brk(0x8d38000) = 0x8d38000
[pid 6187] brk(0x8d39000) = 0x8d39000
[pid 6187] brk(0x8d3a000) = 0x8d3a000
[pid 6187] brk(0x8d3b000) = 0x8d3b000
[pid 6187] brk(0x8d3c000) = 0x8d3c000
[pid 6187] brk(0x8d3e000) = 0x8d3e000
[pid 6187] brk(0x8d3f000) = 0x8d3f000
[ get file info on “/opt/rt2/WebRT/html/Ticket/Display.html” ]
[pid 6187] stat64(“/opt/rt2/WebRT/html/Ticket/Display.html”, {st_mode=S_IFREG|0755, st_size=4226, …}) = 0
[pid 6187] brk(0x8d50000) = 0x8d50000
[pid 6187] brk(0x8d61000) = 0x8d61000
[pid 6187] brk(0x8d72000) = 0x8d72000
[pid 6187] brk(0x8d73000) = 0x8d73000
[pid 6187] brk(0x8d74000) = 0x8d74000
[pid 6187] brk(0x8d75000) = 0x8d75000
[pid 6187] brk(0x8d76000) = 0x8d76000
[pid 6187] brk(0x8d77000) = 0x8d77000
[pid 6187] brk(0x8d78000) = 0x8d78000
[pid 6187] brk(0x8d79000) = 0x8d79000
[pid 6187] brk(0x8d7a000) = 0x8d7a000
[pid 6187] brk(0x8d7b000) = 0x8d7b000
[pid 6187] brk(0x8d7c000) = 0x8d7c000
[pid 6187] brk(0x8d7d000) = 0x8d7d000
[pid 6187] read(3, <unfinished …>
[ process 6179 sends a timeout signal (SIGALRM) to process 6187. Read timeout on the socket on unit 3? ]
[pid 6179] kill(6187, SIGALRM) = 0
[pid 6187] <… read resumed> 0x8cc3388, 18437) = ? ERESTARTSYS (To be restarted)
[pid 6187] — SIGALRM (Alarm clock) —
[ 6187 catches the signal, flushes output to unit 3 (open socket) and closes it ]
[pid 6187] write(3, “\25\3\0\0\22nJ\261z$\223:A!\316\370L\2=\234\31\242\0”, 23) = 23
[pid 6187] close(3) = 0
[ okay, what’s this? Does 6187 catch the SIGALRM, close up shop but fail to exit gracefully? ]
[pid 6187] sigreturn() = ? (mask now )
[ segfault on the socket close? the chdir() to “/etc/httpd”? ]
[pid 6187] — SIGSEGV (Segmentation fault) —
[pid 6187] chdir(“/etc/httpd”) = 0
[pid 6187] rt_sigaction(SIGSEGV, {SIG_DFL}, {SIG_DFL}, 8) = 0
[pid 6187] getpid() = 6187
[pid 6187] kill(6187, SIGSEGV) = 0
[pid 6187] sigreturn() = ? (mask now )
[ Let us observe a moment of silence for process 6187 ]
[pid 6187] — SIGSEGV (Segmentation fault) —
[pid 6179] <… wait4 resumed> [WIFSIGNALED(s) && WTERMSIG(s) == SIGSEGV], WNOHANG, NULL) = 6187
[ … and life goes on for 6179 … ]

My big question is where the SIGALRM is coming from within process 6179. Also, I’m curious about the
‘inet_addr(“192.16192.168.34”)’ call – this looks gruesome unless my mail has been getting mangled in transit.

hth,

– Bob

6

Hi,

This is the part where I sheepishly admit that this is where I start guessing how C and/or unix works. The process reminds me
of reconstructing an airplane crash by spreading out all the pieces in big warehouse and trying to figure out why the plane
suddenly decided to not be airworthy anymore.

Grin - in the same boat as you here.

what bugs me is that I have 10 or so other people who can
use rt fine, and this seems to be to be related to
just my user login.
Looking at the trace, I’d suspect the database, but I’ve
rebuilt indexes, vacuumed and done all the things that normally
you’d normally do to address that.

I’m going to setup 2.0.11 from scratch, re-import the database into
that and see where that takes me.

Maybe its a postgresql related thing?
(the trace doesn’t reveal enough of the actual queries though :frowning: ).

–Calvin

------------------------* My opinions are mine *------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

7

Just for kicks, what’s the breakdown of browser for working
users / failing users?On Sun, Jan 06, 2002 at 10:16:15PM +0200, Calvin Browne wrote:

Hi,

This is the part where I sheepishly admit that this is where I start guessing how C and/or unix works. The process reminds me
of reconstructing an airplane crash by spreading out all the pieces in big warehouse and trying to figure out why the plane
suddenly decided to not be airworthy anymore.

On 6 Jan 2002, at 12:31, Calvin Browne wrote:

Grin - in the same boat as you here.

what bugs me is that I have 10 or so other people who can
use rt fine, and this seems to be to be related to
just my user login.
Looking at the trace, I’d suspect the database, but I’ve
rebuilt indexes, vacuumed and done all the things that normally
you’d normally do to address that.

I’m going to setup 2.0.11 from scratch, re-import the database into
that and see where that takes me.

Maybe its a postgresql related thing?
(the trace doesn’t reveal enough of the actual queries though :frowning: ).

–Calvin

------------------------* My opinions are mine *------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

http://www.bestpractical.com/products/rt – Trouble Ticketing. Free.

8

This sounds similar to a problem I had. Netscape6/mozilla has a problem
submitting forms that had an empty file input field. I know the problem
is fixed in the current mozilla nightly builds.On Sun, 2002-01-06 at 21:40, Jesse Vincent wrote:

Just for kicks, what’s the breakdown of browser for working
users / failing users?

On Sun, Jan 06, 2002 at 10:16:15PM +0200, Calvin Browne wrote:

Hi,

This is the part where I sheepishly admit that this is where I start guessing how C and/or unix works. The process reminds me
of reconstructing an airplane crash by spreading out all the pieces in big warehouse and trying to figure out why the plane
suddenly decided to not be airworthy anymore.

On 6 Jan 2002, at 12:31, Calvin Browne wrote:

Grin - in the same boat as you here.

what bugs me is that I have 10 or so other people who can
use rt fine, and this seems to be to be related to
just my user login.
Looking at the trace, I’d suspect the database, but I’ve
rebuilt indexes, vacuumed and done all the things that normally
you’d normally do to address that.

I’m going to setup 2.0.11 from scratch, re-import the database into
that and see where that takes me.

Maybe its a postgresql related thing?
(the trace doesn’t reveal enough of the actual queries though :frowning: ).

–Calvin

------------------------* My opinions are mine *------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

–
http://www.bestpractical.com/products/rt – Trouble Ticketing. Free.

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

9

Just for kicks, what’s the breakdown of browser for working
users / failing users?

Ok, gotit - I happened to be the only person using mozilla 0.9.7.

The best part about banging your head against a brick wall is the
feeling you get when you stop.

Other browsers I’ve tried work fine (There were other segfaults which
were resolved by re-compiling apache without the DSO stuff).

If I load mozilla 0.9.7 on other machines with other users, I
get the exact same results (These are the redhat 7.x rpm’s
from the mozilla site).

I’m not sure if this is limited to just SSL connections (we
only run rt over SSL) and will leave that to someone else to test.

{truth be told, I should have made the connection earlier,
but I’ve upgraded from 7.0 to 7.2 on my notebook, and changed wi-fi
cards and a whole lot of other things over the last 2 week sigh}

Thanks for all the e-mail and help, its muchly appreciated.

–Calvin

------------------------* My opinions are mine *------------------------------
Calvin Browne calvin@UniForum.org.za c@lvin.co.za
Office phone: 080 314 0077 +27 11 314-0077
http://orange-tree.alt.za Mobile: +27 83 303-0663
Call me for Linux/Internet consulting

10

Get a night built of Mozilla, this issue has been long resolved (check
list archives, there’s a post about Mozilla 0.9.7 problems from me).

A.

Calvin Browne wrote:

11

Sorry for cc’ing so many folk, but I’ve been having problems submitting to the
list (i’m subscribed to the digest), so hopefully at least some people will see
this…

Calvin’s strace output looked strikingly similar to a problem I saw last week
with one of my users, which I’d put on the backburner (in the ‘too hard right
now’ basket). Jesses’ last comment on browser types also struck home…