RT creates CLOSE_WAITS

Hi all I have been struggling with this CLOSE_WAITS.

I have an Request Tracker installed on CEntOS 6.7 (64 bit) RT rt-4.2.12 and Apache httpd-2.2.31. I can log in and all, but after a while I am seeing CLOSE_WAIT socket are created for each selection that I make on the dashboard. Eventually the system goes unresponsive.

root@iplrt logs]# netstat -nalp | grep -i wait
tcp 1 0 192.168.11.254:80 192.168.11.118:57076 CLOSE_WAIT 3176/httpd
tcp 1 0 192.168.11.254:80 192.168.11.118:57083 CLOSE_WAIT 3177/httpd
tcp 1 0 192.168.11.254:80 192.168.11.118:57082 CLOSE_WAIT 3185/httpd
tcp 1 0 192.168.11.254:80 192.168.11.118:57084 CLOSE_WAIT 3365/httpd
[root@iplrt logs]#

I don’t see any errors in the logs! I have downgraded both RT and Apache with no luck and I am STUCK no ware to goooooo

Can some one tell me how to go about diagnosing this

Thanks and Regards

Hi all I have been struggling with this CLOSE_WAITS.

I have an Request Tracker installed on CEntOS 6.7 (64 bit) RT rt-4.2.12 and Apache httpd-2.2.31. I can log in and all, but after a while I am seeing CLOSE_WAIT socket are created for each selection that I make on the dashboard. Eventually the system goes unresponsive.

What does the console (VC1) say?

Are you running systemd? What about the journal?

-m

Hi all I have been struggling with this CLOSE_WAITS.

I have an Request Tracker installed on CEntOS 6.7 (64 bit) RT
rt-4.2.12 and Apache httpd-2.2.31. I can log in and all, but after a
while I am seeing CLOSE_WAIT socket are created for each selection
that I make on the dashboard. Eventually the system goes unresponsive.

root@iplrt logs]# netstat -nalp | grep -i wait
tcp 1 0 192.168.11.254:80 192.168.11.118:57076 CLOSE_WAIT 3176/httpd
tcp 1 0 192.168.11.254:80 192.168.11.118:57083 CLOSE_WAIT 3177/httpd
tcp 1 0 192.168.11.254:80 192.168.11.118:57082 CLOSE_WAIT 3185/httpd
tcp 1 0 192.168.11.254:80 192.168.11.118:57084 CLOSE_WAIT 3365/httpd
[root@iplrt logs]#

I don’t see any errors in the logs! I have downgraded both RT and
Apache with no luck and I am STUCK no ware to goooooo

Can some one tell me how to go about diagnosing this

What interface are you using to run RT via Apache? (Possible answers:
mod_fastcgi, mod_fcgid, mod_perl 1.x, mod_perl 2.x)

That is important because a CLOSE_WAIT state indicates that the OS is
waiting on a local process (in this case multiple Apache httpd
processes) to recognize that the socket has been closed on the other end
and call close() itself. The Apache core doesn’t have that problem much,
so the issue is most likely lurking in whatever module is loaded into
the httpd process to handle running RT proper.

(my bet is that you’re running a slightly broken mod_perl, but I’m
biased by bad experiences…)

Hi thank you for the reply, I am using mod_perl
2.0.9

Best RegardsSubject: Re: [rt-users] RT creates CLOSE_WAITS
To: rt-users@lists.bestpractical.com
Date: Thursday, 29 October, 2015, 2:05

Hi
all I have been struggling with this CLOSE_WAITS.

I have an Request
Tracker installed on CEntOS 6.7 (64 bit) RT
rt-4.2.12 and Apache httpd-2.2.31. I can
log in and all, but after a
while I am
seeing CLOSE_WAIT socket are created for each selection
that I make on the dashboard. Eventually
the system goes unresponsive.

root@iplrt logs]# netstat
-nalp | grep -i wait
tcp 1 0
192.168.11.254:80 192.168.11.118:57076 CLOSE_WAIT
3176/httpd
tcp 1 0 192.168.11.254:80
192.168.11.118:57083 CLOSE_WAIT 3177/httpd
tcp 1 0 192.168.11.254:80
192.168.11.118:57082 CLOSE_WAIT 3185/httpd
tcp 1 0 192.168.11.254:80
192.168.11.118:57084 CLOSE_WAIT 3365/httpd
[root@iplrt logs]#

I don’t see any
errors in the logs! I have downgraded both RT and
Apache with no luck and I am STUCK no ware
to goooooo

Can
some one tell me how to go about diagnosing this

What interface are you using
to run RT via Apache? (Possible answers:
mod_fastcgi, mod_fcgid, mod_perl 1.x, mod_perl
2.x)

That is important
because a CLOSE_WAIT state indicates that the OS is
waiting on a local process (in this case
multiple Apache httpd
processes) to
recognize that the socket has been closed on the other end

and call close() itself. The Apache core
doesn’t have that problem much,
so the
issue is most likely lurking in whatever module is loaded
into
the httpd process to handle running RT
proper.

(my bet is that
you’re running a slightly broken mod_perl, but I’m

biased by bad experiences…)

Hi thank you for the reply, I am using mod_perl
2.0.9

I suggest switching to mod_fastcgi or mod_fcgid as a first step. I use
mod_fastcgi because my RT instances all have long legacies, but on a
newly-built system mod_fcgid is probably a better choice because it is
still being maintained.

I don’t suggest that because I know of a specific problem in the current
RT & mod_perl, but because 3 times over many years & RT versions I’ve
tried to stand up fresh RT instances using mod_perl (because it should
in principle be more efficient) and every time I’ve retreated to FastCGI
after running into some problem I couldn’t figure out and which simply
didn’t happen with FastCGI. I know a lot of people run RT with mod_perl
so it CAN be done, but when it doesn’t work the diagnostic process is
daunting.

If there’s some reason you really want to stick with mod_perl, you
should investigate troubleshooting tactics for mod_perl, which include a
way to load it with a Perl debugger.

Hi Bill, thank you for the reply and I was at a conner and was scratching my head about. Thanks to you I have something to work with

Thanks and Best Regards

AsankaSubject: Re: [rt-users] RT creates CLOSE_WAITS
To: rt-users@lists.bestpractical.com
Date: Saturday, 31 October, 2015, 3:34

Hi
thank you for the reply, I am using mod_perl
2.0.9

I suggest switching to mod_fastcgi or mod_fcgid
as a first step. I use
mod_fastcgi because
my RT instances all have long legacies, but on a
newly-built system mod_fcgid is probably a
better choice because it is
still being
maintained.

I don’t
suggest that because I know of a specific problem in the
current
RT & mod_perl, but because 3
times over many years & RT versions I’ve
tried to stand up fresh RT instances using
mod_perl (because it should
in principle be
more efficient) and every time I’ve retreated to FastCGI

after running into some problem I
couldn’t figure out and which simply
didn’t happen with FastCGI. I know a lot of
people run RT with mod_perl
so it CAN be
done, but when it doesn’t work the diagnostic process is

daunting.

If there’s some reason you really want to
stick with mod_perl, you
should investigate
troubleshooting tactics for mod_perl, which include a
way to load it with a Perl debugger.