Mod_FastCgi and RT

RT Users
1

I have a peculiar situation which crept up only a few weeks ago and I
have been able to narrow it down to RT + mod_FastCgi.

About 3 weeks ago, I updated to the latest FC2 kernel and a number of
other nondescript packages. Around the same time, I started
experiencing, on a nightly basis around 4:30am, where the web server
would be unresponsive. The server was running, but not handling any
requests. Restarting the web server would bring it back on-line until
the next evening.

I’ll omit all of the debugging here as it would not be relevant, but
suffice it to say that at 4:20am, logrotate would come along and kill
-USR1 httpd after rotating the logs, which was the same time that things
tanked out. Sounds like an Apache problem, right?

Well, I dug further, and recompiled the custom modules that we utilize
(mod_auth_ldap2 and mod_fastcgi) to use the newer version of the web
server. We’re pretty stuck on mod_fastcgi 2.4.0 because 2.4.2 does not
support running FastCgiServer processes within VirtualHosts (who thought
disabling that was a good idea?!).

Long story short, it turns out that, upon startup, if a FastCGI server
is configured to run, most of the httpd processes are running as root
(bad) and the remaining 2 or 3 are running as the apache user (not
including the parent process, of course). In the situations where the
server was unresponsive from 4:20am, all of the processes were running
as root. So, if all of the processes were running as root, the web
server is completely unresponsive.

So, if I disable the FastCgiServer processes (just commenting them out)
and start up the server, everything comes up just fine… If I have the
FastCgiServer processes enabled, then 2 or 3 of forked web server
processes run as apache, the rest run as root…

Is this just us?! Could this be kernel related? I’ve personally been
using RT with mod_fastcgi for ages… it is just plain odd that it has
tanked out on us like this.

I’m thinking of moving back from mod_fastcgi to mod_perl…

-Rich

2

Well, I dug further, and recompiled the custom modules that we utilize
(mod_auth_ldap2 and mod_fastcgi) to use the newer version of the web
server. We’re pretty stuck on mod_fastcgi 2.4.0 because 2.4.2 does not
support running FastCgiServer processes within VirtualHosts (who thought
disabling that was a good idea?!).

I’m running a current FC2 with mod_fastcgi 2.4.2 (which I think came
from (http://www.wesmo.com/redhat/i386/mod_fastcgi-2.4.2-1.i386.rpm),
thanks, and it is running in a virtualhost with no problem.

So, if I disable the FastCgiServer processes (just commenting them out)
and start up the server, everything comes up just fine… If I have the
FastCgiServer processes enabled, then 2 or 3 of forked web server
processes run as apache, the rest run as root…

Oh - you have more than one RT instance… What happens if you only
run one? My FastCgiServer statement is in the global section but the
scriptalias and AddHandler that invoke it are in a virtualhost.

Is this just us?! Could this be kernel related? I’ve personally been
using RT with mod_fastcgi for ages… it is just plain odd that it has
tanked out on us like this.

You may be the only one running multiple instances of RT in one httpd.

I’m thinking of moving back from mod_fastcgi to mod_perl…

That will be even worse - you can’t load different versions of the same
libraries at the same time in mod_perl. The only clean way to do it is
to run multiple httpd instances. You can hide the fact that you’ve
done that by moving the ones running mod_perl to different ports and
running an non mod_perl httpd on port 80 using ProxyPass or RewriteRule
and ProxyPassReverse to dispatch requests to the correct server.

Les Mikesell
les@futuresource.com

3

I’m running a current FC2 with mod_fastcgi 2.4.2 (which I think came
from (http://www.wesmo.com/redhat/i386/mod_fastcgi-2.4.2-1.i386.rpm),
thanks, and it is running in a virtualhost with no problem.

Heh… :slight_smile: I’ve been running with mod_fastcgi for some time now… It has
actually worked out rather well. Hence my frustration with things
suddenly misbehaving.

Oh - you have more than one RT instance… What happens if you only
run one? My FastCgiServer statement is in the global section but the
scriptalias and AddHandler that invoke it are in a virtualhost.

It seems to happen if I run one or many. I am beginning to suspect that
one of the upgraded packages that I installed (via ‘yum’) must’ve hosed
something. I am half tempted to set up a dummy server running FC3 and
see how it behaves.

You may be the only one running multiple instances of RT in one httpd.

Probably…

That will be even worse - you can’t load different versions of the same
libraries at the same time in mod_perl. The only clean way to do it is
to run multiple httpd instances. You can hide the fact that you’ve
done that by moving the ones running mod_perl to different ports and
running an non mod_perl httpd on port 80 using ProxyPass or RewriteRule
and ProxyPassReverse to dispatch requests to the correct server.

Yup, that proved to be even worse. :slight_smile: mod_perl works well for single
installations, but it really does not play well with VirtualHosts.

I’m going to try attacking this from the system side.

Thanks for the response! Knowing that I wasn’t losing my marbles did
help a lot. :slight_smile:

-Rich

4

I did manage to fix the problem. It seems that it was directly related
to the load average and the heavy load on the MySQL server caused by
spamassassin and it’s overly HUGE bayes database.

It seems that, because of the load and delays, apache wasn’t able to
properly fork and change ownership of the process in some mystical
timeperiod. Because it took so long, the parent process considered that
child defunct, and just let it sit there.

Once I disabled the MySQL bayes database for SpamAssassin, everything
else started working properly.

Thanks for all of those who offered their ideas and comments. I’m
happily back to running multiple RT instances with mod_fastcgi. :slight_smile:

-Rich