RT 3.8 and FastCGI out of memory via http but not https

I had RT 3.6.7 installed on a Debian Etch machine, and all was well. I
decided to upgrade to RT 3.8.2, and I upgraded the machine from Etch to
Lenny (making sure to clear out old custom-built Perl module packages).
I rebuilt all the Perl dependencies missing from the Debian
repositories, and upgraded my RT install. Now I have a strange
issue…hitting RT 3.8 via HTTP results in an “Out of memory!” error in
Apache’s error.log, but visiting via HTTPS works fine. Another RT 3.6.7
instance on this box works fine with both.

Apache vhost config (slightly edited for privacy on things like cert and
site names):
<VirtualHost 1.2.3.4:80 127.0.1.1:80>
ServerAdmin netadmin
ServerName rt
DocumentRoot /var/www/
<Directory “/home/rt/3.8_rt”>
Options FollowSymLinks ExecCGI
AllowOverride None

ScriptAlias /3.8 /home/rt/3.8_rt/bin/mason_handler.fcgi
<Location /3.8>
RewriteEngine On
RedirectMatch Permanent /3.8$ $1/3.8/

<Directory “/home/rt/3.6_rt”>
Options FollowSymLinks ExecCGI
AllowOverride None

ScriptAlias /3.6 /home/rt/3.6_rt/bin/mason_handler.fcgi
<Location /3.6>
RewriteEngine On
RedirectMatch Permanent /3.6$ $1/3.6/

<VirtualHost 1.2.3.4:443 127.0.1.1:443>
ServerAdmin netadmin
ServerName rt
DocumentRoot /var/www/
<Directory “/home/rt/3.8_rt”>
Options FollowSymLinks ExecCGI
AllowOverride None

ScriptAlias /3.8 /home/rt/3.8_rt/bin/mason_handler.fcgi
<Location /3.8>
RewriteEngine On
RedirectMatch Permanent /3.8$ $1/3.8/

<Directory “/home/rt/3.6”>
Options FollowSymLinks ExecCGI
AllowOverride None

ScriptAlias /3.6 /home/rt/3.6_rt/bin/mason_handler.fcgi
<Location /3.6>
RewriteEngine On
RedirectMatch Permanent /3.6$ $1/3.6/

SSLEngine On
SSLCertificateKeyFile /etc/ssl/private/rt.key
SSLCertificateFile /etc/ssl/certs/rt.crt
SSLCACertificateFile /etc/ssl/certs/intermediate.crt
SetEnvIf User-Agent “.MSIE.” nokeepalive ssl-unclean-shutdown
downgrade-1.0 force-response-1.0

Apache2 error.log excerpt:
[Tue Feb 24 10:29:15 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" started (pid 5872)
[Tue Feb 24 10:29:18 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" started (pid 5875)
Out of memory!
[Tue Feb 24 10:29:19 2009] [error] [client 10.202.35.3] FastCGI:
incomplete headers (0 bytes) received from server
"/home/rt/3.8_rt/bin/mason_handler.fcgi"
[Tue Feb 24 10:29:20 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" (pid 5872) terminated by
calling exit with status '1’
Out of memory!
[Tue Feb 24 10:31:26 2009] [error] [client 10.202.35.3] FastCGI:
incomplete headers (0 bytes) received from server
"/home/rt/3.8_rt/bin/mason_handler.fcgi"
[Tue Feb 24 10:31:26 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" (pid 5875) terminated by
calling exit with status ‘1’
[Tue Feb 24 10:31:26 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" restarted (pid 5898)
Out of memory!
[Tue Feb 24 10:31:30 2009] [error] [client 10.202.35.3] FastCGI:
incomplete headers (0 bytes) received from server
"/home/rt/3.8_rt/bin/mason_handler.fcgi"
[Tue Feb 24 10:31:30 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" (pid 5898) terminated by
calling exit with status ‘1’
[Tue Feb 24 10:31:31 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" restarted (pid 5902)
Out of memory!
[Tue Feb 24 10:31:38 2009] [error] [client 10.202.35.3] FastCGI:
incomplete headers (0 bytes) received from server
"/home/rt/3.8_rt/bin/mason_handler.fcgi"
[Tue Feb 24 10:31:38 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" (pid 5902) terminated by
calling exit with status ‘1’
[Tue Feb 24 10:31:38 2009] [warn] FastCGI: (dynamic) server
"/home/rt/3.8_rt/bin/mason_handler.fcgi" has failed to remain running
for 30 seconds given 3 attempts, its restart interval has been backed
off to 600 seconds

fastcgi.conf:

AddHandler fastcgi-script .fcgi
FastCgiIpcDir /tmp

Oh, and here’s what the system memory looks like:
total used free shared buffers
cached
Mem: 386252 279420 106832 0 45484
103824
-/+ buffers/cache: 130112 256140
Swap: 1044184 88 1044096

Any thoughts? Any other logs or config info that could be useful in
troubleshooting this?

Steve Christou
UWM Information Security
Phone: 414-229-1100
Email: Osa-list@uwm.edu

Re: [rt-users] RT 3.8 and FastCGI out of memory via http but not https.eml (6.65 KB)

Changing the a_sessions table from longtext to longblob fixed it.
Thanks, Steven! Now that I know the answer I’ve found the other threads
relating to this, but I was apparently using poor search terms before :slight_smile:

Changing the a_sessions table from longtext to longblob fixed it.

Shouldn’t the upgrade scripts for “rt-setup-database --action upgrade” take
care of this? Seems weird that it doesn’t.

– ============================
Tom Lahti
BIT Statement LLC

(425)251-0833 x 117
http://www.bitstatement.net/
– ============================

Changing the a_sessions table from longtext to longblob fixed it.

Shouldn’t the upgrade scripts for “rt-setup-database --action upgrade” take
care of this? Seems weird that it doesn’t.

The schema changes necessary for this “major” upgrade were deemed too
invasive (and slow) to automate. UPGRADING.somethingorother talks you
through the update.

You MUST read UPGRADING.mysqlOn Wed, Feb 25, 2009 at 1:33 AM, Nick Kartsioukas change+lists.rt@nightwind.net wrote:

Changing the a_sessions table from longtext to longblob fixed it.
Thanks, Steven! Now that I know the answer I’ve found the other threads
relating to this, but I was apparently using poor search terms before :slight_smile:


http://lists.bestpractical.com/cgi-bin/mailman/listinfo/rt-users

Community help: http://wiki.bestpractical.com
Commercial support: sales@bestpractical.com

Discover RT’s hidden secrets with RT Essentials from O’Reilly Media.
Buy a copy at http://rtbook.bestpractical.com

Best regards, Ruslan.

On Wed, 25 Feb 2009 16:26:49 +0300, "Ruslan Zakirov"
ruslan.zakirov@gmail.com said:

You MUST read UPGRADING.mysql

This was not obvious from the wiki documentation or output from “make
upgrade” or the rt-upgrade-database script.

On Wed, 25 Feb 2009 16:26:49 +0300, "Ruslan Zakirov"
ruslan.zakirov@gmail.com said:

You MUST read UPGRADING.mysql

This was not obvious from the wiki documentation or output from “make
upgrade” or the rt-upgrade-database script.

It is covered in the README file and the UPGRADING file

-kevin