Help - memory leak problems

RT Users
1

Hi all

I recently installed RT 3.0.4 (and later RT 3.0.6) using apache 1.3.28 and
mod_perl 1.29 statically linked.
I followed all the installation instructions and have installed it on a Solaris
2.6 as well as a Redhat Linux 9
box but with the same problem occuring on both.
The problem appears to be some sort of memory leak where memory is allocated but
is never released.
The size of the httpd processes grow after every request.
The end result is that all RAM is consumed and after a while all swap is also
exhausted.

Below are samples taken of the relevant processes (top output - sorted by memory
usage) while a dummy load is applied.
The load is generated by multiple web clients with a URL similar to the
following
http://host.domain.name/index.html?HomeRefreshInterval=2 (Refresh the home page
every 2 seconds - too speed things up a bit)

As you’ll see the size of the processes grow and eventually all memory is used.
The only thing that I have found to help is to set MaxRequestsPerChild=1 in
apache config,
but performance is really bad as after each request is handled that apache child
is killed and another
is restarted in it’s place. I also have to limit MinSpareServers and
MaxSpareServers to about 4 with
MaxClients also set to 4.

I have searched through the mailing list archives but I don’t see anybody else
with
the same problem (some are similar but I don’t seem to find a solution)

I would really appreciate any help you could give me.

22:00 sample one after start up - before first request
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12814 root 19 0 21200 20M 2744 S 0.0 6.6 0:00 0
httpd
12815 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12816 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12817 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12818 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12819 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
2442 mysql 15 0 20892 3636 1448 S 0.0 1.1 1:37 0
mysqld

23:40 Sample two after running for 1h:40m
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12671 nobody 16 0 35780 32M 2304 S 0.0 10.4 1:55 0
httpd
12670 nobody 16 0 34204 31M 2176 S 8.9 10.2 1:14 0
httpd
12675 nobody 15 0 34356 31M 2384 S 2.9 10.1 1:11 0
httpd
12673 nobody 16 0 34968 31M 2360 S 25.7 10.1 1:03 0
httpd
12669 nobody 15 0 34684 31M 2096 S 3.9 10.1 1:00 0
httpd
12672 nobody 15 0 32696 29M 2440 S 3.9 9.5 0:19 0
httpd
12708 nobody 16 0 22988 22M 2596 S 0.0 7.1 0:01 0
httpd
12668 root 15 0 19320 18M 864 S 0.0 6.0 0:00
0 httpd
2442 mysql 16 0 20980 5028 1928 S 0.9 1.5 0:22 0
mysqld

23:50 Sample three 10 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12671 nobody 15 0 43336 28M 1976 S 0.0 8.9 2:47 0
httpd
12670 nobody 16 0 41644 27M 1832 S 0.0 8.8 2:04 0
httpd
12669 nobody 16 0 42352 27M 1844 S 0.0 8.8 1:50 0
httpd
12673 nobody 15 0 42492 27M 2032 S 0.0 8.7 1:56 0
httpd
12675 nobody 16 0 41792 27M 2004 S 0.0 8.7 2:03 0
httpd
12672 nobody 16 0 40188 25M 2120 S 0.0 8.1 1:08 0
httpd
12708 nobody 16 0 30452 19M 2276 S 0.0 6.3 0:53 0
httpd
12726 nobody 16 0 27844 19M 2668 S 0.0 6.2 0:33 0
httpd
12716 nobody 16 0 29492 19M 2628 S 0.0 6.1 0:44 0
httpd
2442 mysql 15 0 21084 3884 1704 S 0.0 1.2 0:42 0
mysqld

00:10 Sample four 20 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12726 nobody 16 0 35996 24M 2668 S 0.0 7.9 1:21 0 httpd
12675 nobody 16 0 50200 23M 2012 S 2.9 7.6 2:57 0 httpd
12673 nobody 16 0 50912 23M 2020 S 24.7 7.5 2:53 0 httpd
12669 nobody 15 0 50568 23M 1844 S 2.9 7.5 2:40 0 httpd
12672 nobody 16 0 48576 23M 2108 S 0.0 7.5 2:05 0 httpd
12742 nobody 16 0 30904 23M 2644 S 12.8 7.5 0:55 0 httpd
12670 nobody 16 0 49984 23M 1844 S 3.9 7.4 2:59 0 httpd
12716 nobody 15 0 37728 23M 2640 S 0.0 7.3 1:33 0 httpd
12671 nobody 16 0 51480 23M 1976 S 0.0 7.3 3:38 0 httpd
12708 nobody 16 0 38944 22M 2276 S 6.9 7.3 1:55 0 httpd
2442 mysql 15 0 20948 3672 1500 S 0.0 1.1 1:06 0
mysqld

00:30 Sample five 20 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12672 nobody 16 0 57480 25M 1848 S 5.9 8.0 3:10 0 httpd
12708 nobody 16 0 47636 24M 2008 S 0.0 7.9 2:53 0 httpd
12675 nobody 16 0 59004 24M 1752 S 22.7 7.9 3:55 0 httpd
12669 nobody 15 0 59332 24M 1576 S 0.0 7.9 3:38 0 httpd
12716 nobody 16 0 46576 24M 2368 S 0.0 7.9 2:30 0 httpd
12742 nobody 16 0 39720 24M 2376 S 0.0 7.9 1:54 0 httpd
12673 nobody 16 0 59572 24M 1752 R 1.9 7.8 3:51 0 httpd
12671 nobody 16 0 60152 24M 1716 R 17.8 7.8 4:34 0 httpd
12726 nobody 16 0 44824 24M 2408 S 2.9 7.7 2:23 0 httpd
12670 nobody 15 0 58692 23M 1584 S 4.9 7.6 3:53 0 httpd
2442 mysql 15 0 21024 3724 1492 S 1.9 1.1 1:33 0
mysqld

Regards
Doug
This email was scanned by ICL’s anti-virus solution at http://www.eject.co.za

2

Do you still see this issue if you turn off Apache::DBI?On Fri, Oct 17, 2003 at 10:19:28AM +0200, douglas.borrie@eject.co.za wrote:

Hi all

I recently installed RT 3.0.4 (and later RT 3.0.6) using apache 1.3.28 and
mod_perl 1.29 statically linked.
I followed all the installation instructions and have installed it on a Solaris
2.6 as well as a Redhat Linux 9
box but with the same problem occuring on both.
The problem appears to be some sort of memory leak where memory is allocated but
is never released.
The size of the httpd processes grow after every request.
The end result is that all RAM is consumed and after a while all swap is also
exhausted.

Below are samples taken of the relevant processes (top output - sorted by memory
usage) while a dummy load is applied.
The load is generated by multiple web clients with a URL similar to the
following
http://host.domain.name/index.html?HomeRefreshInterval=2 (Refresh the home page
every 2 seconds - too speed things up a bit)

As you’ll see the size of the processes grow and eventually all memory is used.
The only thing that I have found to help is to set MaxRequestsPerChild=1 in
apache config,
but performance is really bad as after each request is handled that apache child
is killed and another
is restarted in it’s place. I also have to limit MinSpareServers and
MaxSpareServers to about 4 with
MaxClients also set to 4.

I have searched through the mailing list archives but I don’t see anybody else
with
the same problem (some are similar but I don’t seem to find a solution)

I would really appreciate any help you could give me.

22:00 sample one after start up - before first request
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12814 root 19 0 21200 20M 2744 S 0.0 6.6 0:00 0
httpd
12815 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12816 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12817 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12818 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
12819 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0 httpd
2442 mysql 15 0 20892 3636 1448 S 0.0 1.1 1:37 0
mysqld

23:40 Sample two after running for 1h:40m
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12671 nobody 16 0 35780 32M 2304 S 0.0 10.4 1:55 0
httpd
12670 nobody 16 0 34204 31M 2176 S 8.9 10.2 1:14 0
httpd
12675 nobody 15 0 34356 31M 2384 S 2.9 10.1 1:11 0
httpd
12673 nobody 16 0 34968 31M 2360 S 25.7 10.1 1:03 0
httpd
12669 nobody 15 0 34684 31M 2096 S 3.9 10.1 1:00 0
httpd
12672 nobody 15 0 32696 29M 2440 S 3.9 9.5 0:19 0
httpd
12708 nobody 16 0 22988 22M 2596 S 0.0 7.1 0:01 0
httpd
12668 root 15 0 19320 18M 864 S 0.0 6.0 0:00
0 httpd
2442 mysql 16 0 20980 5028 1928 S 0.9 1.5 0:22 0
mysqld

23:50 Sample three 10 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12671 nobody 15 0 43336 28M 1976 S 0.0 8.9 2:47 0
httpd
12670 nobody 16 0 41644 27M 1832 S 0.0 8.8 2:04 0
httpd
12669 nobody 16 0 42352 27M 1844 S 0.0 8.8 1:50 0
httpd
12673 nobody 15 0 42492 27M 2032 S 0.0 8.7 1:56 0
httpd
12675 nobody 16 0 41792 27M 2004 S 0.0 8.7 2:03 0
httpd
12672 nobody 16 0 40188 25M 2120 S 0.0 8.1 1:08 0
httpd
12708 nobody 16 0 30452 19M 2276 S 0.0 6.3 0:53 0
httpd
12726 nobody 16 0 27844 19M 2668 S 0.0 6.2 0:33 0
httpd
12716 nobody 16 0 29492 19M 2628 S 0.0 6.1 0:44 0
httpd
2442 mysql 15 0 21084 3884 1704 S 0.0 1.2 0:42 0
mysqld

00:10 Sample four 20 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12726 nobody 16 0 35996 24M 2668 S 0.0 7.9 1:21 0 httpd
12675 nobody 16 0 50200 23M 2012 S 2.9 7.6 2:57 0 httpd
12673 nobody 16 0 50912 23M 2020 S 24.7 7.5 2:53 0 httpd
12669 nobody 15 0 50568 23M 1844 S 2.9 7.5 2:40 0 httpd
12672 nobody 16 0 48576 23M 2108 S 0.0 7.5 2:05 0 httpd
12742 nobody 16 0 30904 23M 2644 S 12.8 7.5 0:55 0 httpd
12670 nobody 16 0 49984 23M 1844 S 3.9 7.4 2:59 0 httpd
12716 nobody 15 0 37728 23M 2640 S 0.0 7.3 1:33 0 httpd
12671 nobody 16 0 51480 23M 1976 S 0.0 7.3 3:38 0 httpd
12708 nobody 16 0 38944 22M 2276 S 6.9 7.3 1:55 0 httpd
2442 mysql 15 0 20948 3672 1500 S 0.0 1.1 1:06 0
mysqld

00:30 Sample five 20 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12672 nobody 16 0 57480 25M 1848 S 5.9 8.0 3:10 0 httpd
12708 nobody 16 0 47636 24M 2008 S 0.0 7.9 2:53 0 httpd
12675 nobody 16 0 59004 24M 1752 S 22.7 7.9 3:55 0 httpd
12669 nobody 15 0 59332 24M 1576 S 0.0 7.9 3:38 0 httpd
12716 nobody 16 0 46576 24M 2368 S 0.0 7.9 2:30 0 httpd
12742 nobody 16 0 39720 24M 2376 S 0.0 7.9 1:54 0 httpd
12673 nobody 16 0 59572 24M 1752 R 1.9 7.8 3:51 0 httpd
12671 nobody 16 0 60152 24M 1716 R 17.8 7.8 4:34 0 httpd
12726 nobody 16 0 44824 24M 2408 S 2.9 7.7 2:23 0 httpd
12670 nobody 15 0 58692 23M 1584 S 4.9 7.6 3:53 0 httpd
2442 mysql 15 0 21024 3724 1492 S 1.9 1.1 1:33 0
mysqld

Regards
Doug
This email was scanned by ICL’s anti-virus solution at http://www.eject.co.za

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

3

This was my previous config

ServerName host.domain.name DocumentRoot /opt/rt3/share/html AddDefaultCharset UTF-8 
    PerlModule Apache::DBI
    PerlRequire /opt/rt3/bin/webmux.pl

    <Location />
            SetHandler perl-script
            PerlHandler RT::Mason
    </Location>

I changed it to this (is this what you meant by turning off Apache::DBI?)

ServerName host.domain.name DocumentRoot /opt/rt3/share/html AddDefaultCharset UTF-8 
    PerlRequire /opt/rt3/bin/webmux.pl

    <Location />
            SetHandler perl-script
            PerlHandler RT::Mason
    </Location>

If that is what you meant, then yes the issue is still there.

Quoting Jesse Vincent jesse@bestpractical.com:

Do you still see this issue if you turn off Apache::DBI?

Hi all

I recently installed RT 3.0.4 (and later RT 3.0.6) using apache 1.3.28 and
mod_perl 1.29 statically linked.
I followed all the installation instructions and have installed it on a
Solaris
2.6 as well as a Redhat Linux 9
box but with the same problem occuring on both.
The problem appears to be some sort of memory leak where memory is
allocated but
is never released.
The size of the httpd processes grow after every request.
The end result is that all RAM is consumed and after a while all swap is
also
exhausted.

Below are samples taken of the relevant processes (top output - sorted by
memory
usage) while a dummy load is applied.
The load is generated by multiple web clients with a URL similar to the
following
http://host.domain.name/index.html?HomeRefreshInterval=2 (Refresh the home
page
every 2 seconds - too speed things up a bit)

As you’ll see the size of the processes grow and eventually all memory is
used.
The only thing that I have found to help is to set MaxRequestsPerChild=1
in
apache config,
but performance is really bad as after each request is handled that apache
child
is killed and another
is restarted in it’s place. I also have to limit MinSpareServers and
MaxSpareServers to about 4 with
MaxClients also set to 4.

I have searched through the mailing list archives but I don’t see anybody
else
with
the same problem (some are similar but I don’t seem to find a solution)

I would really appreciate any help you could give me.

22:00 sample one after start up - before first request
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12814 root 19 0 21200 20M 2744 S 0.0 6.6 0:00
0
httpd
12815 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0
httpd
12816 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0
httpd
12817 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0
httpd
12818 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0
httpd
12819 nobody 24 0 21148 20M 2692 S 0.0 6.6 0:00 0
httpd
2442 mysql 15 0 20892 3636 1448 S 0.0 1.1 1:37
0
mysqld

23:40 Sample two after running for 1h:40m
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12671 nobody 16 0 35780 32M 2304 S 0.0 10.4 1:55
0
httpd
12670 nobody 16 0 34204 31M 2176 S 8.9 10.2 1:14
0
httpd
12675 nobody 15 0 34356 31M 2384 S 2.9 10.1 1:11
0
httpd
12673 nobody 16 0 34968 31M 2360 S 25.7 10.1 1:03
0
httpd
12669 nobody 15 0 34684 31M 2096 S 3.9 10.1 1:00
0
httpd
12672 nobody 15 0 32696 29M 2440 S 3.9 9.5 0:19
0
httpd
12708 nobody 16 0 22988 22M 2596 S 0.0 7.1 0:01
0
httpd
12668 root 15 0 19320 18M 864 S 0.0 6.0
0:00
0 httpd
2442 mysql 16 0 20980 5028 1928 S 0.9 1.5 0:22
0
mysqld

23:50 Sample three 10 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12671 nobody 15 0 43336 28M 1976 S 0.0 8.9 2:47
0
httpd
12670 nobody 16 0 41644 27M 1832 S 0.0 8.8 2:04
0
httpd
12669 nobody 16 0 42352 27M 1844 S 0.0 8.8 1:50
0
httpd
12673 nobody 15 0 42492 27M 2032 S 0.0 8.7 1:56
0
httpd
12675 nobody 16 0 41792 27M 2004 S 0.0 8.7 2:03
0
httpd
12672 nobody 16 0 40188 25M 2120 S 0.0 8.1 1:08
0
httpd
12708 nobody 16 0 30452 19M 2276 S 0.0 6.3 0:53
0
httpd
12726 nobody 16 0 27844 19M 2668 S 0.0 6.2 0:33
0
httpd
12716 nobody 16 0 29492 19M 2628 S 0.0 6.1 0:44
0
httpd
2442 mysql 15 0 21084 3884 1704 S 0.0 1.2 0:42
0
mysqld

00:10 Sample four 20 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12726 nobody 16 0 35996 24M 2668 S 0.0 7.9 1:21 0
httpd
12675 nobody 16 0 50200 23M 2012 S 2.9 7.6 2:57 0
httpd
12673 nobody 16 0 50912 23M 2020 S 24.7 7.5 2:53 0
httpd
12669 nobody 15 0 50568 23M 1844 S 2.9 7.5 2:40 0
httpd
12672 nobody 16 0 48576 23M 2108 S 0.0 7.5 2:05 0
httpd
12742 nobody 16 0 30904 23M 2644 S 12.8 7.5 0:55 0
httpd
12670 nobody 16 0 49984 23M 1844 S 3.9 7.4 2:59 0
httpd
12716 nobody 15 0 37728 23M 2640 S 0.0 7.3 1:33 0
httpd
12671 nobody 16 0 51480 23M 1976 S 0.0 7.3 3:38 0
httpd
12708 nobody 16 0 38944 22M 2276 S 6.9 7.3 1:55 0
httpd
2442 mysql 15 0 20948 3672 1500 S 0.0 1.1 1:06
0
mysqld

00:30 Sample five 20 minutes later
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12672 nobody 16 0 57480 25M 1848 S 5.9 8.0 3:10 0
httpd
12708 nobody 16 0 47636 24M 2008 S 0.0 7.9 2:53 0
httpd
12675 nobody 16 0 59004 24M 1752 S 22.7 7.9 3:55 0
httpd
12669 nobody 15 0 59332 24M 1576 S 0.0 7.9 3:38 0
httpd
12716 nobody 16 0 46576 24M 2368 S 0.0 7.9 2:30 0
httpd
12742 nobody 16 0 39720 24M 2376 S 0.0 7.9 1:54 0
httpd
12673 nobody 16 0 59572 24M 1752 R 1.9 7.8 3:51 0
httpd
12671 nobody 16 0 60152 24M 1716 R 17.8 7.8 4:34 0
httpd
12726 nobody 16 0 44824 24M 2408 S 2.9 7.7 2:23 0
httpd
12670 nobody 15 0 58692 23M 1584 S 4.9 7.6 3:53 0
httpd
2442 mysql 15 0 21024 3724 1492 S 1.9 1.1 1:33
0
mysqld

Regards
Doug
This email was scanned by ICL’s anti-virus solution at
http://www.eject.co.za

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

–
Request Tracker... So much more than a help desk — Best Practical Solutions – Trouble Ticketing. Free.

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm
This email was scanned by ICL’s anti-virus solution at
http://www.eject.co.za

This email was scanned by ICL’s anti-virus solution at http://www.eject.co.za

4

Jesse Vincent (jesse@bestpractical.com) wrote:

Do you still see this issue if you turn off Apache::DBI?

I get this type of behaviour RT2 (2.0.13). Starts off small, and then over
time we get this type of thing:

PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
26975 nobody 4 58 0 154M 135M sleep 37:47 0.00% httpd
26978 nobody 4 58 0 153M 133M sleep 37:45 0.00% httpd
27089 nobody 4 58 0 153M 133M sleep 36:23 0.00% httpd
27047 nobody 4 58 0 152M 134M sleep 37:22 0.00% httpd
26976 nobody 4 58 0 152M 135M sleep 39:52 0.00% httpd
26977 nobody 4 14 0 152M 132M sleep 36:03 0.07% httpd

I’d love to turn off Apache::DBI, but wouldn’t that break the functionality?

5

douglas.borrie@eject.co.za wrote:
Look through all config not only VirtualHost section.

6

Andy Coates wrote:

Jesse Vincent (jesse@bestpractical.com) wrote:

Do you still see this issue if you turn off Apache::DBI?

I get this type of behaviour RT2 (2.0.13). Starts off small, and then over
time we get this type of thing:

PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND
26975 nobody 4 58 0 154M 135M sleep 37:47 0.00% httpd
26978 nobody 4 58 0 153M 133M sleep 37:45 0.00% httpd
27089 nobody 4 58 0 153M 133M sleep 36:23 0.00% httpd
27047 nobody 4 58 0 152M 134M sleep 37:22 0.00% httpd
26976 nobody 4 58 0 152M 135M sleep 39:52 0.00% httpd
26977 nobody 4 14 0 152M 132M sleep 36:03 0.07% httpd

I’d love to turn off Apache::DBI, but wouldn’t that break the functionality?
Apache::DBI is wrapper over DBI which try to cache connections.
Disabling this don’t break anything.
From what I’ve seen it increases in memory when the system is in use, so
disabling the web interface so it can’t read tickets kinda would stop it
increasing.
You could also use MaxRequestsPerChild apache variable to force child
restarts. It’s workaround for this problems. On our server value is 15.

And there’s the catch really - can’t disable a production system and can’t
do it to a development system because its not in use like the production one.
NOTE 1: After any changes to apache’s conf you should
stop/start(restart) it.

7

This is complete apache config

ServerType standalone
ServerRoot “/usr/local/apache_1.3.28”
PidFile /usr/local/apache_1.3.28/logs/httpd.pid
ScoreBoardFile /usr/local/apache_1.3.28/logs/httpd.scoreboard
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 50
MaxRequestsPerChild 0
Port 80
User nobody
Group nobody
ServerAdmin serveradmin@host.domain.name
ServerName host.domain.name
DocumentRoot “/usr/local/apache_1.3.28/htdocs”

Options FollowSymLinks AllowOverride None

<Directory “/usr/local/apache_1.3.28/htdocs”>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
Allow from all

UserDir public_html DirectoryIndex index.html

AccessFileName .htaccess
<Files ~ “^.ht”>
Order allow,deny
Deny from all
Satisfy All

UseCanonicalName On

TypesConfig /usr/local/apache_1.3.28/conf/mime.types

DefaultType text/plain

MIMEMagicFile /usr/local/apache_1.3.28/conf/magic

HostnameLookups Off

ErrorLog /usr/local/apache_1.3.28/logs/error_log
LogLevel warn
LogFormat “%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"”
combined
LogFormat “%h %l %u %t "%r" %>s %b” common
LogFormat “%{Referer}i → %U” referer
LogFormat “%{User-agent}i” agent
CustomLog /usr/local/apache_1.3.28/logs/access_log common

ServerSignature Off

Alias /icons/ "/usr/local/apache_1.3.28/icons/" Options Indexes MultiViews AllowOverride None Order allow,deny Allow from all 
Alias /manual/ "/usr/local/apache_1.3.28/htdocs/manual/"
<Directory "/usr/local/apache_1.3.28/htdocs/manual">
    Options Indexes FollowSymlinks MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

ScriptAlias /cgi-bin/ "/usr/local/apache_1.3.28/cgi-bin/"
<Directory "/usr/local/apache_1.3.28/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>
IndexOptions FancyIndexing AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip AddIconByType (TXT,/icons/text.gif) text/* AddIconByType (IMG,/icons/image2.gif) image/* AddIconByType (SND,/icons/sound2.gif) audio/* AddIconByType (VID,/icons/movie.gif) video/* 
AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core

AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^

DefaultIcon /icons/unknown.gif
ReadmeName README
HeaderName HEADER
AddLanguage da .dk AddLanguage nl .nl AddLanguage en .en AddLanguage et .ee AddLanguage fr .fr AddLanguage de .de AddLanguage el .el AddLanguage he .he AddCharset ISO-8859-8 .iso8859-8 AddLanguage it .it AddLanguage ja .ja AddCharset ISO-2022-JP .jis AddLanguage kr .kr AddCharset ISO-2022-KR .iso-kr AddLanguage nn .nn AddLanguage no .no AddLanguage pl .po AddCharset ISO-8859-2 .iso-pl AddLanguage pt .pt AddLanguage pt-br .pt-br AddLanguage ltz .lu AddLanguage ca .ca AddLanguage es .es AddLanguage sv .sv AddLanguage cs .cz .cs AddLanguage ru .ru AddLanguage zh-TW .zh-tw AddCharset Big5 .Big5 .big5 AddCharset WINDOWS-1251 .cp-1251 AddCharset CP866 .cp866 AddCharset ISO-8859-5 .iso-ru AddCharset KOI8-R .koi8-r AddCharset UCS-2 .ucs2 AddCharset UCS-4 .ucs4 AddCharset UTF-8 .utf8 
<IfModule mod_negotiation.c>
    LanguagePriority en da nl et fr de el it ja kr no pl pt pt-br ru ltz ca

es sv tw

AddType application/x-tar .tgz
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz


BrowserMatch “Mozilla/2” nokeepalive
BrowserMatch “MSIE 4.0b2;” nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch “RealPlayer 4.0” force-response-1.0
BrowserMatch “Java/1.0” force-response-1.0
BrowserMatch “JDK/1.0” force-response-1.0

NameVirtualHost *

ServerName host.domain.name
DocumentRoot /opt/rt3/share/html
AddDefaultCharset UTF-8
PerlRequire /opt/rt3/bin/webmux.pl

SetHandler perl-script
PerlHandler RT::Mason

Quoting “Ruslan U. Zakirov” cubic@acronis.ru:

douglas.borrie@eject.co.za wrote:
Look through all config not only VirtualHost section.

This was my previous config

ServerName host.domain.name DocumentRoot /opt/rt3/share/html AddDefaultCharset UTF-8 
    PerlModule Apache::DBI
    PerlRequire /opt/rt3/bin/webmux.pl

    <Location />
            SetHandler perl-script
            PerlHandler RT::Mason
    </Location>

I changed it to this (is this what you meant by turning off Apache::DBI?)

ServerName host.domain.name DocumentRoot /opt/rt3/share/html AddDefaultCharset UTF-8 
    PerlRequire /opt/rt3/bin/webmux.pl

    <Location />
            SetHandler perl-script
            PerlHandler RT::Mason
    </Location>

If that is what you meant, then yes the issue is still there.

Quoting Jesse Vincent jesse@bestpractical.com:

Do you still see this issue if you turn off Apache::DBI?

Hi all

I recently installed RT 3.0.4 (and later RT 3.0.6) using apache 1.3.28
and
mod_perl 1.29 statically linked.
I followed all the installation instructions and have installed it on a

Solaris

2.6 as well as a Redhat Linux 9
box but with the same problem occuring on both.
The problem appears to be some sort of memory leak where memory is

allocated but

is never released.
The size of the httpd processes grow after every request.
The end result is that all RAM is consumed and after a while all swap is

also

exhausted.

Below are samples taken of the relevant processes (top output - sorted by

memory

usage) while a dummy load is applied.
The load is generated by multiple web clients with a URL similar to the
following
http://host.domain.name/index.html?HomeRefreshInterval=2 (Refresh the
home

page

every 2 seconds - too speed things up a bit)

As you’ll see the size of the processes grow and eventually all memory is

used.

The only thing that I have found to help is to set MaxRequestsPerChild=1

in

apache config,
but performance is really bad as after each request is handled that
apache

child

is killed and another
is restarted in it’s place. I also have to limit MinSpareServers and
MaxSpareServers to about 4 with
MaxClients also set to 4.

I have searched through the mailing list archives but I don’t see anybody

else

with
the same problem (some are similar but I don’t seem to find a solution)

I would really appreciate any help you could give me.

This email was scanned by ICL’s anti-virus solution at
http://www.eject.co.za

This email was scanned by ICL’s anti-virus solution at http://www.eject.co.za

8

I’ve had the same problems, and try to get around it by restarting Apache
with a cron job on a daily basis. However last night, ran out of swap again
and sendmail kept tempfailing a mail with an attachment (about 1.5MB) which
resulted in 70 copies of the attachment in the RT database - making it
practically impossible to open the ticket in RT. I’ve remove the attachments
this morning, and I’ve also try removing DBI from the apache config and
reducing the number of MaxRequestsPerChild. Will see what affect this has.

Cheers,

Barry

Barry Byrne, IT Manager,
WBT Systems, Block 2, Harcourt Centre
Harcourt Street, Dublin 2, Ireland

-----Original Message-----
From: rt-users-admin@lists.fsck.com
[mailto:rt-users-admin@lists.fsck.com]On Behalf Of
douglas.borrie@eject.co.za
Sent: 17 October 2003 10:25
To: Ruslan U. Zakirov
Cc: rt-users@lists.fsck.com
Subject: Re: [rt-users] Help - memory leak problems

This is complete apache config

ServerType standalone
ServerRoot “/usr/local/apache_1.3.28”
PidFile /usr/local/apache_1.3.28/logs/httpd.pid
ScoreBoardFile /usr/local/apache_1.3.28/logs/httpd.scoreboard
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 50
MaxRequestsPerChild 0
Port 80
User nobody
Group nobody
ServerAdmin serveradmin@host.domain.name
ServerName host.domain.name
DocumentRoot “/usr/local/apache_1.3.28/htdocs”

Options FollowSymLinks AllowOverride None

<Directory “/usr/local/apache_1.3.28/htdocs”>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
Allow from all

UserDir public_html DirectoryIndex index.html

AccessFileName .htaccess
<Files ~ “^.ht”>
Order allow,deny
Deny from all
Satisfy All

UseCanonicalName On

TypesConfig /usr/local/apache_1.3.28/conf/mime.types

DefaultType text/plain

MIMEMagicFile /usr/local/apache_1.3.28/conf/magic

HostnameLookups Off

ErrorLog /usr/local/apache_1.3.28/logs/error_log
LogLevel warn
LogFormat “%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"”
combined
LogFormat “%h %l %u %t "%r" %>s %b” common
LogFormat “%{Referer}i → %U” referer
LogFormat “%{User-agent}i” agent
CustomLog /usr/local/apache_1.3.28/logs/access_log common

ServerSignature Off

Alias /icons/ "/usr/local/apache_1.3.28/icons/" Options Indexes MultiViews AllowOverride None Order allow,deny Allow from all 
Alias /manual/ "/usr/local/apache_1.3.28/htdocs/manual/"
<Directory "/usr/local/apache_1.3.28/htdocs/manual">
    Options Indexes FollowSymlinks MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

ScriptAlias /cgi-bin/ "/usr/local/apache_1.3.28/cgi-bin/"
<Directory "/usr/local/apache_1.3.28/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>
IndexOptions FancyIndexing AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip AddIconByType (TXT,/icons/text.gif) text/* AddIconByType (IMG,/icons/image2.gif) image/* AddIconByType (SND,/icons/sound2.gif) audio/* AddIconByType (VID,/icons/movie.gif) video/* 
AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core

AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^

DefaultIcon /icons/unknown.gif
ReadmeName README
HeaderName HEADER
AddLanguage da .dk AddLanguage nl .nl AddLanguage en .en AddLanguage et .ee AddLanguage fr .fr AddLanguage de .de AddLanguage el .el AddLanguage he .he AddCharset ISO-8859-8 .iso8859-8 AddLanguage it .it AddLanguage ja .ja AddCharset ISO-2022-JP .jis AddLanguage kr .kr AddCharset ISO-2022-KR .iso-kr AddLanguage nn .nn AddLanguage no .no AddLanguage pl .po AddCharset ISO-8859-2 .iso-pl AddLanguage pt .pt AddLanguage pt-br .pt-br AddLanguage ltz .lu AddLanguage ca .ca AddLanguage es .es AddLanguage sv .sv AddLanguage cs .cz .cs AddLanguage ru .ru AddLanguage zh-TW .zh-tw AddCharset Big5 .Big5 .big5 AddCharset WINDOWS-1251 .cp-1251 AddCharset CP866 .cp866 AddCharset ISO-8859-5 .iso-ru AddCharset KOI8-R .koi8-r AddCharset UCS-2 .ucs2 AddCharset UCS-4 .ucs4 AddCharset UTF-8 .utf8 
<IfModule mod_negotiation.c>
    LanguagePriority en da nl et fr de el it ja kr no pl pt

pt-br ru ltz ca
es sv tw

AddType application/x-tar .tgz
AddEncoding x-compress .Z
AddEncoding x-gzip .gz .tgz


BrowserMatch “Mozilla/2” nokeepalive
BrowserMatch “MSIE 4.0b2;” nokeepalive downgrade-1.0
force-response-1.0
BrowserMatch “RealPlayer 4.0” force-response-1.0
BrowserMatch “Java/1.0” force-response-1.0
BrowserMatch “JDK/1.0” force-response-1.0

NameVirtualHost *

ServerName host.domain.name
DocumentRoot /opt/rt3/share/html
AddDefaultCharset UTF-8
PerlRequire /opt/rt3/bin/webmux.pl

SetHandler perl-script
PerlHandler RT::Mason

Quoting “Ruslan U. Zakirov” cubic@acronis.ru:

douglas.borrie@eject.co.za wrote:
Look through all config not only VirtualHost section.

This was my previous config

ServerName host.domain.name DocumentRoot /opt/rt3/share/html AddDefaultCharset UTF-8 
    PerlModule Apache::DBI
    PerlRequire /opt/rt3/bin/webmux.pl

    <Location />
            SetHandler perl-script
            PerlHandler RT::Mason
    </Location>

I changed it to this (is this what you meant by turning off
Apache::DBI?)

ServerName host.domain.name DocumentRoot /opt/rt3/share/html AddDefaultCharset UTF-8 
    PerlRequire /opt/rt3/bin/webmux.pl

    <Location />
            SetHandler perl-script
            PerlHandler RT::Mason
    </Location>

If that is what you meant, then yes the issue is still there.

Quoting Jesse Vincent jesse@bestpractical.com:

Do you still see this issue if you turn off Apache::DBI?

Hi all

I recently installed RT 3.0.4 (and later RT 3.0.6) using
apache 1.3.28
and
mod_perl 1.29 statically linked.
I followed all the installation instructions and have
installed it on a

Solaris

2.6 as well as a Redhat Linux 9
box but with the same problem occuring on both.
The problem appears to be some sort of memory leak where memory is

allocated but

is never released.
The size of the httpd processes grow after every request.
The end result is that all RAM is consumed and after a while
all swap is

also

exhausted.

Below are samples taken of the relevant processes (top
output - sorted by

memory

usage) while a dummy load is applied.
The load is generated by multiple web clients with a URL
similar to the
following
http://host.domain.name/index.html?HomeRefreshInterval=2 (Refresh the
home

page

every 2 seconds - too speed things up a bit)

As you’ll see the size of the processes grow and eventually
all memory is

used.

The only thing that I have found to help is to set
MaxRequestsPerChild=1

in

apache config,
but performance is really bad as after each request is handled that
apache

child

is killed and another
is restarted in it’s place. I also have to limit MinSpareServers and
MaxSpareServers to about 4 with
MaxClients also set to 4.

I have searched through the mailing list archives but I
don’t see anybody

else

with
the same problem (some are similar but I don’t seem to find
a solution)

I would really appreciate any help you could give me.

This email was scanned by ICL’s anti-virus solution at
http://www.eject.co.za

This email was scanned by ICL’s anti-virus solution at
http://www.eject.co.za
rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

9

I’ve put some of the following into my httpd.conf, and so far, apache has
remained stable and reasonably quick over the last 10 days:

redhat linux 7.3, rt 3.0.4, 1GB ram

MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 2
MaxSpareServers 10
StartServers 6
MaxClients 50
MaxRequestsPerChild 500

Season these values to taste for your environment, and it should help. The
goal is to kill off apache child processes as they get ‘too old and
overworked’.

Hope that helps.

10

“JJ” == John Jasen jjasen@datafoundation.com writes:

JJ> MaxRequestsPerChild 500

JJ> Season these values to taste for your environment, and it should help. The
JJ> goal is to kill off apache child processes as they get ‘too old and
JJ> overworked’.

I’d keep an eyeball on the httpd process size for your mod_perl with
such a setting. I rarely go over 200 requests per child on a
mod_perl server.

11

With this config make sure you have a large number of file descriptors
set in your kernel.-----Original Message-----
From: John Jasen [mailto:jjasen@datafoundation.com]
Sent: Friday, October 17, 2003 10:04 AM
To: Barry Byrne
Cc: douglas.borrie@eject.co.za; rt-users@lists.fsck.com
Subject: RE: [rt-users] Help - memory leak problems

I’ve put some of the following into my httpd.conf, and so far, apache
has
remained stable and reasonably quick over the last 10 days:

redhat linux 7.3, rt 3.0.4, 1GB ram

MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 2
MaxSpareServers 10
StartServers 6
MaxClients 50
MaxRequestsPerChild 500

Season these values to taste for your environment, and it should help.
The
goal is to kill off apache child processes as they get ‘too old and
overworked’.

Hope that helps.

rt-users mailing list
rt-users@lists.fsck.com
http://lists.fsck.com/mailman/listinfo/rt-users

Have you read the FAQ? The RT FAQ Manager lives at http://fsck.com/rtfm

12

Thanks for all the responses.

I think what is becoming obvious from all the replies is that this type of
memory usage is
considered normal in a mod_perl environment. That’s fine for all the
mod_perl experienced
users out there but for first time mod_perlers maybe it’s not so obvious.

I found to get the best response times and to prevent the server from
committing suicide
it is necessary to tune apache in such a way that it kills the large apache
children just
before the server runs out of RAM and starts to swap. To get it to do this I
played
with the following apache config variables :
MaxKeepAliveRequests
KeepAliveTimeout
MaxSpareServers
MaxClients
MaxRequestsPerChild

Thanks again for the help.
This email was scanned by ICL’s anti-virus solution at http://www.eject.co.za