Hi,
after upgrading to RT 4.0.6 I am having problems with mail sending. This
line starts to appear in log:
Scrip Commit 6 died. - Can’t fork at /usr/share/perl5/Mail/Mailer.pm
line 145
and no mail can be sent until I restart the Apache.
Scrip 6 is “On Correspond Notify Requestors and Ccs”
Do you have any idea why it is happening? And more importantly - how can
I solve this so it does not happen again?
Thank you for your insights.
Martin
after upgrading to RT 4.0.6 I am having problems with mail sending. This
line starts to appear in log:Scrip Commit 6 died. - Can’t fork at /usr/share/perl5/Mail/Mailer.pm
line 145and no mail can be sent until I restart the Apache.
Do you have any idea why it is happening? And more importantly - how can
I solve this so it does not happen again?
Can’t fork means your operating system is dangerously out of memory.
You should tune Apache and anything else on the box (your database
server?) to coexist peacefully.
What MailCommand are you using in RT?
after upgrading to RT 4.0.6 I am having problems with mail sending. This
line starts to appear in log:Scrip Commit 6 died. - Can’t fork at /usr/share/perl5/Mail/Mailer.pm
line 145and no mail can be sent until I restart the Apache.
Do you have any idea why it is happening? And more importantly - how can
I solve this so it does not happen again?Can’t fork means your operating system is dangerously out of memory.
You should tune Apache and anything else on the box (your database
server?) to coexist peacefully.What MailCommand are you using in RT?
Hi Thomas,
I am using sendmail as a mail command, but you are most likely right
about insufficient memory. I have checked the logs and Apache processes
were dying of memory starvation. I have doubled the memory (it was only
1 gig) and will check if it keeps happening.
Thanks,
Martin
after upgrading to RT 4.0.6 I am having problems with mail sending. This
line starts to appear in log:Scrip Commit 6 died. - Can’t fork at /usr/share/perl5/Mail/Mailer.pm
line 145and no mail can be sent until I restart the Apache.
Do you have any idea why it is happening? And more importantly - how can
I solve this so it does not happen again?Can’t fork means your operating system is dangerously out of memory.
You should tune Apache and anything else on the box (your database
server?) to coexist peacefully.What MailCommand are you using in RT?
Hi Thomas,
I am using sendmail as a mail command, but you are most likely right
about insufficient memory. I have checked the logs and Apache processes
were dying of memory starvation. I have doubled the memory (it was only
1 gig) and will check if it keeps happening.
Make sure you have plenty of swap allocated; you may not need the memory as physical. I run the RT web server on a machine with only 2GB of RAM, and it’s fine. I don’t run the database on the same machine, though. You have to remember that when Apache/perl forks to exec the sendmail binary, temporarily you double the virtual memory requirement of the Apache/perl setup. You either have to have virtual memory overcommit turned on (which I think is the default on most Linux systems these days) or to have enough swap allocated to cope.
vm.overcommit_memory = 0
in /etc/sysctl.conf is probably what you want. You probably actively don’t want it to be set to 2.
Tim
The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
Make sure you have plenty of swap allocated; you may not need the memory as physical. I run the RT web server on a machine with only 2GB of RAM, and it’s fine. I don’t run the database on the same machine, though. You have to remember that when Apache/perl forks to exec the sendmail binary, temporarily you double the virtual memory requirement of the Apache/perl setup. You either have to have virtual memory overcommit turned on (which I think is the default on most Linux systems these days) or to have enough swap allocated to cope.
vm.overcommit_memory = 0
in /etc/sysctl.conf is probably what you want. You probably actively don’t want it to be set to 2.
Tim
Hi Tim,
thank you for the tips, I was not aware about doubling the virtual
memory requirements. I will look into it.
Martin
thank you for the tips, I was not aware about doubling the virtual
memory requirements. I will look into it.
It’s only for a split second, and the memory isn’t used. UNIX doesn’t just start an application in one go, it uses two steps:
This duplicates the current process, and both processes continue executing.
UNIX does this with so-called copy-on-write, so the pages aren’t actually duplicated until they’re modified, but nevertheless, the potential exists for both copies to be completely required, so if you have vm.overcommit_memory set to 2, you need all that virtual memory to be available just in case.
the newly forked process usually immediately executes the exec() system call, which replaces the current process’ virtual memory image with the desired program. At this point, the virtual memory requirements go back down again (assuming the new process is something small, which in the case of sendmail, it is).
Apologies if there are errors and oversimplifications in the above, I’m not exactly a total UNIX beardy type.
We’ve seen this bite us on some of our HPC clusters at work - there, we do have vm.overcommit_memory set to 2, because we want programs allocating too much memory to die immediately, without risk of the kernel’s out of memory killer zapping the wrong thing. A common problem on such machines is very large computational jobs trying to start sub-processes - as soon as they do, their virtual memory requirement doubles and the fork() fails with an out of memory error.
It just seemed to me that it might be the case with your system. Try the output of:
cat /proc/sys/vm/overcommit_memory
if it’s 2, then this is probably your problem.
Tim
The Wellcome Trust Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.
It’s only for a split second, and the memory isn’t used. UNIX doesn’t just start an application in one go, it uses two steps:
- fork
This duplicates the current process, and both processes continue executing.
UNIX does this with so-called copy-on-write, so the pages aren’t actually duplicated until they’re modified, but nevertheless, the potential exists for both copies to be completely required, so if you have vm.overcommit_memory set to 2, you need all that virtual memory to be available just in case.
- exec
the newly forked process usually immediately executes the exec() system call, which replaces the current process’ virtual memory image with the desired program. At this point, the virtual memory requirements go back down again (assuming the new process is something small, which in the case of sendmail, it is).
Apologies if there are errors and oversimplifications in the above, I’m not exactly a total UNIX beardy type.
We’ve seen this bite us on some of our HPC clusters at work - there, we do have vm.overcommit_memory set to 2, because we want programs allocating too much memory to die immediately, without risk of the kernel’s out of memory killer zapping the wrong thing. A common problem on such machines is very large computational jobs trying to start sub-processes - as soon as they do, their virtual memory requirement doubles and the fork() fails with an out of memory error.
It just seemed to me that it might be the case with your system. Try the output of:
cat /proc/sys/vm/overcommit_memory
if it’s 2, then this is probably your problem.
Tim
I have checked the overcommit settings and it is zero. It seems that the
problem was purely in RT and postgres running with mostly default
settings on one machine with not enough memory. It was good enough for
RT 3.8.7, but it is not for 4.0.6. Although now I remember a few times
when processes were killed because of insufficient memory back then. But
it happened so little that we did not pay attention. I will not be
making the same mistake again…
I will ask some greybeard to check the settings for me or throw memory
at it, until the problem goes away 
Anyway, thanks a lot for your help.
Martin