FastCGI: comm with server "/opt/rt4/sbin/rt-server.fcgi" aborted: idle timeout (300 sec)

Okay, I have tried scouring the archives, but haven’t found a solution
to my problem, although rt-server.fcgi errors is mentioned with other
issues.

Here is a summary of the configuration:

  1. Originally, a working rt-4.2.10 installation under Ubuntu 14.04.
    Perl was installed locally and the following put in .bashrc to point
    to the correct version:

    PERL_MB_OPT="–install_base “/home//perl5"”; export PERL_MB_OPT;
    PERL_MM_OPT=“INSTALL_BASE=/home//perl5”; export PERL_MM_OPT;

  2. Recently upgraded to Ubuntu 16.04.1, and rt stopped working.

  3. Was getting the following error in /var/log/apache2/error.log:

    ‘FastCGI: server “/opt/rt4/sbin/rt-server.fcgi” has failed to remain
    running for 30 seconds given 3 attempts, its restart interval has
    been backed off to 600 seconds’

    The command for FastCGIServer in the site config also failed on the
    command line:

    /opt/rt4/sbin/rt-server.fcgi -processes 5 -idle-timeout 300

    with an error about Perl module dependency.

  4. So, after much effort in trying to upgrade the old (local) Perl
    installation and figuring that it was a Perl issue, I completely
    disabled/removed the original Perl installation.

  5. Installed Perl 5.24.0 via perlbrew and then installed cpanm

  6. Set “export RT_FIX_DEPS_CMD=/opt/perl/bin/cpanm”

  7. Ran fixdeps. Had to use --force to install Date::Manip, but other
    than that have a fresh cpan.

  8. So clean Perl, and clean cpan.

  9. Did a reconfigure of rt-4.2.10 using the revised perlbrew directory,
    and a ‘make install’ (Backed up the original directory. Copied over
    the old RT_SiteConfig.pm over to the new install directory. Checked
    permissions: root:www-data.

  10. Problem is that I am still get the same errors related to
    rt-server.fcgi.

Any guidance as to what might be the issue?

advTHANKSance.

Pavneet Arora m: +1 (647) 406-6843
Waroc Informatik t: +1 (416) 937-9276

Okay, I have tried scouring the archives, but haven’t found a solution
to my problem, although rt-server.fcgi errors is mentioned with other
issues.

Here is a summary of the configuration:

  1. Originally, a working rt-4.2.10 installation under Ubuntu 14.04.
    Perl was installed locally and the following put in .bashrc to point
    to the correct version:

    PERL_MB_OPT=“–install_base "/home//perl5"”; export PERL_MB_OPT;
    PERL_MM_OPT=“INSTALL_BASE=/home//perl5”; export PERL_MM_OPT;

  2. Recently upgraded to Ubuntu 16.04.1, and rt stopped working.

  3. Was getting the following error in /var/log/apache2/error.log:

    ‘FastCGI: server “/opt/rt4/sbin/rt-server.fcgi” has failed to remain
    running for 30 seconds given 3 attempts, its restart interval has
    been backed off to 600 seconds’

    The command for FastCGIServer in the site config also failed on the
    command line:

    /opt/rt4/sbin/rt-server.fcgi -processes 5 -idle-timeout 300

    with an error about Perl module dependency.

  4. So, after much effort in trying to upgrade the old (local) Perl
    installation and figuring that it was a Perl issue, I completely
    disabled/removed the original Perl installation.

  5. Installed Perl 5.24.0 via perlbrew and then installed cpanm

  6. Set “export RT_FIX_DEPS_CMD=/opt/perl/bin/cpanm”

  7. Ran fixdeps. Had to use --force to install Date::Manip, but other
    than that have a fresh cpan.

  8. So clean Perl, and clean cpan.

  9. Did a reconfigure of rt-4.2.10 using the revised perlbrew directory,
    and a ‘make install’ (Backed up the original directory. Copied over
    the old RT_SiteConfig.pm over to the new install directory. Checked
    permissions: root:www-data.

  10. Problem is that I am still get the same errors related to
    rt-server.fcgi.

Any guidance as to what might be the issue?

advTHANKSance.

What are the errors?

Nilesh

Observed behaviour is that the browser request times out.

/var/log/apache2/error.log shows as quoted originally (same error as
immediately after upgrade to 16.04.1):

'FastCGI: server "/opt/rt4/sbin/rt-server.fcgi" has failed to remain
running for 30 seconds given 3 attempts, its restart interval has
been backed off to 600 seconds'

/opt/rt4/var/log/apache2.error has the matching error:

] FastCGI: comm with server “/opt/rt4/sbin/rt-server.fcgi” aborted: idle timeout (300 sec)
] FastCGI: incomplete headers (0 bytes) received from server “/opt/rt4/sbin/rt-server.fcgi”

Just for completeness here is the apache2 config file from
sites-available (with host obscured):

# Tell FastCGI to put its temporary files somewhere sane; this may
# be necessary if your distribution doesn't already set it
#FastCgiIpcDir /tmp

FastCgiServer /opt/rt4/sbin/rt-server.fcgi -processes 5 -idle-timeout 300

<VirtualHost *:80>
  ### Optional apache logs for RT
  # Ensure that your log rotation scripts know about these files
  ErrorLog /opt/rt4/var/log/apache2.error
  TransferLog /opt/rt4/var/log/apache2.access
  LogLevel debug

  AddDefaultCharset UTF-8

  ScriptAlias / /opt/rt4/sbin/rt-server.fcgi/

  ServerAdmin webmaster@localhost
  ServerName <rt-host-XXX>.com
  DocumentRoot /var/www/<rt-host-XXX>.com

  <Location />
    <IfVersion >= 2.4> #
      Require all granted
    </IfVersion>
    <IfVersion < 2.4>
      Order allow,deny
      Allow from all
    </IfVersion>

    Options +ExecCGI
    AddHandler fcgid-script fcgi
  </Location>

</VirtualHost>On 08Sep16, Nilesh wrote:

What are the errors?


Nilesh

Pavneet Arora m: +1 (647) 406-6843
Waroc Informatik t: +1 (416) 937-9276

I was asking about the Perl errors. What do you see on the console or in RT’s
logs (enable debug logging in RT_SiteConfig) when you run rt-server.fcgi
directly?

NileshOn Thu, 2016-09-08 at 13:57 -0400, Pavneet Arora wrote:

Observed behaviour is that the browser request times out.

/var/log/apache2/error.log shows as quoted originally (same error as
immediately after upgrade to 16.04.1):

'FastCGI: server "/opt/rt4/sbin/rt-server.fcgi" has failed to remain
running for 30 seconds given 3 attempts, its restart interval has
been backed off to 600 seconds'

/opt/rt4/var/log/apache2.error has the matching error:

] FastCGI: comm with server “/opt/rt4/sbin/rt-server.fcgi” aborted: idle
timeout (300 sec)
] FastCGI: incomplete headers (0 bytes) received from server
“/opt/rt4/sbin/rt-server.fcgi”

Just for completeness here is the apache2 config file from
sites-available (with host obscured):

# Tell FastCGI to put its temporary files somewhere sane; this may
# be necessary if your distribution doesn't already set it
#FastCgiIpcDir /tmp

FastCgiServer /opt/rt4/sbin/rt-server.fcgi -processes 5 -idle-timeout 300

<VirtualHost *:80>
  ### Optional apache logs for RT
  # Ensure that your log rotation scripts know about these files
  ErrorLog /opt/rt4/var/log/apache2.error
  TransferLog /opt/rt4/var/log/apache2.access
  LogLevel debug

  AddDefaultCharset UTF-8

  ScriptAlias / /opt/rt4/sbin/rt-server.fcgi/

  ServerAdmin webmaster@localhost
  ServerName <rt-host-XXX>.com
  DocumentRoot /var/www/<rt-host-XXX>.com

  <Location />
    <IfVersion >= 2.4> #
      Require all granted
    </IfVersion>
    <IfVersion < 2.4>
      Order allow,deny
      Allow from all
    </IfVersion>

    Options +ExecCGI
    AddHandler fcgid-script fcgi
  </Location>

</VirtualHost>

On 08Sep16, Nilesh wrote:

What are the errors?


Nilesh

Sorry, I misunderstood. Here is the output from the shell command. I
am just reconfiguring RT_SiteConfig and will get that to you shortly.

/opt/rt4/sbin/rt-server.fcgi -processes 5 -idle-timeout 300

Possible attempt to separate words with commas at /opt/rt4/etc/RT_SiteConfig.pm line 24.
[3658] [Thu Sep 8 18:10:42 2016] [info]: Using internal Perl HTML → text conversion (/opt/rt4/sbin/…/lib/RT/Interface/Email.pm:1818)
STDIN is not a socket, and no --listen, --socket, or --port providedOn 08Sep16, Nilesh wrote:

I was asking about the Perl errors. What do you see on the console or in RT’s
logs (enable debug logging in RT_SiteConfig) when you run rt-server.fcgi
directly?

Pavneet Arora m: +1 (647) 406-6843
Waroc Informatik t: +1 (416) 937-9276

Log to syslog option will generate messages in system log which is
/var/log/messages or it’s equivalent based on distribution.

Or you can add

Set(@LogToSyslogConf, (facility => ‘local2’));

To get the messages in local2 facility so that it can be filtered out via
syslog daemon to a separate file.

Turn on the stack trace logging too so you can see where it’s breaking.

NileshOn 08-Sep-2016 11:58 PM, “Pavneet Arora” pavneet_arora@waroc.com wrote:

Simple question. I have tried modifying RT_SiteConfig with the
following options individually, but don’t see anything in syslog or in
the second case, /opt/rt4/var/log/apache2.error. Is this what you were
referring to?

Set( $LogToSysLog, ‘debug’);

or

Set( $LogToSTDERR, ‘debug’);

On 08Sep16, Nilesh wrote:

I was asking about the Perl errors. What do you see on the console or in
RT’s
logs (enable debug logging in RT_SiteConfig) when you run rt-server.fcgi
directly?


Pavneet Arora m: +1 (647) 406-6843
Waroc Informatik t: +1 (416) 937-9276

It seems the configuration you’ve done is being ignored? This shouldn’t
happen.
Are you sure Apache has access to the directory of rt installation
completely since you seem to run it from /home?

NileshOn 09-Sep-2016 1:15 AM, “Pavneet Arora” pavneet_arora@waroc.com wrote:

Actually logging is working. It is just that once apache2 is reloaded
there are no more log entries added.

I just did a fresh 4.4.1 installation and after the ‘make
initialize-database’ command, this the tail of the rt.log:

Trace begun at /home/pavneet/var/Software/rt/rt-4.4.1/sbin/…/lib/RT.pm
line 304
Log::Dispatch::ANON(‘Log::Dispatch=HASH(0x54f75f8)’, ‘Done setting
up database content.’) called at /home/pavneet/var/Software/rt/
rt-4.4.1/sbin/…/lib/RT/Handle.pm line 1428
RT::Handle::InsertData(‘RT::Handle=HASH(0x5789400)’,
‘/opt/rt4/etc/initialdata’, undef) called at sbin/rt-setup-database line 388
main::ANON at sbin/rt-setup-database line 401
main::action_insert(‘prompt-for-dba-password’, 1, ‘package’, ‘RT’,
‘action’, ‘init’) called at sbin/rt-setup-database line 210

It is just that after this the log goes quiet.

I wonder if it is related to apache2 and fastcgi since now with two
different installations, we are getting the same behaviour?


Pavneet Arora m: +1 (647) 406-6843
Waroc Informatik t: +1 (416) 937-9276