RT.conf tossing an Internal Server Error

DavidV · December 16, 2024, 8:20am

Hi,

I am at a bit of a loss. I have seemingly walked through the guide and got all the plugins and dependencies completed. initialized the db and configured my RT_SiteConfig.pm file fine, but when I apply the suggested .conf file to the /ect/httpd/conf.d directory. I was still greeted with the generic CentOS webpage saying apache was working. I then realized I forgot to put in my server name in the first line, but after doing so and restarting httpd; I am greeted with a generic error. I posted it below. I don’t really have any experience with linux and so this is quite the learning curve. Any and all insight is greatly appreciated!

[EDIT]
Though I am getting the error… When I run /opt/rt5/sbin/rt-server --port 8080 I am not getting any errors when making changes.

Environment:
CentOS 9 Stream
RT 5.0.7
MySql v8.0.36
Perl v5.32.1
Apache v2.4.62

" Internal Server Error

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log."

here is my current conf file

    <VirtualHost **<SERVERNAME>**>
        ### Optional apache logs for RT
        # Ensure that your log rotation scripts know about these files
        # ErrorLog /opt/rt5/var/log/apache2.error
        # TransferLog /opt/rt5/var/log/apache2.access
        # LogLevel debug

        AddDefaultCharset UTF-8

        ScriptAlias / /opt/rt5/sbin/rt-server.fcgi/

        DocumentRoot "/opt/rt5/share/html"
        <Location />
            Require all granted
            Options +ExecCGI
            AddHandler fcgid-script fcgi
        </Location>
    </VirtualHost>

rubberduck · December 16, 2024, 9:35am

You have to put in a wildcard or your server’s fqdn here. The **<SERVERNAME>** is just a placeholder.

See: VirtualHost Examples - Apache HTTP Server Version 2.4

In /var/log/apache2/error.log you see the error occuring if there is an internal server error.

DavidV · December 16, 2024, 12:04pm

Thank you for the quick response!

I have gone ahead and made the change so that instead of ; it is now *:80 as shown in the link. I had also placed ServerName below as well.

[Updated Conf]
<VirtualHost *:80>

    ServerName <**ServerName**>

Unfortunately the error still persists. I did follow the path you showed, but I don’t have /var/log/apache2/error.log. I do have /var/log/httpd/error.log which I think is a weird quirk of my server. I cant use the apachectl -M command to list modules. I got to run httpd -M. The error I am seeing in the log is posted below. Did I happen to miss installing a plugin?

Andrew_Ruthven · December 16, 2024, 10:42pm

Is there anything being logged in the RT log file?

DavidV · December 17, 2024, 5:48am

No file seems to be populating in the /opt/rt5/var/log directory. The only logs populating data is /Var/Log/Httpd/Error.log & Acess.log.

Error Log Entries

[/opt/rt5/sbin/rt-server.fcgi:153]
[Mon Dec 16 21:30:53.350875 2024] [fcgid:warn] [pid 36269:tid 36357] (104)Connection reset by peer: [client 192.168.57.21:50451] mod_fcgid: error reading data from FastCGI server, referer: http://cchs-rt.cchd.local/
[Mon Dec 16 21:30:53.350938 2024] [core:error] [pid 36269:tid 36357] [client 192.168.57.21:50451] End of script output before headers: rt-server.fcgi, referer: http://cchs-rt.cchd.local/

Access Log Entries

192.168.57.21 - - [16/Dec/2024:21:30:49 -0800] “GET / HTTP/1.1” 500 527 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36”
192.168.57.21 - - [16/Dec/2024:21:30:51 -0800] “GET /favicon.ico HTTP/1.1” 500 527 “http://cchs-rt.cchd.local/” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36”

I have gone ahead and created the following config and restarted the httpd service. I am seeing these entries.

Config in RT_SiteConfig.pm
#Logging Information
Set( $LogToFile, ‘debug’);
Set( $LogToFileNamed, ‘cchs-rt.log’);
Set( $LogDir, ‘/Opt/rt5/var/log/’);

No file is created in /opt/rt5/var/log

/var/log/httpd/error.log

[Mon Dec 16 21:44:38.393887 2024] [core:notice] [pid 37014:tid 37014] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Mon Dec 16 21:44:38.394771 2024] [suexec:notice] [pid 37014:tid 37014] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server’s fully qualified domain name, using fe80::c0e7:7fcf:4359:d6cf%ens192. Set the ‘ServerName’ directive globally to suppress this message
[Mon Dec 16 21:44:38.419890 2024] [lbmethod_heartbeat:notice] [pid 37014:tid 37014] AH02282: No slotmem from mod_heartmonitor
[Mon Dec 16 21:44:38.429610 2024] [mpm_event:notice] [pid 37014:tid 37014] AH00489: Apache/2.4.62 (CentOS Stream) mod_fcgid/2.3.9 mod_perl/2.0.12 Perl/v5.32.1 configured – resuming normal operations
[Mon Dec 16 21:44:38.429640 2024] [core:notice] [pid 37014:tid 37014] AH00094: Command line: ‘/usr/sbin/httpd -D FOREGROUND’
Log file ‘/Opt/rt5/var/log/cchs-rt.log’ couldn’t be written or created.
RT can’t run. at /opt/rt5/sbin/…/lib/RT.pm line 343.
[Mon Dec 16 21:44:41.050599 2024] [fcgid:warn] [pid 37023:tid 37178] (104)Connection reset by peer: [client 192.168.57.21:50601] mod_fcgid: error reading data from FastCGI server
[Mon Dec 16 21:44:41.050646 2024] [core:error] [pid 37023:tid 37178] [client 192.168.57.21:50601] End of script output before headers: rt-server.fcgi
Log file ‘/Opt/rt5/var/log/cchs-rt.log’ couldn’t be written or created.
RT can’t run. at /opt/rt5/sbin/…/lib/RT.pm line 343.
[Mon Dec 16 21:44:41.440207 2024] [fcgid:warn] [pid 37022:tid 37129] (104)Connection reset by peer: [client 192.168.57.21:50602] mod_fcgid: error reading data from FastCGI server, referer: http://cchs-rt.cchd.local/
[Mon Dec 16 21:44:41.440258 2024] [core:error] [pid 37022:tid 37129] [client 192.168.57.21:50602] End of script output before headers: rt-server.fcgi, referer: http://cchs-rt.cchd.local/

Andrew_Ruthven · December 17, 2024, 7:34am

Umm, I’d like to point out these lines in the /var/log/httpd/error.log excerpt you’ve included:

Log file ‘/Opt/rt5/var/log/cchs-rt.log’ couldn’t be written or created.
RT can’t run. at /opt/rt5/sbin/…/lib/RT.pm line 343.

I suggest you check the permissions of /Opt/rt5/var/log .

DavidV · December 17, 2024, 8:55am

I have taken a look at the permissions and have ensured that they were updated from root to apache. After restarting the service I am still greeted with the same error. Is there another account I should have applied permissions to this for?

DavidV · December 17, 2024, 9:08am

As odd as it is… I have changed my RT_SiteConfig.pm file to the following

Config in RT_SiteConfig.pm
#Logging Information
Set( $LogToFile, ‘debug’);
Set( $LogToFileNamed, ‘cchs-rt.log’);
Set( $LogDir, ‘/var/log/rt5’);

I am now getting this error

[/opt/rt5/sbin/…/lib/RT/PlackRunner.pm:72]
[/opt/rt5/sbin/rt-server.fcgi:153] (/opt/rt5/sbin/…/lib/RT.pm:409)
[40662] [Tue Dec 17 09:02:00 2024] [debug]: Using w3m for HTML → text conversion (/opt/rt5/sbin/…/lib/RT/Interface/Email.pm:1562)
[40662] [Tue Dec 17 09:02:00 2024] [critical]: Could not create ‘/opt/rt5/var/mason_data/obj/.__obj_create_marker’: Permission denied
Stack:
[/usr/local/share/perl5/5.32/HTML/Mason/Interp.pm:222]
[/usr/local/share/perl5/5.32/HTML/Mason/Interp.pm:169]
[/usr/local/share/perl5/5.32/HTML/Mason/Interp.pm:155]
[/usr/local/share/perl5/5.32/Class/Container.pm:329]
[/usr/local/share/perl5/5.32/Class/Container.pm:53]
[/usr/local/share/perl5/5.32/HTML/Mason/CGIHandler.pm:38]
[/usr/local/share/perl5/5.32/HTML/Mason/PSGIHandler.pm:14]
[/opt/rt5/sbin/…/lib/RT/Interface/Web/Handler.pm:114]
[/opt/rt5/sbin/…/lib/RT/Interface/Web/Handler.pm:270]
[/opt/rt5/sbin/…/lib/RT/PlackRunner.pm:108]
[/opt/rt5/sbin/…/lib/RT/PlackRunner.pm:72]
[/opt/rt5/sbin/rt-server.fcgi:153] (/opt/rt5/sbin/…/lib/RT.pm:409)

It’s clear that my issue is permissions, but what account do I give permissions to this directory to? I can only think of apache.

sollericos · December 17, 2024, 10:27am

This happened to me some time ago. From source install directory check:

make fixperms

Maybe it’s better to look inside Makefile first.

Jim_Brandt · December 17, 2024, 4:33pm

This will also cause issues. I’m not aware of a complete SELinux policy for running RT, so you need to disable SELinux or set to not enforcing.

DavidV · December 17, 2024, 9:09pm

I have gone ahead and navigated back to the directory and ran the command and though it completes with no errors the issue persists. Could you let me know where the makefile is located and what I should look for? I apologize for asking you to walk me through it. I am just a little lacking on experience.

sollericos · December 17, 2024, 9:32pm

The Makefile is where you extracted the source files before installing RT (see README - RT 5.0.7 Documentation - Best Practical). It’s where you executed ./configure... ; make testdeps... ; etc.

I usually leave these files in /usr/local/src so I can check the original parameters I added to ./configure.

DavidV · December 17, 2024, 9:38pm

I have run the command and no issues present upon completion.

Steps taken:

Navigated to /ect/SELinux/Config
Changed line from SELINUX=enforcing TO SELINUX=disabled
Rebooted my Server

The logs I am getting are as follows:

/var/log/rt5/cchs-rt.log
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 11. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 12. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 13. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 14. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 15. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 16. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 17. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1776)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: done. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1797)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: Creating attributes… (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1875)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 4. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1942)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 5. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1942)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 6. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1942)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 7. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1942)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: 8. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1942)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: done. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1945)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: Running final actions… (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1948)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: done. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1955)
[35040] [Tue Dec 17 05:08:24 2024] [debug]: Done setting up database content. (/usr/local/src/rt-5.0.7/sbin/…/lib/RT/Handle.pm:1967)

/var/log/httpd/error.log
[Tue Dec 17 13:11:38.739373 2024] [core:notice] [pid 4688:tid 4688] AH00094: Command line: ‘/usr/sbin/httpd -D FOREGROUND’
Log file ‘/Opt/rt5/var/log/cchs-rt.log’ couldn’t be written or created.

RT can’t run. at /opt/rt5/sbin/…/lib/RT.pm line 343.*
[Tue Dec 17 13:12:23.236137 2024] [fcgid:warn] [pid 4695:tid 4793] (104)Connection reset by peer: [client 192.168.254.136:53116] mod_fcgid: error reading data from FastCGI server
[Tue Dec 17 13:12:23.236191 2024] [core:error] [pid 4695:tid 4793] [client 192.168.254.136:53116] End of script output before headers: rt-server.fcgi
Log file ‘/Opt/rt5/var/log/cchs-rt.log’ couldn’t be written or created.
RT can’t run. at /opt/rt5/sbin/…/lib/RT.pm line 343.*
[Tue Dec 17 13:12:23.819587 2024] [fcgid:warn] [pid 4697:tid 4806] (104)Connection reset by peer: [client 192.168.254.136:53117] mod_fcgid: error reading data from FastCGI server, referer: http://cchs-rt.cchd.local/
[Tue Dec 17 13:12:23.819638 2024] [core:error] [pid 4697:tid 4806] [client 192.168.254.136:53117] End of script output before headers: rt-server.fcgi, referer: http://cchs-rt.cchd.local/

Will dig into this line: Log file ‘/Opt/rt5/var/log/cchs-rt.log’ couldn’t be written or created.

sollericos · December 17, 2024, 10:00pm

Is /opt/rt5/var and /opt/rt5/var/log writable by the user/group that run the server? Usually, www-data (WEB_USER and WEB_GROUP in Makefile)

DavidV · December 17, 2024, 10:37pm

So it seems as though in the make fixperms command it applies the apache user permissions to /opt/rt5/val/mason_data & session_data directories and the log directory is not present to receive the permissions.

Make the web ui readable by all.

chmod -R u+rwX,go-w,go+rX /opt/rt5/share/html
/opt/rt5/local/html
/opt/rt5/share/po
/opt/rt5/local/po
/opt/rt5/share/static
/opt/rt5/local/static
chown -R root /opt/rt5/share/html
/opt/rt5/local/html
/opt/rt5/share/po
/opt/rt5/local/po
/opt/rt5/share/static
/opt/rt5/local/static
chgrp -R root /opt/rt5/share/html
/opt/rt5/local/html
/opt/rt5/share/po
/opt/rt5/local/po
/opt/rt5/share/static
/opt/rt5/local/static

Make the web ui’s data dir writable

chmod 0770 */opt/rt5/var/mason_data *
/opt/rt5/var/session_data
chown -R apache */opt/rt5/var/mason_data *
/opt/rt5/var/session_data
chgrp -R apache */opt/rt5/var/mason_data *
/opt/rt5/var/session_data

I could not make heads or tails of where to add the directory to get permissions in the .configure file, so I just navigated to the directory and applied the apache user to the file permissions similar to the other two directories. I am getting the following logs in /var/log/httpd/error.log though the permissions error is no longer apparent.

[Tue Dec 17 14:36:42.443288 2024] [suexec:notice] [pid 7143:tid 7143] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
AH00558: httpd: Could not reliably determine the server’s fully qualified domain name, using fe80::c0e7:7fcf:4359:d6cf%ens192. Set the ‘ServerName’ directive globally to suppress this message
[Tue Dec 17 14:36:42.469189 2024] [lbmethod_heartbeat:notice] [pid 7143:tid 7143] AH02282: No slotmem from mod_heartmonitor
[Tue Dec 17 14:36:42.480499 2024] [mpm_event:notice] [pid 7143:tid 7143] AH00489: Apache/2.4.62 (CentOS Stream) mod_fcgid/2.3.9 mod_perl/2.0.12 Perl/v5.32.1 configured – resuming normal operations
[Tue Dec 17 14:36:42.480530 2024] [core:notice] [pid 7143:tid 7143] AH00094: Command line: ‘/usr/sbin/httpd -D FOREGROUND’

DavidV · December 17, 2024, 11:00pm

Oh no! Im stupid - haha found the file!

quite literally /usr/local/src/rt-5.0.7/makefile.in. I have made the follow changes and ran the make fixdeps, but its not applying the permissions (I manually undid my change and confirmed the error is appeared once more).

# Make upgrade scripts executable if they are in the source.
#
( cd etc/upgrade && find . -type f -not -name '*.in' -perm @FINDPERM@0111 -print ) | while read file ; do \
	chmod a+x "$(DESTDIR)$(RT_ETC_PATH)/upgrade/$$file" ; \
done

# Make the web ui readable by all. 
chmod -R  u+rwX,go-w,go+rX 	$(DESTDIR)$(MASON_HTML_PATH) \
				$(DESTDIR)$(MASON_LOCAL_HTML_PATH) \
				$(DESTDIR)$(RT_LEXICON_PATH) \
				$(DESTDIR)$(LOCAL_LEXICON_PATH) \
				$(DESTDIR)$(RT_STATIC_PATH) \
				$(DESTDIR)$(LOCAL_STATIC_PATH)
chown -R $(LIBS_OWNER) 	$(DESTDIR)$(MASON_HTML_PATH) \
			$(DESTDIR)$(MASON_LOCAL_HTML_PATH) \
			$(DESTDIR)$(RT_LEXICON_PATH) \
			$(DESTDIR)$(LOCAL_LEXICON_PATH) \
			$(DESTDIR)$(RT_STATIC_PATH) \
			$(DESTDIR)$(LOCAL_STATIC_PATH)
chgrp -R $(LIBS_GROUP) 	$(DESTDIR)$(MASON_HTML_PATH) \
			$(DESTDIR)$(MASON_LOCAL_HTML_PATH) \
			$(DESTDIR)$(RT_LEXICON_PATH) \
			$(DESTDIR)$(LOCAL_LEXICON_PATH) \
			$(DESTDIR)$(RT_STATIC_PATH) \
			$(DESTDIR)$(LOCAL_STATIC_PATH)

# Make the web ui's data dir writable
chmod 0770  	$(DESTDIR)$(MASON_DATA_PATH) \
		$(DESTDIR)$(MASON_SESSION_PATH)
		**$(DESTDIR)$(LOG_PATH)**
chown -R $(WEB_USER) 	$(DESTDIR)$(MASON_DATA_PATH) \
			$(DESTDIR)$(MASON_SESSION_PATH)
			**$(DESTDIR)$(LOG_PATH)** 
chgrp -R $(WEB_GROUP) 	$(DESTDIR)$(MASON_DATA_PATH) \
			$(DESTDIR)$(MASON_SESSION_PATH)
			**$(DESTDIR)$(LOG_PATH)**

dirs:
$(INSTALL) -m 0755 -d $(DESTDIR)$(RT_LOG_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(RT_FONT_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(RT_LEXICON_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(RT_STATIC_PATH)
$(INSTALL) -m 0770 -d $(DESTDIR)$(MASON_DATA_PATH)
$(INSTALL) -m 0770 -d $(DESTDIR)$(MASON_DATA_PATH)/cache
$(INSTALL) -m 0770 -d $(DESTDIR)$(MASON_DATA_PATH)/etc
$(INSTALL) -m 0770 -d $(DESTDIR)$(MASON_DATA_PATH)/obj
$(INSTALL) -m 0770 -d $(DESTDIR)$(MASON_SESSION_PATH)
$(INSTALL) -m 0770 -d $(DESTDIR)$(MASON_DATA_PATH)
$(INSTALL) -m 0770 -d $(DESTDIR)$(LOG_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(MASON_HTML_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(MASON_LOCAL_HTML_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(LOCAL_ETC_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(LOCAL_LIB_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(LOCAL_PLUGIN_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(LOCAL_LEXICON_PATH)
$(INSTALL) -m 0755 -d $(DESTDIR)$(LOCAL_STATIC_PATH)

Andrew_Ruthven · December 18, 2024, 7:52am

Something to double check, you’re using “/opt” and “/Opt”. Unix file systems are case sensitive. Which one is it?

GreenJimll · December 18, 2024, 10:37am

We run RT under SELinux (with Apache) on Oracle Linux 9 and it seems fine. We deploy instances using Ansible, and the relevant bits of the play are:

      - name: Deal with SELinux messing up the Mason cache and logs
        shell: 
          chcon -Rv -t httpd_sys_rw_content_t /opt/rt5/var

      - name: Set up the default contexts for the top level RT directory
        sefcontext:
            target: "{{ rtDirectory | default('/opt/rt5')}}{{ item.dir }}(/.*)?"
            seuser: unconfined_u
            setype: '{{ item.setype }}'
            state: present

        with_items:
          - dir: ''
            setype: usr_t

          - dir: '/var'
            setype: tmp_t

          - dir: '/var/mason_data/cache'
            setype: httpd_tmp_t

          - dir: '/bin'
            setype: httpd_user_content_t

          - dir: '/sbin'
            setype: bin_t