Rt-mailgate suddenly can't connect to RT instance due to SSL error

Hello all,

I use Fetchmail to process emails and hand them to rt-mailgate. Nginx hosts RT on the same server as fetchmail. RT is secured with a wildcard certificate we use for all our subdomains. This cert was renewed a couple months ago and has had no problems.

My problem is that, sometime over the weekend (May 29-31), RT stopped processing emails entirely. The error in the fetchmail log is:

/opt/rt4/bin/rt-mailgate: connecting to https://tickets.example.com/REST/1.0/NoAuth/mail-gateway
HTTP request failed: 500 Can't connect to tickets.example.com:443 (certificate verify failed). Your webserver logs may have more information or there may be a network problem.

I’ve tried everything I could think of, and am completely out of ideas and web search results. I have:

  • tried http, https, and no protocol at all for the --url option
  • tried specifying port 443 in the URL
  • tried using rt.example.com and tickets.example.com, as Nginx is set up to respond to both of those
  • tried a couple different file paths using the --ca-file option for rt-mailgate, but our cert is from Comodo, so that shouldn’t be necessary
  • tried using --no-verify-ssl
  • restarted RT
  • reloaded, though not restarted, Nginx
  • gone to the URL that rt-mailgate is trying in a browser, and not gotten a security warning in Firefox
  • checked that other RT pages don’t have security problems–they don’t, as far as I can tell
  • checked the Nginx logs for the RT site, and found nothing at all in the error log, and no requests from the server’s IP in the access log

Since I assume you’ll want to see it, here’s the fetchmail file I’ve been using as a test. The full file has a bunch of queue addresses, but I’ve made a test file that only tries to process one queue’s messages for now. This file causes the problem to happen.

set logfile /var/log/test_fetchmail.log

poll imap.gmail.com proto IMAP port 993:
username tickets+queueName@example.com password PASSWORD
	mda "/opt/rt4/bin/rt-mailgate --no-verify-ssl --debug --url https://tickets.example.com --queue queueName --action correspond" options ssl folder queue_folder

Of course, the --no-verify-ssl is new. With or without it, I get the exact same error in the fetchmail log, and nothing whatsoever in the Nginx access/error logs.

I have no idea where to go next, nor do I know what changed to cause this to suddenly start happening. Any suggestions anyone has are appreciated. Thanks for reading.

Try

openssl s_client -showcerts -connect tickets.example.com:443

That may show problems that affect older clients but not most web browsers. In particular, you could be getting hit by the Sectigo AddTrust External CA Root expiration, see

https://support.sectigo.com/articles/Knowledge/Sectigo-AddTrust-External-CA-Root-Expiring-May-30-2020

so look for expired certificates in the trust chain. If your web server is serving up expired intermediaries, you’ll need to remove those from the servers certificate chain.

1 Like

Thank you! The openssl command shows that the Sectigo cert is no longer valid. I think you got the problem exactly right. I’ll see what I can do to fix this (I’m not well-versed in SSL). My version of Debian is pretty old, so hopefully this is as simple as an OpenSSL update or finding a newer cert.