WebRemoteUserAutocreate not working with REMOTE_USER

When using auth_openidc module with apache and authenticating with external IdP KeyCloak, it sets REMOTE_USER variable properly, but RT does not autocreate users

User story: When user goes to RT page, user gets redirected to IdP to authenticate, where IdP also adds cookie to the browser, after successful authentication redirects back to RT login page with REMOTE_USER header, but RT does not create new user and not logging in.

Does anyone know what the problem is?

Settings in RT 5.0.2:

Set($WebRemoteUserAuth, 1);
Set($WebRemoteUserContinuous, 1);
Set($WebRemoteUserAutocreate, 1);
Set($WebFallbackToRTLogin, 1);
Set($UserAutocreateDefaultsOnLogin, { Privileged => 1 } );

What do you see in the logs when this user tries to login?

RT logs in debug mode display nothing at all

Though apache logs show a lot, and the bottom part shows that there is remote user loggedin as demo1

==> /var/log/apache2/access.log <==
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /rtir/ HTTP/2.0" 302 1539 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/realms/demorealm/protocol/openid-connect/auth?response_type=code&scope=openid%20email%20profile&client_id=democlient&state=43MdhxZPNgJCUGv_VlJznxeC-WA&redirect_uri=https%3A%2F%2Fext.dom.ain.name%2Frtir%2Fsso&nonce=uuZGImyjuwuK9DCoRBJVHb0tHr5WebepKwN12CenpgY HTTP/2.0" 200 3231 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/common/keycloak/web_modules/@patternfly/react-core/dist/styles/base.css HTTP/2.0" 200 6070 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/common/keycloak/node_modules/patternfly/dist/css/patternfly.min.css HTTP/2.0" 200 31966 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/common/keycloak/web_modules/@patternfly/react-core/dist/styles/app.css HTTP/2.0" 200 51768 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/common/keycloak/lib/pficon/pficon.css HTTP/2.0" 200 626 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/login/keycloak/css/login.css HTTP/2.0" 200 3535 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/login/keycloak/css/tile.css HTTP/2.0" 200 1567 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:30 +0000] "GET /auth/resources/q0whe/common/keycloak/node_modules/patternfly/dist/css/patternfly-additions.min.css HTTP/2.0" 200 31155 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:31 +0000] "GET /auth/resources/q0whe/login/keycloak/img/favicon.ico HTTP/2.0" 200 955 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:31 +0000] "GET /auth/resources/q0whe/common/keycloak/node_modules/patternfly/dist/img/bg-login.jpg HTTP/2.0" 200 47918 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:31 +0000] "GET /auth/resources/q0whe/login/keycloak/img/keycloak-bg.png HTTP/2.0" 200 82115 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:31 +0000] "GET /auth/resources/q0whe/common/keycloak/web_modules/@patternfly/react-core/dist/styles/assets/fonts/overpass-webfont/overpass-regular.woff2 HTTP/2.0" 200 35396 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:31 +0000] "GET /auth/resources/q0whe/common/keycloak/web_modules/@patternfly/react-core/dist/styles/assets/fonts/overpass-webfont/overpass-light.woff2 HTTP/2.0" 200 34740 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - - [19/Nov/2021:16:41:33 +0000] "POST /auth/realms/demorealm/login-actions/authenticate?session_code=iXqvo5zN8jMdvO_4yq0aYd4ItSQ-osN2NkCh7sDtrwg&execution=9b40d7fd-99c3-4754-a195-3df08735b10f&client_id=democlient&tab_id=h8nEukZLQY8 HTTP/2.0" 302 2938 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
bbb.bbb.bbb.bbb - - [19/Nov/2021:16:41:33 +0000] "POST /auth/realms/demorealm/protocol/openid-connect/token HTTP/2.0" 200 4079 "-" "mod_auth_openidc"
bbb.bbb.bbb.bbb - - [19/Nov/2021:16:41:33 +0000] "GET /auth/realms/demorealm/protocol/openid-connect/userinfo HTTP/2.0" 200 501 "-" "mod_auth_openidc"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:33 +0000] "GET /rtir/sso?state=43MdhxZPNgJCUGv_VlJznxeC-WA&session_state=577179c5-bc91-440c-919d-34575572ebcb&code=07431e9c-ab50-4817-8a55-36717007e4cd.577179c5-bc91-440c-919d-34575572ebcb.240f36e3-eedc-4685-9e73-cf029c5b7b92 HTTP/2.0" 302 610 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:33 +0000] "GET /rtir/ HTTP/2.0" 200 2328 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/static/images/request-tracker-logo.svg HTTP/2.0" 200 10049 "https://ext.dom.ain.name/rtir/" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/NoAuth/css/elevator-light/squished-6fba3aaf5f8eb27a40b2ccb9507570e3.css HTTP/2.0" 200 47552 "https://ext.dom.ain.name/rtir/" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/NoAuth/js/squished-3fcf0554ef23ebbe9cc0dc63861df04e.js HTTP/2.0" 200 432383 "https://ext.dom.ain.name/rtir/" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/static/css/fonts/inter/Inter-Medium.woff2 HTTP/2.0" 200 126043 "https://ext.dom.ain.name/rtir/NoAuth/css/elevator-light/squished-6fba3aaf5f8eb27a40b2ccb9507570e3.css" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/static/css/fonts/inter/Inter-Regular.woff2 HTTP/2.0" 200 118915 "https://ext.dom.ain.name/rtir/NoAuth/css/elevator-light/squished-6fba3aaf5f8eb27a40b2ccb9507570e3.css" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/static/images/favicon.png HTTP/2.0" 200 564 "https://ext.dom.ain.name/rtir/" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"
aaa.aaa.aaa.aaa - demo1 [19/Nov/2021:16:41:34 +0000] "GET /rtir/NoAuth/js/squished-3fcf0554ef23ebbe9cc0dc63861df04e.js HTTP/2.0" 200 432533 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0"

aaa.aaa.aaa.aaa = my own IP
bbb.bbb.bbb.bbb = keycloak/apache/RT behind another external IP

Do you have RT set to debug level? I’d expect to see something about how users are being authenticated

Yes, I have RT set in debug mode

Set($LogToFile,      'debug');
Set($LogToFileNamed, 'rt5.log');
Set($LogDir,         '/var/log/rt5');

P.S. Local user login works fine as expected, but not via External Authentication via apache OpenIDC module

Oh so external auth login doesn’t work? Not just the creating of non-existent users within RT? If you login via KeyCloak with a user that does exist in RT already does it still fail?

Yes, it fails all parts when logging in with external user:

  1. does not create that external user inside RT
  2. still shows login screen (though there is present REMOTE_USER variable in the environment)
  3. nothing happens… though it could show some errors or whatever, that missing some headers, or something CSRF related - nothing is visible in the logs…

it just seems, RT does not respect REMOTE_USER variable, and it is not bypassing login screen form

Last line in RT logs, is when it started:

Trace begun at /opt/rt5/sbin/../lib/RT.pm line 314
Log::Dispatch::__ANON__('Log::Dispatch=HASH(0x55a86134a308)', 'The RTAddressRegexp option is not set in the config. Not setting this option results in additional SQL queries to check whether each address belongs to RT or not. It is especially important to set this option if RT receives emails on addresses that are not in the database or config.') called at /opt/rt5/sbin/../lib/RT/Config.pm line 664
RT::Config::__ANON__('RT::Config=HASH(0x55a8601a8078)', undef) called at /opt/rt5/sbin/../lib/RT/Config.pm line 2281
RT::Config::PostLoadCheck('RT::Config=HASH(0x55a8601a8078)') called at /opt/rt5/sbin/../lib/RT.pm line 209
RT::Init('RT', 'Heavy', 1) called at /opt/rt5/sbin/rt-server line 126

there is nothing aftrerwards… very odd

I am not using anything for ExternalSettings, though there is a browser cookie set by IdP (external authenticator keycloak), which I probably need to define in $ExternalSettings ? or not?
RT is using Postgres, but documentation mentions only MySQL table for SSO Cookie… but is 100% unclear, how to use it… googling does not help at all!
I could probably make some kind of mappings from the browser cookie, which I can see in Chrome (Developer Tools->Application->Storage) or Firefox (Web developer tools->Storage->Cookies) or similar places if there was enough documentation present… but unfortunately all documentation is so vague, that I cannot make proper configs :frowning:

If you go to Admin->Tools->System Configuration in the web UI, is WebRemoteUserAuth set to 1? If I recall correctly that should be all thats needed. You can also disable WebFallbackToRTLogin to maybe see a change in behavior and confirm that RT is trying to use the REMOTE_USER variable

Yes, it is set to 1 from SiteConfig

Hey there, I recently got this working with Authelia in front of RT5. The biggest challenge was understanding REMOTE_USER behavior when you are authenticating on an external reverse proxy.

Can you try something like the following:

Apache config:

# log %{REMOTE_USER} in access requests for troubleshooting    
CustomLog logs/ssl_request_log \
    "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b \"%{REMOTE_USER}i\""

# Authelia was sending the Remote-User header 
# so map it to the Environment variable REMOTE_USER
SetEnvIfNoCase Remote-User "(.*)" REMOTE_USER=$1

RT_SiteConfig.pm:

Set($WebRemoteUserAuth, 1);
Set($WebRemoteUserContinuous, 1);
Set($WebFallbackToRTLogin, 0);

I haven’t tested WebRemoteUserAutoCreate but the above worked for me. Hopefully there’s something in the above that helps you to solve the puzzle :slight_smile:

Example logs:

[18/Nov/2021:22:59:06 +0000] 192.168.XXX.YYY TLSv1.3 TLS_AES_256_GCM_SHA384 "GET / HTTP/1.1" 163837 "my@username.example"
[18/Nov/2021:22:59:08 +0000] 192.168.XXX.YYY TLSv1.3 TLS_AES_256_GCM_SHA384 "GET /Ticket/Display.html?id=XXX HTTP/1.1" 102178 "my@username.example"
[18/Nov/2021:22:59:09 +0000] 192.168.XXX.YYY TLSv1.3 TLS_AES_256_GCM_SHA384 "GET /static/css/fonts/inter/Inter-Italic.woff2 HTTP/1.1" 126120 "my@username.example"