We are trying to migrate our database of public GPG keys that has already been working in RT4 (a rough estimate of size):
gpg --homedir /opt/rt4/var/data/gpg/ --list-keys | grep pub | wc -l
gpg: WARNING: unsafe ownership on homedir '/opt/rt4/var/data/gpg'
2050
After the migration, the UI Admin->Tools->Manage GnuPG Keys seems to hang and load. Hang on load / fail / … , as in, the browser just appears to be loading page but getting no output from the server.
Nothing in the Apache, nor in RT logs. Server CPU usage appears to be negligible. The Apache/FCGI/RT is running until the session hangs (or the admin restarts the apache2 via systemctl). RT works fine for other users in the mean time, though.
The host is Debian 12.2, the RT has been upgraded from 4.2.12 to 5.0.5 together with corresponding RT::IR; following the publicly documented upgrade steps and everything except the GnuPG keys appears to be working right now.
Just as a side note, as far as the RT4-RT5 upgrade is concerned: currently, a fresh install is (sadly) a no-go for us (although it would have been highly preferred by me).
How it began: the “upgrade RT and RT::IR in 4 steps”:
- clone the production VM to
upgrade-test - follow the RT4->RT5 and RT::IR upgrade path successfully (with its own caveats, snapshot rollbacks, re-tries, sanity checks, head-desks, etc)
- try to migrate the gpg keybase
3.5) explore whetherAdmin->Tools->Manage GnuPG Keysworks
3.6) discover the fact that the UI gets stuck on browsing the keys
3.99) cleargpgfolder, andabort, retry, continuestep 3 about 20 times
On the serious side: this is what have I tried after full migration to 5.0.5 (except GnuPG keyring):
At first, I have tried to copy the entire gpg folder. Fix permissions. Restart apache. Not working.
Copy just the pubring.kbx. Fix permissions. Restart Apache. Also no dice.
I have also been trying with:
gpg --homedir /opt/rt4/var/data/gpg/ --export > gpg_rt4_exported
cat gpg_rt4_exported | gpg --homedir /opt/rt5/var/data/gpg/ --import
chown -fR www-data:www-data /opt/rt5/var/data/gpg
chmod 600 -R /opt/rt5/var/data/gpg/*
systemctl restart apache2
The gpg folder contents:
ls -laFh /opt/rt5/var/data/gpg
total 21M
drwxr-xr-x 2 www-data www-data 4.0K May 15 07:50 ./
drwx------ 4 www-data www-data 4.0K May 15 07:43 ../
-rw------- 1 www-data www-data 11M May 15 07:44 pubring.kbx
-rw------- 1 www-data www-data 11M May 15 07:44 pubring.kbx~
-rw------- 1 www-data www-data 1.2K May 15 07:44 trustdb.gpg
ls -laFh /opt/rt4/var/data/gpg
total 20M
drwx------ 3 www-data www-data 4.0K Nov 8 2023 ./
drwx------ 3 www-data www-data 4.0K Apr 13 2023 ../
-rw-r--r-- 1 www-data www-data 0 Nov 2 2023 .gpg-v21-migrated
drwx------ 2 www-data www-data 4.0K Nov 2 2023 private-keys-v1.d/
-rw------- 1 www-data www-data 9.7M Oct 24 2023 pubring.gpg
-rw------- 1 www-data www-data 9.7M Oct 24 2023 pubring.gpg~
-rw------- 1 www-data www-data 600 Nov 2 2023 random_seed
-rw------- 1 www-data www-data 1.7K Mar 26 2017 secring.gpg
srwx------ 1 www-data www-data 0 Nov 8 2023 S.gpg-agent=
srwx------ 1 www-data www-data 0 Nov 8 2023 S.gpg-agent.browser=
srwx------ 1 www-data www-data 0 Nov 8 2023 S.gpg-agent.extra=
srwx------ 1 www-data www-data 0 Nov 8 2023 S.gpg-agent.ssh=
-rw------- 1 www-data www-data 1.2K Oct 24 2023 trustdb.gpg
The aforementioned “Manage GnuPG keys” page still refuses to load, even after extending timeouts in Apache2 config to high (or even higher, bizzare) numbers:
<IfModule mod_fcgid.c>
FcgidBusyTimeout 65534
FcgidIOTimeout 65534
</IfModule>
According to github source for GnuPG Admin tools, the fetch happens with RT::Crypt::GnuPG::GetKeysInfo, but with my highly basic and rough Perl knowledge, I have no idea what would have caused the “hang up”.
I have been also trying migrating the keys by using keyserver (to search for updated keys). Please excuse the ugliness of the “oneliner”:
NUM=206; chown -fR root:www-data /opt/rt5/var/data/gpg; chmod 600 -R /opt/rt5/var/data/gpg/*; \
for key in $(gpg --homedir /opt/rt4/var/data/gpg --list-keys | grep -A1 "pub " | grep -e "^ " | grep -oE ".{16}$" | tail -n $NUM ); \
do echo $key; if [[ $(gpg --homedir /opt/rt5/var/data/gpg --list-keys $key > /dev/null; echo $?) -eq 2 ]]; \
then echo "$key not existant, getting it"; gpg --batch --homedir /opt/rt5/var/data/gpg/ --keyserver hkps://pgp.circl.lu --receive-keys $key ; \
else echo "$key EXISTS"; fi; done; chown -fR www-data:www-data /opt/rt5/var/data/gpg; chmod 600 -R /opt/rt5/var/data/gpg/*; systemctl restart apache2
TL;DR: dump last (or first if I use head instead of tail) NUM public keys, get fingerprints, search on keyserver, receive key, and insert them into keyring, fix permissions, restart; don’t try the same key twice if you happen to run the oneliner two times on non-clean gpg dir.
I have been trying how many keys I can insert into the RT GPG keyring with | head -n $NUM and | tail -n $NUM. (The GPG folder has been wiped clean between the attempts.)
By using head, the above “oneliner” migrates first 134 keys. When I try to add 135th key, the GPG keys management UI fails to load.
If I try with tail, it imports 206 keys until the UI fails to load after importing 207th.
Same manner happens if i skip a key; i.e. import 1-135, but skip 134 - the UI will fail to load after importing 136th; and likewise, import last 208 keys by skipping 207th, the UI will fail to load after importing 209th.
If i try to selectively import import i.e. GPG keys 135, 136, 137 from the start of the old keyring, or 207th, 208th, 209th from the end, the GPG keys management UI works and the keys appear in there, so I assume that there is no issue with the keys themselves (meaning they are OK, they exist on keyserver, they get ingested into the local GPG database just fine)
The difference (head 135, tail 206) also makes me think there is nothing wrong with any for loops itself in any manner in the source code, as this would have most highly likely caused the UI loading freeze on same NUM of keys, regardless of using head or tail.
I would gladly take any ideas, clues, pointers or directions in what (and how) to try in order to migrate our old public GPG keyring. (The private keyring is, I presume, far easier to handle, although I am probably yet to discover that. :))
Sorry for the possible misplacement of this wall of text in the forum and also kind thanks in advance,
KW