Locked resources post execution

[suhasdotcom]

Locked resources post execution

Describe the bug

Resources are locked in the --homedir when keys are generated using python-gnupg on windows-11.

To Reproduce

Steps to reproduce the behavior:

1. Simply generate a key in a directory (homedir) using this script:

   # This script is just for illustration purposes and should not be taken as a good reference on how to generate keys 
   # using the python-gnupg lib.
   # 
   # Things bad in this script:
   # * bad passphrase selection
   # * bad comment selection
   
   import gnupg
   from pathlib import Path
   
   # 1. generate a gpg instance
   gpg = gnupg.GPG(gpgbinary=Path('C:', 'Program Files (x86)', 'GnuPG', 'bin', 'gpg.exe'),
               gnupghome=Path('D:', 'TestTmpDir', 'python-gnupg-lock-issue', '.gpg-home'), # setting up a homedir
               verbose=True)
   
   # 2. generate stdin input for --gen-key
   the_key_cmd = gpg.gen_key_input(name_real='<masked>', name_email='<masked>', key_type='dsa', passphrase='any-pass',
                                   name_comment="a sample comment")
   
   # 3. generate actual key
   gpg.gen_key(the_key_cmd)
   print('Generated keys:')
   print(gpg.list_keys())

   The key gets generated in the homedir.
2. Try to get the generated keys using gpg CLI from homedir:

   $ gpg --homedir=.gpg-home/ --list-keys
   gpg: invalid size of lockfile '/tmp/python-gnupg-test/keygen/.gpg-home/pubring.kbx.lock'
   gpg: cannot read lockfile
   gpg: can't lock '/tmp/python-gnupg-test/keygen/.gpg-home/pubring.kbx'
   /tmp/python-gnupg-test/keygen/.gpg-home/pubring.kbx
   ---------------------------------------------------
   pub   dsa2048 2025-06-14 [SCA]
         A4184... <some-masked-key>
   uid           [ultimate] <masked> (a comment here and there) <<masked>>
   

   The lock-files persist even after the keys are generated.
3. list all files in homedir and see the lockfiles:

   $ ls -al .gpg-home/
   total 24
   drwxr-xr-x 1 suhas 197609    0 Jun 14 18:39 .
   drwxr-xr-x 1 suhas 197609    0 Jun 14 13:28 ..
   -rw-r--r-- 1 suhas 197609    0 Jun 14 13:27 gnupg_spawn_agent_sentinel.lock
   drwxr-xr-x 1 suhas 197609    0 Jun 14 18:39 openpgp-revocs.d
   drwxr-xr-x 1 suhas 197609    0 Jun 14 18:39 private-keys-v1.d
   -rw-r--r-- 1 suhas 197609 1196 Jun 14 13:27 pubring.kb_
   -rw-r--r-- 1 suhas 197609 2360 Jun 14 18:39 pubring.kbx
   -rw-r--r-- 1 suhas 197609    0 Jun 14 12:16 pubring.kbx.lock
   -rw-r--r-- 1 suhas 197609 1360 Jun 14 18:39 trustdb.gpg
   -rw-r--r-- 1 suhas 197609    0 Jun 14 12:28 trustdb.gpg.lock
   

   Here, gnupg_spawn_agent_sentinel.lock, pubring.kbx.lock and trustdb.gpg.lock are locks on resources which produce the error warnings illustrated previously.
4. Try signing a file in using this homedir:

   $ gpg --homedir=.gpg-home --sign --detach --armor a-file
   gpg: invalid size of lockfile '/tmp/python-gnupg-test/keygen/.gpg-home/pubring.kbx.lock'
   gpg: cannot read lockfile
   gpg: can't lock '/tmp/python-gnupg-test/keygen/.gpg-home/pubring.kbx'
   gpg: invalid size of lockfile '/tmp/python-gnupg-test/keygen/.gpg-home/gnupg_spawn_agent_sentinel.lock'
   gpg: cannot read lockfile
   gpg: can't connect to the gpg-agent: Invalid argument
   gpg: invalid size of lockfile '/tmp/python-gnupg-test/keygen/.gpg-home/gnupg_spawn_agent_sentinel.lock'
   gpg: cannot read lockfile
   gpg: can't connect to the gpg-agent: Invalid argument
   gpg: keydb_search failed: No agent running
   gpg: no default secret key: No agent running
   gpg: a-file: clear-sign failed: No agent running
   

5. Delete the lock-files and try signing file in shell:

   $ rm -rf .gpg-home/*.lock
   $ gpg --homedir=.gpg-home --sign --detach --armor a-file
   $ gpg --homedir=.gpg-home --verify a-file.asc a-file
   gpg: Signature made Sat Jun 14 18:52:06 2025 IST
   gpg:                using DSA key A4184... <some-masked-key>
   gpg: Good signature from "<masked> (a comment here and there) <<masked>>" [ultimate]
   

Expected behavior

• Keys generated from python-gnupg lib should be usable and accessible from the CLI as well.
• All resources should be released when subprocess exits, i.e. no lock-files.

Screenshots

A bit verbose, but this is the key generation log:

$ GPG_TEST_PATH="C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen" python -m python_gnupg_lock_issue
INFO:root:gpg_test_path: C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen
"C:Program Files (x86)\GnuPG\bin\gpg.exe" --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen\.gpg-home --list-config --with-colons
DEBUG:gnupg:7060: "C:Program Files (x86)\GnuPG\bin\gpg.exe" --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen\.gpg-home --list-config --with-colons
DEBUG:gnupg:stderr reader: <Thread(Thread-1 (_read_response), initial daemon)>
DEBUG:gnupg:stdout reader: <Thread(Thread-2 (_read_data), initial daemon)>
DEBUG:root:Key-Type: dsa
Name-Real: <masked>
Name-Email: <masked>
Passphrase: any-pass
Name-Comment: a comment here and there
Key-Length: 2048
%commit

"C:Program Files (x86)\GnuPG\bin\gpg.exe" --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen\.gpg-home --gen-key
DEBUG:gnupg:8548: "C:Program Files (x86)\GnuPG\bin\gpg.exe" --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen\.gpg-home --gen-key
DEBUG:gnupg:data copier: <Thread(Thread-3 (_copy_data), initial daemon)>, <_io.BytesIO object at 0x000001E7DAE52390>, <_io.BufferedWriter name=3>
DEBUG:gnupg:stderr reader: <Thread(Thread-4 (_read_response), initial daemon)>
DEBUG:gnupg:closed output, 140 bytes sent
DEBUG:gnupg:stdout reader: <Thread(Thread-5 (_read_data), initial daemon)>
gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest size
DEBUG:gnupg:gpg: WARNING: some OpenPGP programs can't handle a DSA key with this digest size
[GNUPG:] KEY_CONSIDERED A4184... <some-masked-key> 0
DEBUG:gnupg:[GNUPG:] KEY_CONSIDERED A4184... <some-masked-key> 0
DEBUG:gnupg:message ignored: KEY_CONSIDERED, A4184... <some-masked-key> 0
gpg: revocation certificate stored as 'C:\\Users\\suhas\\AppData\\Local\\Temp\\python-gnupg-test\\keygen\\.gpg-home\\openpgp-revocs.d\\A4184... <some-masked-key>.rev'
DEBUG:gnupg:gpg: revocation certificate stored as 'C:\\Users\\suhas\\AppData\\Local\\Temp\\python-gnupg-test\\keygen\\.gpg-home\\openpgp-revocs.d\\A4184... <some-masked-key>.rev'
[GNUPG:] KEY_CREATED P A4184... <some-masked-key>
DEBUG:gnupg:[GNUPG:] KEY_CREATED P A4184... <some-masked-key>
INFO:root:Signing key generated.
"C:Program Files (x86)\GnuPG\bin\gpg.exe" --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen\.gpg-home --list-keys --fingerprint --fingerprint --with-keygrip
DEBUG:gnupg:8716: "C:Program Files (x86)\GnuPG\bin\gpg.exe" --status-fd 2 --no-tty --no-verbose --fixed-list-mode --batch --with-colons --homedir C:\Users\suhas\AppData\Local\Temp\python-gnupg-test\keygen\.gpg-home --list-keys --fingerprint --fingerprint --with-keygrip
DEBUG:gnupg:stderr reader: <Thread(Thread-6 (_read_response), initial daemon)>
DEBUG:gnupg:stdout reader: <Thread(Thread-7 (_read_data), initial daemon)>
gpg: checking the trustdb
DEBUG:gnupg:gpg: checking the trustdb
[GNUPG:] KEY_CONSIDERED A4184... <some-masked-key> 0
DEBUG:gnupg:[GNUPG:] KEY_CONSIDERED A4184... <some-masked-key> 0
[GNUPG:] KEY_CONSIDERED A4184... <some-masked-key> 0
DEBUG:gnupg:[GNUPG:] KEY_CONSIDERED A4184... <some-masked-key> 0
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
DEBUG:gnupg:gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
DEBUG:gnupg:gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
tru:o:1:1749887827:1:3:1:5
DEBUG:gnupg:line: 'tru:o:1:1749887827:1:3:1:5'
pub:u:2048:17:C0B10... <some-masked-key>:1749887819:::u:::scaSCA::::::23::0:
DEBUG:gnupg:line: 'pub:u:2048:17:C0B10... <some-masked-key>:1749887819:::u:::scaSCA::::::23::0:'
fpr:::::::::A4184... <some-masked-key>:
DEBUG:gnupg:line: 'fpr:::::::::A4184... <some-masked-key>:'
grp:::::::::49C79... <some-masked-key>:
DEBUG:gnupg:line: 'grp:::::::::49C79... <some-masked-key>:'
uid:u::::1749887819::51477... <some-masked-key>::<masked> (a comment here and there) <<masked>>::::::::::0:
DEBUG:gnupg:line: 'uid:u::::1749887819::51477... <some-masked-key>::<masked> (a comment here and there) <<masked>>::::::::::0:'
pub:u:2048:17:96504... <some-masked-key>:1749906536:::u:::scaSCA::::::23::0:
DEBUG:gnupg:line: 'pub:u:2048:17:96504... <some-masked-key>:1749906536:::u:::scaSCA::::::23::0:'
fpr:::::::::A4184... <some-masked-key>:
DEBUG:gnupg:line: 'fpr:::::::::A4184... <some-masked-key>:'
grp:::::::::54479... <some-masked-keygrp>:
DEBUG:gnupg:line: 'grp:::::::::54479... <some-masked-keygrp>:'
uid:u::::1749906536::51477... <some-masked-key>::<masked> (a comment here and there) <<masked>>::::::::::0:
DEBUG:gnupg:line: 'uid:u::::1749906536::51477... <some-masked-key>::<masked> (a comment here and there) <<masked>>::::::::::0:'
DEBUG:root:Generated signing key: [{'type': 'pub', 'trust': 'u', 'length': '2048', 'algo': '17', 'keyid': 'C0B10... <some-masked-key>', 'date': '1749887819', 'expires': '', 'dummy': '
', 'ownertrust': 'u', 'sig': '', 'cap': 'scaSCA', 'issuer': '', 'flag': '', 'token': '', 'hash': '', 'curve': '', 'compliance': '23', 'updated': '', 'origin': '0', 'keygrip'
: '49C79... <some-masked-key>', 'uids': ['<masked> (a comment here and there) <<masked>>'], 'sigs': [], 'subkeys': [], 'fingerprint': 'A4184... <some-masked-key>'}, {'type': 'pub', 'trust': 'u', 'length': '2048', 'algo': '17', 'keyid': '96504... <some-masked-key>', 'date': '1749906536', 'expires': '', 'dummy': '', 'ownertrust'
: 'u', 'sig': '', 'cap': 'scaSCA', 'issuer': '', 'flag': '', 'token': '', 'hash': '', 'curve': '', 'compliance': '23', 'updated': '', 'origin': '0', 'keygrip': '54479FEED214
<some-masked-keygrp>', 'uids': ['<masked> (a comment here and there) <<masked>>'], 'sigs': [], 'subkeys': [], 'fingerprint': 'A4184... <some-masked-key>'}]

Environment

• OS: Windows 11 Home 24H2
• Version of this library: 0.5.4
• Version of GnuPG: 2.4.5-unknown

  $ gpg --version
  gpg (GnuPG) 2.4.5-unknown
  libgcrypt 1.9.4-unknown
  Copyright (C) 2024 g10 Code GmbH
  License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
  This is free software: you are free to change and redistribute it.
  There is NO WARRANTY, to the extent permitted by law.
  
  Home: /c/Users/suhas/.gnupg
  Supported algorithms:
  Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
  Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
          CAMELLIA128, CAMELLIA192, CAMELLIA256
  Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
  Compression: Uncompressed, ZIP, ZLIB, BZIP2
  

[vsajip]

python-gnupg does not itself lock any files, nor does it create any lockfiles, so this looks like it's a GnuPG issue. Can you check with more recent versions of gpg? Also, is this error repeatable? Have you checked the GnuPG issue tracker to see if it's a known issue there?

[suhasdotcom]

Really appreciate your prompt response @vsajip .
I apologiese for my delayed response as I was travelling and didn't have time to experiment some more.

I'll check all the mentioned actions today and be back with my findings.

[suhasdotcom]

I guess this is a gpg issue since when I tried to do a simple subprocess.run() to run gpg, I still got the lock files.
This is what I ran:

subprocess.run([gpg_bin_path, '--homedir', str(gpg_dot_home_homedir), '--batch', '--gen-key'], check=True, cwd=gpg_test_root, input=b'Key-Type: dsa\nName-Real: <masked>\nName-Email: <masked>\nName-Comment: a comment here and there\nKey-Length: 2048\n%no-protection\n%commit', text=False)

I tried to check the gpg bug-tracker but there is a registration process for its view access, so I've requested for registration now. Let's wait for the approval and then I can check if this bug was already highlighted over there.

Also, I guess that as I am using a windows machine hence I should try to get this running using GPG4WIN instead of plain gpg which came bundled with git+mingw. The thing is, that I can't find the binary/executable to run from GPG4WIN.

If possible, and as the resources are scarce for this, may I request you to point me the gpg4win executable to run from the subprocess?

Thanks

[vsajip]

I'm afraid I can't help you, as I don't use gpg4win. I can tell you that in the CI setup for this project, I use Chocolatey to install gpg on Windows. You could try the same approach.

[suhasdotcom]

Thanks @vsajip . I've now raised this issue with GPG.

[suhasdotcom]

Hi @vsajip ,
I do not know whether this reopens the issue but this is a response that I got from GPG folks:

On Sun, 22 Jun 2025 21:46, Suhas Srivastava said:

> library. After examining that issue, the author suggested that the leftover
> lock files may be a gpg issue and not a library issue. The whole error

These lock files like

  ~/.gnupg/#lk0x0000558d3fe5e990.foo.24728

are created as part of atomic updates.  If an update fails for example
due to improper spawning of the gpg binary, gpg's cleanup can't run and
thus the files will not removed.  All classical Unix installations
employ a cron job to remove files name ".#*" after a few days.

If you have a lot of them there might be a problem in your python-gnupg
library (SIGPIPE ignored?).  I would suggest to switch to a gpgme based
library like gpgme 1.24 or the new gpgmepy 2.0.0.

• I know the lock file pattern mentioned here doesn't match the lock file patterns that appear on windows, but is it possible that there maybe a broken/ignored SIGPIPE?
• As they suggested, is the gpg binary being spawned incorrectly in my call?:

  pg = gnupg.GPG(gpgbinary=Path('C:', 'Program Files (x86)', 'GnuPG', 'bin', 'gpg.exe'),
              gnupghome=Path('D:', 'TestTmpDir', 'python-gnupg-lock-issue', '.gpg-home'), # setting up a homedir
              verbose=True)

PS: installing gpgme or gpgmepy is a real pain on windows and as majority of my software users are on windows thus, I am leaning more towards the subprocess (and hence, python-gnupg) approach.

[vsajip]

a response that I got from GPG folks

Yes, I saw that on the gpg-users mailing list. Werner said

are created as part of atomic updates

and I'm not sure what is meant by "atomic updates" or why they would be happening when you just spawn the gpg process.

is it possible that there maybe a broken/ignored SIGPIPE?

SIGPIPE isn't a thing on Windows - it's a thing that you normally only get on Linux-like operating systems. You're not using Windows Subsystem for Linux, are you?

gpg binary being spawned incorrectly in my call?

That call sets up a gnupg.GPG instance for further operations. It does call gpg, but only to determine the version and one or two other things. There are other method calls on the GPG instance which invoke gpg again, e.g. to generate keys, encrypt/decrypt, sign/verify etc. and I would guess it's one of those where the problem is occurring.

PS: installing gpgme or gpgmepy is a real pain on windows and as majority of my software users are on windows

Yes. Why do you think I maintain this project despite the officially blessed way being gpgme 😉 ? Believe me, the subprocess-based approach is a lot easier. (I've used the approach with other languages than Python too, with success.) IME the gpgme approach is more painful on other platforms, too.

I suggest you try some or all of the following:

1. Run test_gnupg.py from the source distribution of this project. Does it run successfully?
2. Try upgrading to GnuPG 2.4.8 and see if the problem goes away. (It's available using Chocolatey - the GitHub CI for this project installs it on their test runners.)
3. Try locating all your files (including gpg.exe) in a directory without spaces. Some Unix-centric developers don't always consider spaces in filenames as carefully as they should.
4. Disable any anti-virus software. It sometimes opens newly-created files for scanning for viruses, preventing deletion. See if that cures the issue. If you are forced to use anti-virus software and can't disable it for corporate-related reasons, consider running a VM (assuming that's allowed) and doing your testing in there.
5. Write a minimal standalone script which demonstrates the problem. Enable logging in this script, create throwaway keys.
6. The logging will show the exact command lines used to invoke gpg when you run the script. Copy those from the log file into a .cmd file, adapting as needed, and run that as far as possible, to see exactly where the lock files are being created. If they are being created and not being tidied up, that means it can't be python-gnupg as you're just running a .cmd file.

[suhasdotcom]

Thanks for the detailed analysis, I'll try these and get back whether the issue persists.