GPGSigner (and gpg.functions.create_signature) produce invalid signature dicts

[jku]

        signer = GPGSigner(self.signing_subkey_keyid, self.gnupg_home)
        signature = signer.sign(self.test_data)
        print(signature.to_dict())

The result does not look like a TUF compliant signature (notice the key name "signature" instead of "sig"):

{
  'keyid': 'c5a0abe6ec19d0d65f85e2c39be9df5131d924e9',
  'signature': '5630c9b9436ed438cdf1386abe9ca3201612c24dee868e7294f6f4592fa05820a194cf1f87fd21f5ea0a596578a20491ca8829ef8d352df1139a953e28cf2176e3c3dc484e669782f65a1cc7ac4d1798ef3874e1f4a113aa59e5b6d350aacfa3c9c58a3ef2770995396971db3aae93d122a73795bb6b6b8d6e5b49a9be64bed8c13f849b09d43c7e916366a6290e92e41987636107703147b2fc86eec59909ddf1d002f39655c11176c18f04b927ecce48781258aa4e8f715014e2e30332839475528afa7faac0c48b425984eb726176092a2cbc3ece7d7e7eea372b8ffdf82715564536b961e5d195f1746b578ccaea19a5eb5936a2d38e0187af490ecc902a',
  'other_headers': '04000108001d162104c5a0abe6ec19d0d65f85e2c39be9df5131d924e90502635bbb2e'
}

Did no-one ever actually try to put a GPG signature in TUF metadata?

[jku]

• The source of this issue is that gpg.create_signature() and GPG_SIGNATURE_SCHEMA use "signature" instead of "sig". I assume they too should change to be a superset of SIGNATURE_SCHEMA, but GPGSigner.sign() return value definitely needs to change.
• it's IMO made more difficult to see by GPGSignature which has no purpose that I can see: we could remove the class completely if we make GPGSigner.sign() use return Signature.from_dict(sig_dict) instead of return GPGSignature(**sig_dict)

[jku]

WRT possible fixes:

• The draft PR is the "fix everything and break API" option
• another option is to leave the "signature" field in the dict along with "sig" -- that just feels wasteful EDIT: also likely does not work as this won't match either signature schema
• yet another is to not touch GPG_SIGNATURE_SCHEMA and gpg.functions.create_signature() at all, and just fix the GPGSigner.sign() return value -- that definitely should be inline with Signer.sign() return value

[jku]

If we change GPG_SIGNATURE_SCHEMA, then old signatures (like the ones in in-toto metadata) will start to fail securesystemslib.formats.ANY_SIGNATURE_SCHEMA.check_match() and also won't verify successfully any more so... I suppose we need some backwards compatible fix :(

[lukpueh]

Did no-one ever actually try to put a GPG signature in TUF metadata?

I actually did, but not beyond memory (see release post on slack):

>>> from tuf.api.metadata import Metadata, Root
>>> from securesystemslib.signer import GPGSigner
>>>
>>> root = Metadata(signed=Root()) # create empty root metadata
>>> sig = root.sign(GPGSigner())   # sign with default key in default keyring
>>> sig.to_dict()
{
  'keyid': '8ba69b87d43be294f23e812089a2ad3c07d962e8',
  'signature': 'afd45fea08005bb8be587ab04fd5495cfe1f9070406c61a8a63ae4a7526724c238026dd78dbc7c4a54582164f5d0972464b24bd9f2e5e1c9ec22a7bb4ee9c22ce9b68c13e26d24fcb079e1e5315d3cf5daddd52246c1d9ac61f5e68ff7058308030dd317b46de7010a2662646d2450bf1b9b3f2d08b066d0ed42f7a2bd1149737b5ea40164bf6a4c3500c10573cf6b2f0466ab2b51af3ffca138c74204daf80fe9099e1a5087a2144584df5a705402e6872752438c60a025512e0b64d3082e0447aab07565f3d7960d7d347de98562c581b732cce55fefa5eb54e43e688c2c4dd838f0aca90e49e88b944dea63c31ae324c2161b5d2c74c2199b2cacaaa331a846cc53994d1df9b3c649bba3fabf28a7bebcf973dcfa33e72fa60e307369a29cb3f26bb6e037edc532c8cf624bf686f78af7799d81fa8827e08a5d58aca90e168483eb68f35157545c48fa673ce097a2e7da1d891b6abdf445d8c0e7ab8695ef3bd8350e277d6e7b2673ee67dc72373b2df2dd8cb33ba874610d41171d36cf95fea47efc15d568ea28decd77ab988186671fcd4b3250b734436b98aac4dfc14406a58a45a7857809b99a718bb4acb76b3d3635f7848022f422733aea4eb0ed8010c134cc7b55b57873226c9d38ffeaeec326d10cae129f65f6cf7dc78dce7aaaad70deca7841f806792c61bdb4990c50235f9f76ca794e89f623986b5d70e8e0',
  'other_headers': '04000108001d1621048ba69b87d43be294f23e812089a2ad3c07d962e8050263218ca0'
}

Writing to_file still works, but deserialization will fail.

[lukpueh]

Btw. we just recently discussed the sig <-> signature inconsistency, when fixing a related bug in #418.

There I suggested to switch to 'sig' in any new signatures that are created with GPGSigner, but keep supporting Signature classes that use 'sig' or 'signature' for parsing and verification until we fully deprecate the latter.

[lukpueh]

Regarding spec compliance, at least at the time of adding this feature neither TUF nor in-toto specs explicitly allowed unrecognized fields, which would have made the signatures non-compliant due to the other_headers field anyways.

In hindsight and now that TUF spec and python-tuf supports them, it would have been smart to make sure that the sig/signature field is consistent.

[lukpueh]

There I suggested to switch to 'sig' in any new signatures that are created with GPGSigner, but keep supporting Signature classes that use 'sig' or 'signature' for parsing and verification until we fully deprecate the latter.

For python-tuf we don't even have to support parsing and verifying 'signature'-gpg signatures.

[jku]

Regarding spec compliance, at least at the time of adding this feature neither TUF nor in-toto specs explicitly allowed unrecognized fields, which would have made the signatures non-compliant due to the other_headers field anyways.

That makes sense: so I think the reasonable plan is:

• current gpg signatures and keys stay as is -- they were never designed to be TUF or intoto spec compliant
• we should document the above somewhere
• if someone wants to use GPG in TUF for example, they should create a new keytype and signature format that does not have these flaws

[jku]

• current gpg signatures and keys stay as is -- they were never designed to be TUF or intoto spec compliant

What I still dislike is GPGSigner and especially GPGSignature which claims to be a Signature but clearly does not pass for one -- using it would completely break TUF metadata for example.