Repository navigation
Suitable to use for signing of models today? #359
Description
Activity
We are planning to do a v1.0 release by the end of month if all goes well. From that point on, everything would be stable, backwards compatible.
Right now, you can test this, but we might need to change some parts here and there, so you might need to run some migration work after the v1.0 release.
Thanks for the response.
I have been playing around with signing with this library and found this particularly useful: instructlab/instructlab#1181 (comment) So, there could be a file with (potential) name oidc.yaml containing the identity and issuer of the signer... and maybe more (?). The file would be generated by the signing tool and help the lazy user (who exactly knows who the expected signer is supposed to be) just verify the model files by just confirming these:
oidc: identity: user@gmail.com issuer: https://accounts.google.comWhat do you think about this type of file as part of 'standard model signing'?
I think this could be part of the deployment strategy and could work if all the identities are expected to come from the same issuer.
Adding @haydentherapper for another point of view here
I think this could be part of the deployment strategy and could work if all the identities are expected to come from the same issuer.
The format and contents of this oidc.yaml file has to correspond the verification process in some way. Do you have support for multiple identities or are you planning on supporting something like this?
As for the oidc.yaml file I have to admit that the info is stored in the x509 cert embedded in the signature file as well but not as easy to get to for everyone:
jq -r .verificationMaterial.certificate.rawBytes < model.sig | base64 -d | openssl x509 -inform der -textI am not sure whether this will justify the additional file but users may need some sort of tool for it or maybe a method
sigstore-easyto get to the signer that they would only have to confirm.So, the identity should be distributed out of band if it is to be trusted. Someone working at company Z could take a model released by company Z, finetune it to make it malicious and then sign it with their own email at company Z. You now have a tampered model and a signature signed by an identity at company Z.
If you just take the identity from the certificate or by trusting all emails from company Z then you would then validate that the hashes of the tampered model match with the ones of the model on disk (which they should).
For an example on how out of band identities are distributed, see Python. Of course, that is not the only way, it's part of the deployment strategy for this library to decide how to handle the identity verification. That's why we have it a parameter for the verification side.
Of course, if your risk model assumes that the identity doesn't matter (e.g., you only want to validate the model hash and properties), then you can extract the identity from the certificate, but this is a weaker security model.
The idea of a configuration/policy file that stores the expected identities is an interesting one though, I might suggest that we bring that issue up in https://github.com/sigstore/sigstore-conformance to make this work in a uniform way across clients, if it doesn't exist already. (This probably would also need a way to map filenames or filename patterns to identities)
Reacted by Mihai MaruseacSo, the identity should be distributed out of band if it is to be trusted. Someone working at company Z could take a model released by company Z, finetune it to make it malicious and then sign it with their own email at company Z. You now have a tampered model and a signature signed by an identity at company Z.
There needs to be a table of (ideally) over-time-constant trusted identities for each company that are signing models. Someone verifying the model needs to know who the expected signer is and it ideally only one email identity will be used per company (to keep it manageable for consumers I would think).
If you just take the identity from the certificate or by trusting all emails from company Z then you would then validate that the hashes of the tampered model match with the ones of the model on disk (which they should).
The idea here was to present the identity from the certificate to the signature-verifying user and have him confirm these parameters. The identity and issuer have to meet expectations of the "(ideally) over-time-constant trusted identities for each company that are signing models" and for this the user may then choose to not confirm the shown identity + issuer if it doesn't meet expectations.
Even if the user was to blindly accept the shown parameters it would allow him to verify that the model is the original one -- so there's this weaker benefit as well.
For an example on how out of band identities are distributed, see Python. Of course, that is not the only way, it's part of the deployment strategy for this library to decide how to handle the identity verification. That's why we have it a parameter for the verification side.
Yes, I see the table there. I would expect a similar table for models signed by companies/their authors.
Of course, if your risk model assumes that the identity doesn't matter (e.g., you only want to validate the model hash and properties), then you can extract the identity from the certificate, but this is a weaker security model.
Reacted by Mihai MaruseacThe idea of a configuration/policy file that stores the expected identities is an interesting one though, I might suggest that we bring that issue up in https://github.com/sigstore/sigstore-conformance to make this work in a uniform way across clients, if it doesn't exist already. (This probably would also need a way to map filenames or filename patterns to identities)
I just got rid of the oidc.yaml file thinking it isn't justifyable - I am picking the parameters out of the signature file now: https://github.com/stefanberger/instructlab/commits/model-signing-lib/
This function here:
def get_oidc_params_from_sigfile(sigfile: Path) -> (Optional[str], Optional[str]):Moving to v1.1 release for the part about incorporating default identities inside the library.
Question
I was wondering whether the library and the format it is producing/using are suitable for signing models today or whether there may be something still missing?