Create valid Gaia-X Credenials according to Gaia-X Tagus release

[anjastrunk]

GX Credential Generator generates one huge json-ld file with ALL information about SCS cluster, currently. In favor to flexibility and maintenance, each credential should be serialized in its own file and referenced by its DID in other credentials.

This issue contains two tasks:

• Implement DID creator in python similar to Gaia-X Lab DID Creator written in JavaScript. See https://github.com/SovereignCloudStack/scs-did-creator/
• Add DID of SCS cluster and provider to generated GX Credentials, - scs-did-creator#3
• Create Gaia-X compliant credential for CSP's as LegalPerson and compliant credential for OpenStack cloud as VirtualMachineServiceOffering- 70 support did in generated gaia x credentials #88
• Figure out how GX Credential can be retrieved from a given DID. --> moved to a separate issue (Link Gaia-X Credentials with each other #102 ), for now, all credentials are in one file.

[anjastrunk]

My first task will be to investigate DID, VCs and Gaia-X Compliance (defined by PRC Documnet in more detail and figure out, how these concepts relate to each other.

I will document results of my investigations below.

[anjastrunk]

Here are the Key results of my investigations

Concept	Explanation	Scope in Gaia-X
DID (Decentralized Identifier)	A digital identifier of an entity, which is self-hosted and not provided by a centralized Identity Provider. DIDs are formatted as URIs conformant with [RFC3986]. Note: DIDs do not map digital identifiers to physical ones. DIDs are resolvable to DID Documents, which associates digital identifiers to cryptographic material and a set of services relevant to interact with the entity identified by a DID. To map a digital identifier to a physical identifier, additional services/concepts are required, such as Notary Services.	Used to identify Participants, Service Offerings and Resources. One of the first tasks would be to provide a way to create DIDs for SCS clusters and their providers. Note: Gaia-X does only support DID web as did-method , currently.
VC (Verifiable Credential	A temper-evident set of claims, whose authorship can be cryptographically verified.	Used to describe properties of participants, service offerings and resources. In context of SCS, each Cloud Service (SCS cluster) and each Cloud Service Provider needs its own VC. A further task, will be to add DID of provider and SCS cluster to generated GX Credentials.
Gaia-X/GX Credential	Synonym for Verifiable Credentials using Gaia-X ontology
Gaia-X Compliance	Set of rules, which define how a Gaia-X conformant Verifiable Credential for Participant, Service Offering or Resource looks like, i.e. which claims are mandatory.

A good starting point to play around with Gaia-X Credentials is Gaia-X Wizard offered by Gaia-X Lab. There is also a good tech theater, explaining the concepts behind Gaia-X on-boarding.

[anjastrunk]

Task 1: Implement DID generator

Gaia-X Lab offers a DID generator. This generator supports did:web method specification only. There is also free python demo, how to generate DIDs. Additional python skeleton with respect to DID handling are published by Gaia-X Lab. I will have a closer look into these scripts...

The first thing, we need is a public-private key pair and a x509 certificate (chain). According to Gaia-X Trust Framework, for development purpose let's encrypt is sufficent, in production, we need at least an eIDAS certificate or EV SSL certificate .

I created a python script to generate did documents for a given DID. Files are stored in #88 and should be moved to its own repo.

[anjastrunk]

Task 2: Add cluster's and provider's DID to GX Credential Generator

Gaia-X uses DIDs to identity Participants (CSP) Service Offerings and Resources. Each CSP must have a DID and each SCS cluster offered as Service in Gaia-X needs a DID, too. These DIDs have to be placed in GX Credentials. SCS GX Credential Generator is configured by config file. I will place DIDs there.

[anjastrunk]

Task 3: Create Gaia-X compliant credential for CSP's and OpenStack clouds

Gaia-X published an ontology containing several classes and attributes to describe CSP's and service's properties. Some of them are mandatory and not discoverable from Cloud. Furthermore, some properties must be provided in a temper.proof way as Verifiable Credential issued by a Gaia-X Clearing House. This is, e.g. the case for CSP's registration number.

We have two options to handle these challenges

1. CSPs provide any attributes, which are not discoverable in generator's configuration file. Every run of credential generator calls Gaia-X Clearing House Services to issue required credentials. This option is simple, but has two major drawbacks: (1) We mirror GX Credential Schema in configuration file, which cause a huge synchronization effort. Furthermore, we loose flexibility of linked data approach and self-sovereignty, where credential holder decides with whom to share which data.
2. CSPs request credentials by themselves and store them in a wallet. Credential generator looks in wallet for information required by Gaia-X, automatically. Such a wallet can be a simple file system folder or a professional wallets, such as OCM from XCFS.

I go for option 1, first and limit data in configuration file to mandatory attributes of Legal Person (= GX Credential for CSP) and Service Offering. According to Trust Framework, these are:

• Legal Person

  • RegistrationNumber as VAT
  • legalAddress.countryCode
  • headquarterAddress.countryCode

• Service Offering

  • providedBy
  • termsAndConditions
  • servicePolicy
  • dataAccountExport

Source code can be found in #89

Challenges

Inconsistency of shapes in Gaia-X Clearing House and WG Service Characteristics GitLab

Shapes used in Gaia-X Clearing House for verification and shapes published in WG Service Characteristics GitLab are not in sync. Gaia-X Clearing House is based on version 22.10, released in October 2022! A lot of concepts, required to describe IaaS offerings are not included in version 22.10. There will be an update in release "Loire", but its release date is not jet published.
Furthermore Gaia-X ontology used to create Gaia-X Credentials still needs improvements.

I summarized my investigations in bug reports within the appropriate Gaia-X repositories:

• https://gitlab.com/gaia-x/lab/gaia-x-onboarding-prototypes/gx-signing-tool/-/issues/7
• https://gitlab.com/gaia-x/technical-committee/service-characteristics/-/merge_requests/247 - merged

Misleading/Inconsistent definition of Gaia-X Credential

Gaia-X dictates to publish all properties of CSPs and services as Gaia-X Credential. Gaia-X Credentials "... are W3C Verifiable Credentials with claims expressed in RDF [Gaia-X Architecture Framework, section Conceptual Model

The same document says in section "Operating Model", "Gaia-X Self-Descriptions ... refers to a self-signed set of claims. This is equivalent to a Verifiable Credential where the holder is the issuer. In general, the term “Gaia-X Self Description” is replaced by “Gaia-X Credential”" [Architecture Document - section Operating Model]

Wheres the first definition says a Gaia-X Credential is a Verifiable Credential, the second limits this to a Verifiable Credential, where the holder is the same entity as the issuer. IMO, both definitions are incorrect, as a Gaia-X Credential is a Verifiable Presentation, as defined by W3C.

Furthermore, W3C VC Spec does allow several viable proof mechanisms. However, Gaia-X does not mention, which proof they prefer or support, giving the impression all viable proof mechanisms are supported. But this is not he case.

I summarizes my findings in an issue waiting for clarification.

Non-conform usage of Verifiable Credentials

There is an example on how to create compliance Gaia-X Credentials in python. The example sets JSONWebSignature2020 as proof mechanism and uses the python library jwcrypto to create proof object. Gaia-X hashes credential and uses this hash as payload for calculation of signature. However, jwcrypto calculates hash for payload as input for signature, hence hash of hash is calculated. I neither know, if this is intended nor the side effects of this behavior. I opened an issue to raise this question.

Gaia-X uses a compact serialization as JWS, which is a string consisting of protected header, payload and signature. However, Gaia-X omit payload part. Instead of [header].[payload].[signature], Gaia-X uses [header]..[signature]. I claims, this is to reduce size of credential, however, I did not found any official source to do so. If this is a special feature of Gaia-X, it should be mentioned explicitly in official Gaia-X documents. Otherwise, verification of signature fails. I raised this question to Gaia-X lab.

Requesting and validating a VC

Some Verifiable Credentials (VCs) are not created and signed locally by the CSP but an external entity.
One example is the Legal Registration Number (LRN).
A request must be made to the Notary API of the Clearing House Service using a valid LRN such as a VAT ID.
The Clearing House Service will validate the LRN using a public index and then sign and return a VC.

After receiving the response from the "https://registrationnumber.notary.lab.gaia-x.eu/v1/registrationNumberVC" Notary API, the included proof.verificationMethod response field will contain a DID identifier that, when resolved, will point to the did.json of the signee (the Gaia-X Clearing House Service). This DID in turn references the validation public key certificate in the x5u field: https://registry.lab.gaia-x.eu/.well-known/x509CertificateChain.pem.

Using this certificate, the VC from the Notary API response can be cryptographically verified:

notary_url = "https://registrationnumber.notary.lab.gaia-x.eu/v1/registrationNumberVC"
lrn_vc_id = "https://bakeup.io/lrn.json"
lrn_response = requests.post(notary_url + "?vcid=" + lrn_vc_id, ...)
lrn = lrn_response.json()

# Certificate used as public key to verify the JWS
# (this URL is part of the DID referenced in the response's proof object within the x5u field of
# the verificationMethod entry)
verification_cert_url = "https://registry.lab.gaia-x.eu/.well-known/x509CertificateChain.pem"

verify_credential(json.dumps(lrn), verification_cert_url)

The verify_credential() function can be implemented as follows:

import requests
import json
from jwcrypto import jwk, jws
# utils refers to https://gitlab.com/gaia-x/lab/workshops/gaia-x-101/-/blob/master/utils.py?ref_type=heads
from utils import sha256_normalized_vc, normalize


def verify_credential(credential_json_str, cert_url):

    verifiable_credential = json.loads(credential_json_str)

    # Retrieve the registry certificate which serves as the verification
    # public key (JWK) for the JWS later
    reg_cert_response = requests.get(cert_url)
    if reg_cert_response.status_code != 200:
        raise Exception(
            f"Unable to retrieve verification certificate "
            f"from: {cert_url}"
        )
    verification_cert_pem = reg_cert_response.text.encode('UTF-8')
    verification_key = jwk.JWK.from_pem(verification_cert_pem)

    # The proof object is part of the credential response, however
    # it resembles JWS data applicable to the response without the
    # proof object. Hence, we need to strip the proof object from
    # the response.
    proof = verifiable_credential.pop("proof")

    # The remaining structure is the actual credential data that
    # JWS was created for. The signature was applied to its
    # normalized and hashed form, which we need to recreate here
    # in order to verify the signature.
    # See: https://w3c.github.io/vc-data-integrity/#how-it-works
    normalized_credential = normalize(verifiable_credential)
    hashed_credential = sha256_normalized_vc(normalized_credential)

    # Instantiate a JWS object based on the jws attribute of the
    # proof object, which contains a base64 representation of the JWS.
    received_jws_token = jws.JWS()
    received_jws_token.deserialize(proof["jws"])

    # Finally, use the verification key (Gaia X registry public cert)
    # and the hashed credential (which is the JWS' detached payload)
    # in conjunction with the JWS token to verify the credential.
    # This method will throw an exception if verification fails.
    received_jws_token.verify(
        verification_key,
        detached_payload=hashed_credential.hexdigest()
    )

Bug in linkML's pyhton generator

Gaia-X uses linkML to define Gaia-X Ontology. We use linkML's pyhton generator to create python classes for ontology. Several attributes of classes in Gaia-X ontology refers other classes. These, so called inlined objects will added to python object in special post initialization step via method self._normalize_inlined_as_dict() in method __post_init__(). For none obvious reason self._normalize_inlined_as_dict() fails with TypeError: unhashable type: 'list' while object creation. I will go deeper in this issue later on. For now and in favor of finalizing this issue, I will use a workaround. I comment self._normalize_inlined_as_dict() out.

[anjastrunk]

Task 4: Put all credentials in a VP, sign it and retrieve Gaia-X Compliance Credential

Recap: technical prerequisites

The following preparing steps must be taken.
This is essentially a brief summary the steps resulting from tasks 1 through 3:

1. All locally or externally signed Verifiable Credentials (VCs) that will be part of the Verifiable Presentation (VP).
2. The public key that corresponds to the private key used to sign any locally signed VCs.
3. A X.509 certificate chain in PEM format hosted publicly which contains the public key from (2) in the first certificate of the chain. A chain consisting of a single certificate is valid.
4. A Decentralized Identifier (DID) of the CSP hosted at <csp-domain>/.well-known/did.json.
   • The DID must contain the public key (in JWK format) from (2).
   • The DID must reference the certificate chain from (3) in one of the verificationMethod.*.publicKeyJwk.x5u fields.

Note: it is not required for the X.509 certificate from (3) used for proof verification to be issued for a Common Name (CN) matching any of the CSP's hosting domains or having any relation to the FQDN of the DID host. The only requirements for this certificate are:

• Its contained public key must match the other attributes in the corresponding verificationMethod.*.publicKeyJwk.* fields (including the public key encoded therein).
• It must be hosted on a TLS-protected HTTP host.

Compliance process using Let's Encrypt

The list of Gaia-X Trust Anchors¹ points to Mozilla's included certificates² which lists the Let's Encrypt root certificates, such as "ISRG Root X1".
Hence, the Compliance process for Gaia-X should be achievable using certificates issued by Let's Encrypt.

I created a setup for establishing a Provider Participant using Verifiable Credentials based on a Let's Encrypt certificate and key pair. I took most of the implementation from gaia-x-101.ipynb and adjusted the code where necessary.

In short:

1. set up a web server with a DNS record gaia-x-testing.cloudandheat.com
2. use Let's Encrypt (LE) to generate key pair and signed certificate
   • it's best to forcibly switch LE to use a RSA key pair because the default ECDSA seems incompatible with the script in gaia-x-101.ipynb; as an alternative adjust utils.py to pass "alg": "ES256" to the jwcrypto library instead
3. generate DID using did:web:gaia-x-testing.cloudandheat.com
4. on the web server host two files:
   • /.well-known/did.json = the generated DID
   • /.well-known/cert.pem = a copy of the signed LE certificate
5. use the process of gaia-x-101.ipynb in conjunction with the private/public LE key pair to generate the Verifiable Credentials and the Verifiable Presentation
6. submit the Verifiable Presentation to compliance

Using the process of gaia-x-101.ipynb I'm failing at the very last step:

Exception: Unable to submit to compliance
{"message":"X509 certificate chain could not be resolved against
registry trust anchors for VC https://bakeup.io/tsandcs.json.",
"error":"Conflict","statusCode":409}

About using Let's Encrypt certificates as trust anchor

Based on a reported issue in the Gaia-X registry service³, it seems the above error happens when the Compliance API talks to the Registry API's trust anchors endpoint to verify the individual VC's certificate chains.

Let's Encrypt (LE) produces three certificate files with certbot:

• /etc/letsencrypt/live/*/cert.pem = the leaf certificate for the DNS name
• /etc/letsencrypt/live/*/chain.pem = an intermediate certificate called "R3", parent of the leaf certificate
• /etc/letsencrypt/live/*/fullchain.pem = the concatenation of the above 2

Directly checking any of the three files against the Gaia-X Registry API fails:

sudo cp /etc/letsencrypt/live/gaia-x-testing.cloudandheat.com/fullchain.pem /srv/.well-known/cert.pem

curl -X POST -H "Content-Type: application/json" -d '''
  {
    "uri": "https://gaia-x-testing.cloudandheat.com/.well-known/cert.pem"
  }''' https://registry.gaia-x.eu/v1/api/trustAnchor/chain/file
{"message":"Unable to validate certificate chain","error":"The parent certificate did not issue the given child certificate; the child certificate's issuer does not match the parent's subject.","statusCode":409}

As such, the certificate chain generated by certbot seems insufficient as DID reference for the Gaia-X Registry Trust Anchor API.

Including the LE's root certificate

When accessing the webserver delivering the LE certificate using a browser such as Mozilla Firefox, a third certificate is added when downloading the certificate via the menu: the "ISRG Root X1" root certificate of Let's Encrypt.

This certificate chain will consist of the following certificates:

1. Leaf certificate for the webserver
2. Intermediate "R3" certificate
3. ISRG Root X1

The last certificate (root certificate) is missing from the chain file produced by certbot of LE.
It has to be added manually. The link to the root certificate can be taken from the fingerprint column of Mozilla's index of included root certificates:

# start building the certificate chain from the leaf certificate and R3 intermediate
sudo cat \
    /etc/letsencrypt/live/gaia-x-testing.cloudandheat.com/cert.pem \
    /etc/letsencrypt/live/gaia-x-testing.cloudandheat.com/chain.pem \
    | sudo tee /srv/.well-known/fullchain.pem

# append the ISRG Root X1 (Let's Encrypt's root certificate)
curl -s \
    https://crt.sh/?d=96BCEC06264976F37460779ACF28C5A7CFE8A3C0AAE11A8FFCEE05C0BDDF08C6 \
    | sudo tee -a /srv/.well-known/fullchain.pem

In your did.json link this certificate chain as x5u:

{
  ...
  "verificationMethod": [
    {
      ...
      "publicKeyJwk": {
        ...
        "x5u": "https://gaia-x-testing.cloudandheat.com/.well-known/fullchain.pem"

Attempting to make a request to the Registry API of Gaia-X using the full chain now results in:

curl -X POST -H "Content-Type: application/json" -d '''
  {
    "uri": "https://gaia-x-testing.cloudandheat.com/.well-known/fullchain.pem"
  }''' https://registry.gaia-x.eu/v1/api/trustAnchor/chain/file
{"message":"Unable to validate certificate chain","error":"Certificate Root not found in trusted anchors","statusCode":409}

At first that seems to contradict the official documentation on trust anchors¹ where the "ISRG Root X1" is listed as included by Mozilla⁴ within the "Extended Validation (EV) Secure Sockets Layer (SSL)" category which is marked as "temporarily valid Trust Service Providers".

However, the "v1" API of the Gaia-X Registry API only allows these root certificates in the context of EV SSL:

v1 respects the trustframework 2210 trusted-anchors section, thus allowing only EVSSL and EIDAS certificate.

Let's Encrypt itself does not offer EV SSL certificates at all: https://letsencrypt.org/docs/faq/#what-services-does-let-s-encrypt-offer

Using a development API version of the Gaia-X Registry API instead

As stated in the issue³, the "v1" API version of the Gaia-X Registry does not accept certificates acquirable by mere mortals and strictly requires EIDAS or EV SSL certificates.
Only the "development" API version accepts other certificate types.

Replacing the "registry.gaia-x.eu/v1/api/" by "registry.lab.gaia-x.eu/development/api/" as the target finally makes the request succeed (only if the full chain including the "ISRG Root X1" certificate as downloaded from the browser is present as explained above!):

curl -X POST -H "Content-Type: application/json" -d '''
  {
    "uri": "https://gaia-x-testing.cloudandheat.com/.well-known/fullchain.pem"
  }''' https://registry.lab.gaia-x.eu/development/api/trustAnchor/chain/file
{"result":true}

As such, the Registry API is now accepting the LE certificate.

Now after also redeploying the /.well-known/did.json on the webserver to reference the full certificate chain pem file (including the LE ISRG Root X1 certificate), the compliance submission can be re-attempted against the "development" (not "v1") Compliance API:

compliance_response = requests.post(
    "https://compliance.lab.gaia-x.eu/development/api/credential-offers",
    vp
)

Exception: Unable to submit to compliance:
{"statusCode":500,"message":"Internal server error"}

Any previous error is now gone but the server runs into an unspecified error.

Turns out that the "development" API environment is not stable enough but there is a "v1-staging"⁵ API path which serves as a middle ground: it does still accept Let's Encrypt certificates (unlike "v1") but does not run into internal 500 errors (unlike "development").

Conclusion

1. Set the key pair type for the Let's Encrypt certificate to RSA in order to be compatible with the JWT signing process implemented by [gaia-x-101.ipynb](https://gitlab.com/gaia-x/lab/workshops/gaia-x-101/-/blob/master/gaia-x-101.
   • the default of Let's Encrypt is ECDSA which only works if the utils.py is adjusted to pass "alg": "ES256" to the jwcrypto library call
2. When preparing the certificate chain for hosting the DID reference target, include all three LE certificate parts: ISRG Root X1, Intermediate "R3", leaf certificate
   • the fullchain.pem as produced by certbot is not sufficient as it lacks the root certificate
   • you have to manually add the ISRG Root X1 certificate to certbot's fullchain.pem
   • you can check compatibility by sending a POST request to https://registry.lab.gaia-x.eu/development/api/trustAnchor/chain/file with { "uri": "myserver.com/.well-known/cert-chain.pem" } for example
3. Host a /.well-known/did.json that contains the public key of the leaf certificate as publicKeyJwk.n as well as the reference to the full certificate chain file from (2) as publicKeyJwk.x5u
   • you may host the certificate chain file from (2) under /.well-known/ as well; make sure the x5u path in the did.json is correct
4. For submitting the Verifiable Presentation based on a Let's Encrypt key pair, use the "v1-staging" path of the Gaia-X Compliance API, i.e. https://compliance.lab.gaia-x.eu/v1-staging/api/credential-offers
   • do not use the "development" API path, as it will break with a 500 internal error⁵
   • do not use the "v1" stable API path, as it will not accept Let's Encrypt certificates³

Footnotes

1. https://gaia-x.gitlab.io/policy-rules-committee/trust-framework/trust_anchors/#list-of-defined-trust-anchors ↩ ↩²

2. https://wiki.mozilla.org/CA/Included_Certificates ↩

3. https://gitlab.com/gaia-x/lab/compliance/gx-registry/-/issues/32 ↩ ↩² ↩³

4. https://ccadb.my.salesforce-sites.com/mozilla/IncludedCACertificateReport ↩

5. https://gitlab.com/gaia-x/lab/compliance/gx-compliance/-/issues/66#note_1914512870 ↩ ↩²

[markus-hentsch]

(content of this comment containing the LRN VC verification code sample has been merged into #70 (comment))