Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

spire-server 1.6 crashes when using aws_pca as the UpstreamAuthority #4351

Closed
SQUIDwarrior opened this issue Jul 19, 2023 · 6 comments · Fixed by #4352
Closed

spire-server 1.6 crashes when using aws_pca as the UpstreamAuthority #4351

SQUIDwarrior opened this issue Jul 19, 2023 · 6 comments · Fixed by #4352
Labels
priority/urgent Issue is approved and is must be completed in the assigned milestone
Milestone
1.7.1

Comments

@SQUIDwarrior
Copy link

SQUIDwarrior commented Jul 19, 2023 •
edited
Loading

  • Version: 1.6.4
  • Platform: Linux x86
  • Subsystem: server

When attempting to upgrade from 1.5.5 to 1.6.4, our spire-server pods immediately crash with this error:

{"error":"rpc error: code = InvalidArgument desc = X509 CA minted by upstream authority is invalid: invalid upstream-signed X509 CA: invalid X509 CA: only keyCertSign and cRLSign key usage can be set","level":"error","msg":"Fatal run error","time":"2023-07-18T15:59:24Z"}

We are using the aws_pca plugin for the UpdatemAuthority.
@SQUIDwarrior
Copy link
Author

Here is the CA from spire-server bundle show on a working (1.5.5) deployment, pushed through openssl -noout -text:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            47:4c:78:18:63:1c:be:96:28:8d:89:c4:9d:e1:be:ae
        Signature Algorithm: sha512WithRSAEncryption
        Issuer: CN = service-ca.dev.cloud.aurora.tech
        Validity
            Not Before: May  9 19:30:35 2022 GMT
            Not After : May  9 20:30:35 2027 GMT
        Subject: CN = service-ca.dev.cloud.aurora.tech
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)
                Modulus:
                    00:9b:72:6e:92:76:f6:b2:f0:6c:9e:12:52:e7:28:
                    97:35:9d:12:70:66:97:d7:4b:1b:e9:0d:eb:68:60:
                    70:52:33:e2:d7:bf:5e:fa:77:dc:9b:f6:de:a4:77:
                    49:ad:dd:fb:29:d5:69:06:b8:9a:b8:9c:1a:88:c3:
                    30:7d:f5:3e:90:8a:fd:11:9e:5d:a3:60:1f:4b:90:
                    64:f6:8c:81:b5:23:30:bd:d0:5a:4f:0d:a1:f3:00:
                    d8:8c:78:9a:d8:86:1f:eb:0e:a3:b1:93:43:f9:0e:
                    ca:6a:57:4e:bd:8b:3e:28:0b:45:8e:21:0b:72:d0:
                    be:d1:2f:46:aa:64:51:e3:c3:7e:38:92:db:be:99:
                    5f:94:a9:e2:32:70:f6:bd:52:75:0c:64:f1:5b:d2:
                    04:09:f9:fb:f9:b8:26:0a:bf:a3:2d:e0:4f:01:b8:
                    35:d6:f7:fa:00:9e:1d:80:51:3b:7f:ad:3e:cc:ed:
                    8f:56:55:a3:67:80:a5:16:4f:2a:5d:9f:42:c2:bb:
                    ce:9e:46:bc:ae:61:64:15:bb:c9:c9:4f:a3:7b:b4:
                    cb:a4:73:1f:fb:35:28:9b:bb:bf:de:41:e8:d6:7d:
                    c3:8d:ad:d8:fc:3e:b8:38:8d:d5:54:7b:62:5e:52:
                    16:59:1d:12:db:ae:3f:ed:5e:db:c5:13:d8:df:9a:
                    4c:84:ee:d1:f8:01:a3:09:f3:e5:0d:a5:7a:12:0f:
                    84:ed:0e:f2:6c:53:81:3a:58:45:a3:93:ed:1a:28:
                    81:dd:d3:7b:34:52:1e:6b:23:42:b2:bd:16:55:8d:
                    21:ce:a9:66:b2:ab:6d:13:d7:8a:64:a8:5e:b8:f1:
                    16:11:17:16:16:6c:7c:82:ae:86:04:07:e4:28:2d:
                    03:f9:1b:77:e8:cf:d5:cc:6d:01:88:57:7c:5d:29:
                    b8:37:5c:39:5a:45:40:2f:4e:70:08:41:82:f0:6e:
                    f2:12:64:9b:ff:11:9f:12:06:0d:e1:a4:76:98:99:
                    6e:e6:ca:a8:23:bc:fe:78:73:ce:a2:c7:c8:1a:18:
                    67:c5:bf:c4:39:24:b7:10:e4:8c:45:1f:d6:09:4e:
                    10:dc:e1:40:c1:07:a0:30:19:b7:9b:ca:b0:ef:74:
                    e3:0a:d9:03:20:8d:aa:aa:47:92:2e:bb:b6:88:7d:
                    d6:b8:a2:97:e8:d5:c0:0d:8c:95:81:f2:28:17:ec:
                    b7:f7:df:b9:62:29:30:74:fb:a5:13:fc:90:02:e9:
                    b8:55:f9:ca:80:4e:8b:6a:ba:d8:e3:84:42:ed:02:
                    47:84:05:90:db:1a:a2:52:54:1a:ac:d6:78:c6:84:
                    85:d9:d7:7c:56:15:ae:41:4c:ae:aa:7a:b0:ae:ce:
                    58:2d:d5
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Subject Key Identifier:
                19:47:05:77:3A:87:87:A5:6E:67:8C:96:5F:D7:88:0E:4A:BF:C9:CF
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
    Signature Algorithm: sha512WithRSAEncryption
    Signature Value:
        01:ba:b8:e0:ba:17:05:55:be:b1:26:8f:17:8f:97:d0:14:b2:
        9b:33:64:0a:a9:c9:68:4f:15:00:82:8b:b0:9e:09:f9:7a:8a:
        f6:bc:93:d3:8f:79:a7:98:1d:ef:db:41:bd:59:ac:31:eb:af:
        c9:ba:58:5c:c1:0a:17:c6:00:23:6d:77:ea:c7:3c:39:25:8c:
        15:18:25:eb:d9:f8:92:7f:3f:fb:08:ce:4e:6d:2d:5b:42:30:
        2a:a7:03:19:3d:58:d0:a0:37:e1:0d:05:b0:c0:2d:93:2d:36:
        c0:25:d0:89:0f:2a:54:9c:3a:41:3f:6c:ea:d4:be:8a:c1:61:
        f9:48:bb:3c:41:53:42:32:fb:c9:d6:d6:41:1b:f6:ee:6d:a4:
        a2:38:4b:8e:39:36:cf:4a:a7:9b:e9:eb:39:d6:a5:9d:4a:7b:
        f9:62:c5:26:ca:ea:08:d3:02:bc:17:11:ab:a2:82:e2:dd:eb:
        ac:3e:cd:da:0c:18:5b:f4:db:9d:16:db:60:85:ad:62:f2:53:
        61:b8:c9:28:c0:72:9b:3f:4c:37:e2:4b:81:ff:bf:c0:4c:d6:
        ca:e8:28:d1:ce:19:2a:0c:c5:4c:de:b1:7f:28:9a:fb:24:37:
        16:78:37:11:66:7f:09:e2:59:9c:5b:03:fd:bc:90:40:0e:69:
        d3:0a:b2:02:45:46:0e:24:cc:d9:6e:7e:57:a7:fe:ea:31:f0:
        b9:62:26:ee:34:70:f9:be:cd:b6:1e:e2:0c:30:dc:ff:01:12:
        ba:2c:0d:6d:61:48:b6:5b:74:73:17:a2:fd:39:b1:58:fb:3a:
        01:fd:19:c6:6f:01:3d:31:e4:aa:a0:36:d7:2d:d3:1b:6a:b9:
        02:35:d9:be:c4:cf:cd:2f:9c:70:99:fa:eb:08:a5:e4:ee:d4:
        b6:58:a4:ff:1a:d1:23:96:65:2b:cd:c1:34:ee:fa:14:c8:6f:
        b3:c1:bd:26:f8:72:46:45:22:1e:99:f5:16:21:30:0e:bf:23:
        f2:fe:e0:75:0c:50:a8:e4:bf:dd:5c:da:96:b4:28:c4:7c:a6:
        60:1f:30:e0:79:d5:c4:7b:65:4e:f0:8b:e7:b1:e7:61:44:af:
        41:5a:27:cf:a4:41:83:ad:7c:1b:47:cc:a4:23:dc:61:07:5c:
        46:ea:ac:cf:c3:83:35:83:54:57:ba:0a:ce:f1:86:9f:5b:dd:
        c7:a1:6c:28:5d:d6:0f:1e:e4:cd:6b:02:70:23:29:84:b1:b6:
        bf:50:d3:38:9a:07:83:2b:14:e1:cf:d5:2f:a5:ec:4d:5c:d0:
        66:72:52:a3:75:2c:cc:d4:58:92:b2:ca:5d:a2:a9:c3:25:1a:
        f4:10:e5:43:08:7d:39:83

@SQUIDwarrior SQUIDwarrior changed the title spire-server 1.6 crashes when using aws_kms as the KeyManager spire-server 1.6 crashes when using aws_pca as the UpstreamAuthority Jul 20, 2023
@azdagron
Copy link
Member

Yikes. We added extra validation in 1.6.x because of the CredentialComposer work. Side effect was that we now validate upstream authority signed certs a little more strictly.

The template in use adds digitalSignature, which we now do not allow.

https://docs.aws.amazon.com/privateca/latest/userguide/UsingTemplates.html#SubordinateCACertificate_PathLen0-V1

We'll likely need to loosen the validation of the signing certificate key usage to allow for this.

@azdagron azdagron mentioned this issue Jul 20, 2023
Merged
azdagron added a commit to azdagron/spire that referenced this issue Jul 20, 2023
@azdagron
Allow digitalSignature key usage on signing certs
6ea90e7 
Turns out `aws_pca` sets digitalSignature on subordinate certificates.

The browser forum also has digitalSignature as a minimum requirement for
CAs that sign OCSP responses.

Fixes: spiffe#4351

Signed-off-by: Andrew Harding <azdagron@gmail.com>
@evan2645 evan2645 added the priority/urgent Issue is approved and is must be completed in the assigned milestone label Jul 20, 2023
@evan2645 evan2645 added this to the 1.7.1 milestone Jul 20, 2023
@SQUIDwarrior
Copy link
Author

Is this going to be backported to the 1.6.x version of Spire? I see its included in 1.7.1, but since we're stuck on 1.5 due to this bug so we'd have to jump from 1.5.5 to 1.7.1 in one upgrade, and that isn't technically supported.

@evan2645
Copy link
Member

Yes we will backport this to a new 1.6 point release since the upgrade path is broken

@PetrMc
Copy link

PetrMc commented Jul 25, 2023

any update here - 1.7.1 seems to be not available yet. To test my settings - what version I can use today, when 1.7.0 fails with aws_pca? Thank you!

@PetrMc PetrMc mentioned this issue Jul 25, 2023
Merged
evan2645 added a commit that referenced this issue Jul 26, 2023
@azdagron @evan2645
Allow digitalSignature key usage on signing certs (#4352)
3f553e3 
Turns out `aws_pca` sets digitalSignature on subordinate certificates.

The browser forum also has digitalSignature as a minimum requirement for
CAs that sign OCSP responses.

Fixes: #4351

Signed-off-by: Andrew Harding <azdagron@gmail.com>
Co-authored-by: Evan Gilman <evan@spirl.com>
@amartinezfayo
Copy link
Member

@PetrMc The scheduled release date for 1.7.1 is tomorrow, 7/27/2023. We will also be releasing 1.6.5 with this fix.

amartinezfayo pushed a commit to amartinezfayo/spire that referenced this issue Jul 27, 2023
@azdagron @evan2645 @amartinezfayo
Allow digitalSignature key usage on signing certs (spiffe#4352)
e556af5 
Turns out `aws_pca` sets digitalSignature on subordinate certificates.

The browser forum also has digitalSignature as a minimum requirement for
CAs that sign OCSP responses.

Fixes: spiffe#4351

Signed-off-by: Andrew Harding <azdagron@gmail.com>
Co-authored-by: Evan Gilman <evan@spirl.com>
amartinezfayo pushed a commit to amartinezfayo/spire that referenced this issue Jul 27, 2023
@azdagron @evan2645 @amartinezfayo
Allow digitalSignature key usage on signing certs (spiffe#4352)
242ba41 
Turns out `aws_pca` sets digitalSignature on subordinate certificates.

The browser forum also has digitalSignature as a minimum requirement for
CAs that sign OCSP responses.

Fixes: spiffe#4351

Signed-off-by: Andrew Harding <azdagron@gmail.com>
Co-authored-by: Evan Gilman <evan@spirl.com>
amartinezfayo added a commit that referenced this issue Jul 27, 2023
@amartinezfayo
Allow digitalSignature key usage on signing certs (#4352) (#4366)
8968b98 
Turns out `aws_pca` sets digitalSignature on subordinate certificates.

The browser forum also has digitalSignature as a minimum requirement for
CAs that sign OCSP responses.

Fixes: #4351

Signed-off-by: Andrew Harding <azdagron@gmail.com>
amartinezfayo added a commit that referenced this issue Jul 27, 2023
@amartinezfayo
Allow digitalSignature key usage on signing certs (#4352) (#4365)
57e397d 
Turns out `aws_pca` sets digitalSignature on subordinate certificates.

The browser forum also has digitalSignature as a minimum requirement for
CAs that sign OCSP responses.

Fixes: #4351

Signed-off-by: Andrew Harding <azdagron@gmail.com>
kfox1111 added a commit to spiffe/helm-charts that referenced this issue Jul 27, 2023
@marcofranssen @kfox1111
Add aws_pca to the spire-server (#404)
9f4d4ac 
This change allows aws_pca to be configured via values of this chart.

__Requires 1.7.1 version__ per
[bug](spiffe/spire#4351) - this will not work
until 1.7.1 is released.

---------

Signed-off-by: Petr McAllister <petr.mcallister@gmail.com>
Signed-off-by: Petr McAllister <petr@tetrate.io>
Signed-off-by: Kevin Fox <Kevin.Fox@pnnl.gov>
Co-authored-by: Marco Franssen <marco.franssen@gmail.com>
Co-authored-by: kfox1111 <Kevin.Fox@pnnl.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
priority/urgent Issue is approved and is must be completed in the assigned milestone
Projects
None yet
Milestone
1.7.1
Development

Successfully merging a pull request may close this issue.

Allow digitalSignature key usage on signing certs azdagron/spire
5 participants
@SQUIDwarrior @evan2645 @azdagron @PetrMc @amartinezfayo