v0.37.3 is unable to decrypt using sops provider encrypted via KMS, 0.37.2 works perfectly.

[nnsense]

Hi, I've spent a few hours trying to find a reason why this isn't working but, since it fails even in its simplest form, it seems this is actually an issue and not my misunderstanding, or at least it seems so..

In short, I have a KMS key deployed into an AWS account, and a simple test.yaml file:

value: "1123j123j1j31j23"

Now, I encrypt this with sops

sops -e -i --kms arn:aws:kms:us-east-1:123456789:key/xyz123456-5e88-4683-b5e9-12345668a3b test.yaml

Taking into account that sops -d test.yaml correctly decrypt the file, this happens when I use vals:

$ vals get ref+sops://test.yaml#/value
expand sops://test.yaml#/value: Error getting data key: 0 successful groups required, got 0

If I check the file's sops data, it correctly shows the KMS ARN (which is expected since sops is working).

If I add an additional age key, vals works perfectly, which points this issue to the KMS.

The KMS I created for testing has open permissions, to check if that was the issue:

{
    "Version": "2012-10-17",
    "Id": "customPolicy",
    "Statement": [
         {
            "Sid": "Allow use of the key",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        }
    ]
}

I'm using an SSO user, but I can't see how this might affect vals, also sops works perfectly which is quite confusing.

I have the feeling I'm missing something simple.. can you please point me to the right direction?

Thanks guys :)

[nnsense]

Additional tests:
I tried re-encrypting the test yaml with sops using the role (--kms arn+role), profile (--aws-profile xyz), I even created a dedicated IAM user and set it as default in .aws/credentials, so no role or profile should be involved, but the issue is still occurring (sops always works). There's no documentation on how vals is getting its creds to decrypt using sops with kms, so I'm not even sure I can use the .aws/credentials files. Interestingly, vals with the awsssm provider works great with the same setup.

[yxxhero]

see: getsops/sops#1552

[nnsense]

I'm not sure I understand, and how that can help, can you elaborate?

This is clearly a bug into the latest version of vals.

I have the latest sops version installed, so vals is defintiely not relying on that:

$ sops -v
sops 3.9.0 (latest)

and latest vals version:

$ vals version
Version: 0.37.3
Git Commit: c13086c88cdde87e139316fa463dfc4f3d56506a

If I use that it fails:

$ vals get ref+sops://test.yaml#/value
expand sops://test.yaml#/value: Error getting data key: 0 successful groups required, got 0

If I use the previous one it works:

$ ./vals version
Version: 0.37.2
Git Commit: 88bdb7def637d93466f9dba324cb9fc1dd93f0b5

$ ./vals get ref+sops://test.yaml#/value
sops: successfully retrieved key=test.yaml
19041h941084h129048

Please, don't answer that "a PR is welcome", I'm a user not a programmer, the best that I can do is to raise the issue to devs' attention and use the previous release in the meantime.

[yxxhero]

@nnsense thanks so much. we will try to find a better solution.

[zhaque44]

I'm not sure I understand, and how that can help, can you elaborate?

This is clearly a bug into the latest version of vals.

I have the latest sops version installed, so vals is defintiely not relying on that:

$ sops -v
sops 3.9.0 (latest)

and latest vals version:

$ vals version
Version: 0.37.3
Git Commit: c13086c88cdde87e139316fa463dfc4f3d56506a

If I use that it fails:

$ vals get ref+sops://test.yaml#/value
expand sops://test.yaml#/value: Error getting data key: 0 successful groups required, got 0

If I use the previous one it works:

$ ./vals version
Version: 0.37.2
Git Commit: 88bdb7def637d93466f9dba324cb9fc1dd93f0b5

$ ./vals get ref+sops://test.yaml#/value
sops: successfully retrieved key=test.yaml
19041h941084h129048

Please, don't answer that "a PR is welcome", I'm a user not a programmer, the best that I can do is to raise the issue to devs' attention and use the previous release in the meantime.

You also need to be patient, we’re open source contributors, we don’t get paid for this and this a free product for you, saying things like “please don’t say a PR is welcome is rude” also adding a thumbs down button when we ask others to add a PR is also rude

[zhaque44]

Additional tests: I tried re-encrypting the test yaml with sops using the role (--kms arn+role), profile (--aws-profile xyz), I even created a dedicated IAM user and set it as default in .aws/credentials, so no role or profile should be involved, but the issue is still occurring (sops always works). There's no documentation on how vals is getting its creds to decrypt using sops with kms, so I'm not even sure I can use the .aws/credentials files. Interestingly, vals with the awsssm provider works great with the same setup.

I can add documentation for you to better assist you, next week

[nnsense]

@zhaque44
Well, well, well. One year later, and the issue is still here. We’ve all been patient, and nothing has changed apparently. The standard “create a PR” response.. oh no, that’s not rude, right? Well, it’s not help either. It’s a brush-off. It shows no real will to engage. Like I did a year ago, I’m sharing my view again, very politely, because KMS is still not working, feel free to hate me. And sure, I’ll create a PR eventually… I’m clearly getting all the time I need to become a developer..