[Bug]: private_key variable is not in PEM format

[NLPatras]

Terraform CLI Version

1.8.5

Terraform Provider Version

0.92.0

Terraform Configuration

Changing the authentication of Snowflake in Terraform Cloud UI from user and password to user and key-pair.

When the input variables in the terraform config files are as in the example provided below, then everything is working fine. However, this is not a good practice to have the variables hard coded, especially for production tier.
 
provider "snowflake" {
    account = "account_id"
    user = "technical_user"
    authenticator = "JWT"
    private_key = "-----BEGIN ENCRYPTED PRIVATE KEY-----\.....\n-----END ENCRYPTED PRIVATE KEY-----\n"
    private_key_passphrase = "password"
}

The alternative would be to use these same variables in Terraform Cloud UI environment variables, as in the example provided below.
 
Terraform Cloud UI env variables :
TF_VAR_snowflake_account
TF_VAR_snowflake_user
TF_VAR_snowflake_authenticator
TF_VAR_snowflake_private_key - sensitive
TF_VAR_snowflake_private_key_passphrase - sensitive
 
Whereas in terraform config file to just have the following :
 
provider "snowflake" {
  account =  var.snowflake_account
  user  =  var.snowflake_user
  authenticator  =  var.snowflake_authenticator
  private_key  =  var.snowflake_private_key
  private_key_passphrase  = var.snowflake_private_key_passphrase
}

The issue with this method is that we are getting the error “Error: could not retrieve private key: could not parse private key, key is not in PEM format”. The private_key variable used is the same in both cases yet when hardcoded within the config file it works, on the other hand when provided within the terraform cloud variables it fails.

Category

category:provider_config

Object type(s)

resource:account_parameter

Expected Behavior

Successful run of terraform apply/plan

Actual Behavior

Error: could not retrieve private key: could not parse private key, key is not in PEM format

Steps to Reproduce

Terraform Cloud UI env variables :
TF_VAR_snowflake_account
TF_VAR_snowflake_user
TF_VAR_snowflake_authenticator
TF_VAR_snowflake_private_key - sensitive
TF_VAR_snowflake_private_key_passphrase - sensitive

provider "snowflake" {
account = var.snowflake_account
user = var.snowflake_user
authenticator = var.snowflake_authenticator
private_key = var.snowflake_private_key
private_key_passphrase = var.snowflake_private_key_passphrase
}
3. terraform plan

How much impact is this issue causing?

Medium

Logs

No response

Additional Information

No response

Would you like to implement a fix?

• Yeah, I'll take it 😎

[sfc-gh-jmichalak]

Hi @NLPatras 👋 Could you first confirm that you follow the docs? To read the key from file, you can use file function like this: private_key = file("<filepath>"). Also, you can read troubleshooting if you encounter any errors returned from snowflake.

Related issues: #2646, #1515, #2432.

[NLPatras]

Hi @NLPatras 👋 Could you first confirm that you follow the docs? To read the key from file, you can use file function like this: private_key = file("<filepath>"). Also, you can read troubleshooting if you encounter any errors returned from snowflake.

Related issues: #2646, #1515, #2432.

I am not using the pem file since I can not input the file within the Terraform Cloud UI variables, instead i am using the value of the pem file. When using the value hardcoded as shown below it works.

provider "snowflake" {
account = "account_id"
user = "technical_user"
authenticator = "JWT"
private_key = "-----BEGIN ENCRYPTED PRIVATE KEY-----.....\n-----END ENCRYPTED PRIVATE KEY-----\n"
private_key_passphrase = "password"
}

However, when using the same exact value in Terraform Cloud UI variable TF_VAR_snowflake_private_key, it fails with the error "Error: could not retrieve private key: could not parse private key, key is not in PEM format".

[sfc-gh-asawicki]

@NLPatras did you check the related issues in detail? The last one describes almost the same case with the solution (#2432 (comment)).

[NLPatras]

@NLPatras did you check the related issues in detail? The last one describes almost the same case with the solution (#2432 (comment)).

Yes, I did, the pem key that I have used hard coded and it worked was already of the below format having a \n for every new line.
"-----BEGIN ENCRYPTED PRIVATE KEY-----\n.....\n-----END ENCRYPTED PRIVATE KEY-----\n"

[sfc-gh-asawicki]

Did you try the solution from #2432 (comment) ?

It does not seem like a problem with the provider but with how it is passed to it (as you said, the key directly in the provider config works).

[NLPatras]

Did you try the solution from #2432 (comment) ?

It does not seem like a problem with the provider but with how it is passed to it (as you said, the key directly in the provider config works).

The solution provided above, seems to not be suitable for my case. My private key is already in the format that he was trying to achieved with the following commands:
` locals {
processed-secret-string = jsondecode(data.aws_secretsmanager_secret_version.snowflake-secret-version.secret_string)

pre-processed-private-key-body = regex("-----BEGIN PRIVATE KEY----- (.*)-----END PRIVATE KEY-----", local.processed-secret-string["PrivateKey"])[0]

processed-private-key-body = replace(local.pre-processed-private-key-body, " ", "\n")

processed-private-key = join("\n", ["-----BEGIN PRIVATE KEY-----", local.processed-private-key-body, "-----END PRIVATE KEY-----"])
}`

[sfc-gh-asawicki]

What I can recommend to check is to set SNOWFLAKE_PRIVATE_KEY env instead of TF_VAR_snowflake_private_key (the same with the SNOWFLAKE_PRIVATE_KEY_PASSPHRASE) and leave them unset in the provider config:

provider "snowflake" {
account = "account_id"
user = "technical_user"
authenticator = "JWT"
}

(they will be taken from the above envs). It should not change anything but it's worth try. Otherwise, I would still claim that the format must be incorrect in some way (because we do not process it anyhow, and as you claim, setting it directly works).

[NLPatras]

What I can recommend to check is to set SNOWFLAKE_PRIVATE_KEY env instead of TF_VAR_snowflake_private_key (the same with the SNOWFLAKE_PRIVATE_KEY_PASSPHRASE) and leave them unset in the provider config:

provider "snowflake" {
account = "account_id"
user = "technical_user"
authenticator = "JWT"
}

(they will be taken from the above envs). It should not change anything but it's worth try. Otherwise, I would still claim that the format must be incorrect in some way (because we do not process it anyhow, and as you claim, setting it directly works).

Getting same exact error!

[sfc-gh-asawicki]

I currently see no other option than the key being formatted badly (assuming no hidden configs like envs are set additionally). For the current implementation, there is no difference between passing the key directly through the config and setting it as an environment variable.

If you have already tried out the solution referenced above (this one: #2899 (comment)), then there is not much I can offer more.

You can spin out a short test in Golang that checks the output of pem.Decode (the "encoding/pem" package). If the first returned value is nil, then the key is bad essentially, something like:

import (
	"encoding/pem"
	"os"
	"testing"

	"github.com/stretchr/testify/require"
)

func Test_properlyFormattedKey(t *testing.T) {

	t.Run("verify key", func(t *testing.T) {
		// you can uncomment it to experiment with the key formats
		// leaving it commented out should test against your env value 
		//t.Setenv("SNOWFLAKE_PRIVATE_KEY", "<your key>")

		value := os.Getenv("SNOWFLAKE_PRIVATE_KEY")
		require.NotEmpty(t, value)
		p, _ := pem.Decode([]byte(value))
		require.NotNil(t, p)
	})
}

[NLPatras]

I currently see no other option than the key being formatted badly (assuming no hidden configs like envs are set additionally). For the current implementation, there is no difference between passing the key directly through the config and setting it as an environment variable.

If you have already tried out the solution referenced above (this one: #2899 (comment)), then there is not much I can offer more.

You can spin out a short test in Golang that checks the output of pem.Decode (the "encoding/pem" package). If the first returned value is nil, then the key is bad essentially, something like:

import (
	"encoding/pem"
	"os"
	"testing"

	"github.com/stretchr/testify/require"
)

func Test_properlyFormattedKey(t *testing.T) {

	t.Run("verify key", func(t *testing.T) {
		// you can uncomment it to experiment with the key formats
		// leaving it commented out should test against your env value 
		//t.Setenv("SNOWFLAKE_PRIVATE_KEY", "<your key>")

		value := os.Getenv("SNOWFLAKE_PRIVATE_KEY")
		require.NotEmpty(t, value)
		p, _ := pem.Decode([]byte(value))
		require.NotNil(t, p)
	})
}

I have tried asserting the private key using python script, and it passed the test. Would it make a difference asserting it with golang or python?

[sfc-gh-asawicki]

Yes. The suggested pem.Decode is the same one that is used in the provider's implementation.

[NLPatras]

Yes. The suggested pem.Decode is the same one that is used in the provider's implementation.

The assertion of private key with the above script running in golang is a PASS. It is the same exact private key that was hard coded. To recall the private key has the below format. Just a remainder \n has a single backslash.

-----BEGIN ENCRYPTED PRIVATE KEY-----\n...\n...\n...\n-----END ENCRYPTED PRIVATE KEY-----\n

[NLPatras]

Yes. The suggested pem.Decode is the same one that is used in the provider's implementation.

The actual issue is that the Terraform Cloud UI adds to the variables an extra back slash thus if my input has "\n" or "\t" , Terraform Cloud UI returns it as double backslash "\\n" or "\\t" . Therefore the solution is as follow:

provider "snowflake" {
  account = var.snowflake_account
  user = var.snowflake_user
  authenticator = var.snowflake_authenticator
  private_key = replace(var.snowflake_private_key, "\\n", "\n")
  private_key_passphrase = replace(var.snowflake_private_key_passphrase, "\\t", "\t")

With Terraform Cloud UI env variables named in this way:

TF_VAR_snowflake_account
TF_VAR_snowflake_user
TF_VAR_snowflake_authenticator
TF_VAR_snowflake_private_key - sensitive
TF_VAR_snowflake_private_key_passphrase - sensitive

[sfc-gh-asawicki]

@NLPatras, so as I understand you were able to make it work with the above solution, then? :)

[NLPatras]

@NLPatras, so as I understand you were able to make it work with the above solution, then? :)

Yes, made it work. I had to print out the private_key and private_key_passphrase to see how Terraform Cloud UI is injecting these variables. It was observed that Terraform Cloud UI adds extra back slash to the above mentioned input values. Thus making the private_key not being PEM format. The solution was to replace "\\n" to "\n", and since my private_key_passphrase contained a "\t", also had to replace "\\t" to "\t".

[sfc-gh-asawicki]

Great to hear that!

I will add this to the documentation for future reference. I am closing the issue.