Proposal: First class support for secrets

[eljog]

What are secrets

Secrets are variables where it's values are sensitive and needs to be handled securely at all times (API keys, passwords etc.). Users can change a secret's value at anytime, and the CLI should support dynamically changing secrets without having to rebuild the container.

Goal

Support Secrets as a first class feature in devcontainers CLI.

This feature should comprise of the ability to securely pass in the secrets in CLI commands, make them available for users to consume similar to remoteEnv and containerEnv, and be processed and handled securely at all times.
This functionality will allow consumers including Codespaces to be able to use secrets more predictably and securely with devcontainers CLI.

Motivation

Today Codespaces pass secret variables as remoteEnv since there are no other mechanisms to pass them explicitly as secrets. The devcontainers CLI do not (and need not) treat remoteEnv as secrets, which recently has resulted in high severity security incidents. Having explicit support for secrets will help avoiding such situations and in improving customer confidence.

Proposal

Following are the three main functional changes to the CLI to support this feature.

1. Ability to pass secrets to commands
2. Apply/use secrets similar to remoteEnv
3. Securely handle secrets

Passing secrets to commands

Secrets are not part of devcontainers specification and we do not expect users to store secrets inside devcontainer.json file.

Secrets should be passed to the following devcontainers CLI commands.

• up
• run-user-commands
• read-configuration

Best way to pass secrets would be through a json/env file, where in the file path is supplied in a command line argument --secrets-file

up, run-user-commands will use the secrets for usecases like lifecycle hooks (similar to remoteEnv). read-configuration will need to apply secrets during variable expansion.

Using secrets in the CLI

Today Codespaces as a consumer is leveraging remoteEnv to pass secrets into the CLI. So it is a basic requirement that secrets should be used and behave the same way as remoteEnv do.

Securely handle secrets

Secrets should be handled securely at all times.

• Secrets should not be passed as plain text in command line
• Do not be persist secrets on the container labels or long lived permanent caches
• Redact secrets to prevent them being written in plain text into logs and console output.

[samruddhikhandale]

Thanks for writing this up! ✨

Should secrets be able to be passed to the devcontainer build command?

[eljog]

Should secrets be able to be passed to the devcontainer build command?

Primarily we need secrets to be made available to lifecycle hooks. build command or features do not depend/consume secret variables today afaik.

@joshspicer any thoughts?

[chrmarti]

Allowing secrets to be passed with build would allow for building base images involving access to private registries / packages. The set-up and exec commands would also benefit from support for secrets. Not sure about read-configuration but that might be the only command where the actual secret values are not of use.

(Not to redirect the discussion, but as a side-note: The spec should focus only on the abstract side of dev containers without mentioning implementation specifics. Maybe we can just add a note to the declarative secrets saying that implementations should handle secret values separately and that they should not be persisted in the image or the container.)

[GMkonan]

The devcontainers CLI do not (and need not) treat remoteEnv as secrets, which recently has resulted in high severity security incidents.

Sorry if I'm missing the mark here, but can you give me an example of does security incidents?

[eljog]

Not sure about read-configuration but that might be the only command where the actual secret values are not of use.

I have updated the proposal to include read-configuration command, since the variable expansion may use secrets?
/cc @dmgardiner25

Allowing secrets to be passed with build would allow for building base images involving access to private registries / packages. The set-up and exec commands would also benefit from support for secrets.

Sounds good. The immediate focus areas do not depend on secret support for these commands. But I can add them to the proposal, for consideration in a phase2

[chrmarti]

I have updated the proposal to include read-configuration command, since the variable expansion may use secrets? /cc @dmgardiner25

It might do so in the future or is that somehow part of the proposal? Which of the existing variable expansions would expand secrets?

[eljog]

We discussed this offline with @dmgardiner25.
He explained that remoteEnv variables are expanded using environment variables. We already pass/inject the secrets as environment variables when starting the cli process, and that is how it is used for variable expansion. Since that is not changing, I do not see an immediate need to pass the secrets in --secrets-file argument to read-configuration command.
I will remove it from proposal

[eljog]

Proposal PR was checked in