Skip to content

App layer crypto codec #780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 136 commits into
base: main
Choose a base branch
from
Open

App layer crypto codec #780

wants to merge 136 commits into from

Conversation

hulto
Copy link
Collaborator

@hulto hulto commented Jun 13, 2024

What type of PR is this?

/kind feature

What this PR does / why we need it:

  • Adds a custom codec to encrypt the grpc protocol with xchacha20-poly1305
  • Derives a shared key between the server and client by using a diffie hellman key exchange
  • Adds a secrets manager (gcp for prod, yaml file on disk for dev)
  • Stores the server private key in the secrets manager
  • Creates a more locked down service account for cloud run (roles/logging.logWriter, roles/monitoring.metricWriter, roles/cloudsql.client, and roles/secretmanager.secretAccessor@REALM_tavern_encryption_private_key

TODO

  • Add secrets management interface
  • Add secrets impl as a file on disk for debug builds
  • Add GCP Secrets manager to Terraform
  • Add secrets impl in GCP HSM
  • Store crypto private key with secrets management interface
  • Prevent VScode from throwing lint warnings about env!
  • Update docs
  • Cleanup

Which issue(s) this PR fixes:

@hulto
Copy link
Collaborator Author

hulto commented Jul 12, 2024

An error (new to me) seems to occur intermittently image

Hasn't been an issue the last four commits.
Could have been an issue in the github actions env the other day.


// TODO: Find a better way
// This is terrible, slow, and should never be used.
func goid() (int, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My eyes 😭

Value string
}

type Secrets struct {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, use this:

type Secrets []Secret

@hulto hulto added this to the v0.2.0 milestone Jan 26, 2025
Copy link
Collaborator

@Cictrone Cictrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed rust code. Golang will come later.

@@ -81,14 +82,17 @@ tokio-stream = "0.1.9"
tokio-test = "*"
tokio-util = { version = "0.7.10", features = ["io"] }
tonic = { git = "https://github.com/hyperium/tonic.git", rev = "07e4ee1" }
tonic-build = "0.10"
tonic-build = { git = "https://github.com/hyperium/tonic.git", rev = "c783652" } # Needed git for `.codec_path` in build.rs - previous version of codec setting is really gross. https://github.com/hyperium/tonic/blob/ea8cd3f384e953e177f20a62aa156a75676853f4/examples/build.rs#L44
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we still need this/can we update our tonic version?

**We strongly recommend building agents inside the provided devcontainer `.devcontainer`**
Building in the dev container limits variables that might cause issues and is the most tested way to compile.

**Imix requires a server public key so it can encrypt messsages to and from the server check the server log for `[INFO] Public key: <SERVER_PUBKEY>`**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: should be base64 (DIR?) encoded; regardless we should mention the encoding and maybe include that in the name of the env variable

/* Compile-time constant for the server pubkey, derived from the IMIX_SERVER_PUBKEY environment variable during compilation.
* To find the servers pubkey check the startup messages on the server look for `[INFO] Public key: <SERVER_PUBKEY>`
*/
static SERVER_PUBKEY: [u8; 32] = const_decode::Base64.decode(env!("IMIX_SERVER_PUBKEY").as_bytes());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens if env var not set/not correct size - like empty? does it fail at compile time or does it 0 append or does it disable app-layer crypto?

.lock()
.unwrap() // Mutex's must unwrap
.get(&pub_key)
.context("Key not found")?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a bit more detail in the comments and in the error itself here around what the key is used for, why is there a key history, and how it functions.

if !buf.has_remaining_mut() {
// Can't add to the buffer.
#[cfg(debug_assertions)]
log::debug!("DANGER can't add to the buffer.");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

continue anyway? maybe add to the comment how this alters behavior

secretsManager, err = secrets.NewDebugFileSecrets(EnvSecretsManagerPath.String())
}
if err != nil {
log.Printf("[ERROR] Unable to setup secrets manager\n")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log the error, remove new line(unneeded), maybe return/fatal?

priv_key_string, err := secretsManager.GetValue("tavern_encryption_private_key")
if err != nil {
// Generate a new one if it doesn't exist
pub_key, priv_key, err := generate_key_pair()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no snake case switch all names to camel case

return nil, nil
}

priv_key_bytes, err := x509.MarshalPKCS8PrivateKey(priv_key)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no snake case switch all names to camel case

if err != nil {
log.Printf("[ERROR] Unable to parse private key %v\n", err)
}
priv_key := tmp.(*ecdh.PrivateKey)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no snake case switch all names to camel case

}
priv_key := tmp.(*ecdh.PrivateKey)

public_key, err := x22519.NewPublicKey(priv_key.PublicKey().Bytes())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: no snake case switch all names to camel case

@KCarretto KCarretto removed this from the v0.2.0 milestone Feb 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants