Security concern: Pub/Sub to Splunk HEC requires a token as a parameter in plain text #122
Description
What's the issue?
The pub/sub to splunk beta dataflow recently announced in this blog post (docs here) has a bit of a security gap in that it requires a plain text token as input. Code for getting that token is here. I know the team considers security a top priority so I'm hoping there's a known stopgap solution. Without some way to hide this param, anyone with viewer access across the service, project, or org would likely have visibility to this token - not ideal.
Alternative solutions
Some potential workarounds and expected behavior:
- Dataflow job integration directly with KMS or secrets manager - much in the way cloudbuild can decrypt a secret as an initial step defined in the build template, it would be slick if dataflow had a similar capability, pointing to pre-existing secrets that are automatically injected into the run context at job start. e.g. the SA running a dataflow job has a decrypter role to a KMS key.
sensitiveParameters.foospecifies 2 attributes - a KMS key ID and the corresponding ciphertext - upon job launch, decryption happens automatically, with the raw contents of the secret available as environment variablefoo. - Instead of the
tokenparameter, the template could instead require a secret manager secret version ID and attempt to fetch that secret as an initial step in the job. This post takes a similar approach using KMS. This is slightly different than Rename repository #1 in that it isn't a feature request to dataflow the product, just a change to the relevant template.
Though I haven't used these myself, I've asked around and heard some transforms provide KMS integration e.g. bigqueryio support a withKmsKey. I wonder if there's a pattern that can be reused from that. +@melbrodrigues for input.