File tree Expand file tree Collapse file tree 3 files changed +11
-0
lines changed Expand file tree Collapse file tree 3 files changed +11
-0
lines changed Original file line number Diff line number Diff line change 66ack'd
77acks
88ad-hoc
9+ AET
910API
1011async
1112AWS
Original file line number Diff line number Diff line change @@ -22,6 +22,7 @@ p2 --> d5(Republisher)
2222m1 --> d5
2323d5 --> p5 ( Per DocType Topics)
2424d5 --> p6 ( Monitoring Sample Topics)
25+ d2 -.-> m2 ( Cloud KMS)
2526
2627subgraph Colors
2728 d ( Dataflow jobs are green)
Original file line number Diff line number Diff line change @@ -73,6 +73,15 @@ This document specifies the architecture for GCP Ingestion as a whole.
7373 ` Republisher ` and it being checked in ` Decoder `
7474- Must send messages rejected by transforms to a configurable error destination
7575 - Must allow error destination in BigQuery
76+ - Must provide decryption support that can be enabled for specific use cases
77+ - The Pioneer project and Account Ecosystem Telemetry (AET) each have specific
78+ requirements for decrypting values within the pipeline
79+ - Private keys must be provided to the Decoder in encrypted form, to be decrypted
80+ via Cloud KMS calls at startup and held only in memory
81+ - Must remove or redact all AET ` ecosystem_anon_id ` values from the payload before
82+ passing to any durable output, including errors
83+ - Must have access restricted to a limited set of operators to avoid exposing private keys
84+ - Encrypted fields must be JOSE JWE objects in Compact Serialization form
7685
7786### Republisher
7887
You can’t perform that action at this time.
0 commit comments