Skip to content

Commit 8f8440f

Browse files
committed
Update docs
1 parent b0d73cb commit 8f8440f

File tree

3 files changed

+11
-0
lines changed

3 files changed

+11
-0
lines changed

.spelling

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ ack
66
ack'd
77
acks
88
ad-hoc
9+
AET
910
API
1011
async
1112
AWS

docs/architecture/diagram.mmd

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ p2 --> d5(Republisher)
2222
m1 --> d5
2323
d5 --> p5(Per DocType Topics)
2424
d5 --> p6(Monitoring Sample Topics)
25+
d2 -.-> m2(Cloud KMS)
2526

2627
subgraph Colors
2728
d(Dataflow jobs are green)

docs/architecture/overview.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,15 @@ This document specifies the architecture for GCP Ingestion as a whole.
7373
`Republisher` and it being checked in `Decoder`
7474
- Must send messages rejected by transforms to a configurable error destination
7575
- Must allow error destination in BigQuery
76+
- Must provide decryption support that can be enabled for specific use cases
77+
- The Pioneer project and Account Ecosystem Telemetry (AET) each have specific
78+
requirements for decrypting values within the pipeline
79+
- Private keys must be provided to the Decoder in encrypted form, to be decrypted
80+
via Cloud KMS calls at startup and held only in memory
81+
- Must remove or redact all AET `ecosystem_anon_id` values from the payload before
82+
passing to any durable output, including errors
83+
- Must have access restricted to a limited set of operators to avoid exposing private keys
84+
- Encrypted fields must be JOSE JWE objects in Compact Serialization form
7685

7786
### Republisher
7887

0 commit comments

Comments
 (0)