security: encryption at rest #19783
Description
This issue tracks the status of encryption-at-rest. Comments and reviews should be done on the individual PRs or in separate issues.
Approximate breakdown of work to be done (not complete, order may change):
- finalize RFC RFC: encryption at rest #19785
- add crypto++ dependency Add crypto++ 5.6.5 #19827
- upgrade rocksdb to 5.9 rocksdb: upgrade to 5.9.0 #20070
- add CCL-only encryption flags storage: enable switching env through encryption flags #20225
- add support for switching env (plaintext only) storage: enable switching env through encryption flags #20225
- pass CCL encryption options through to libroachccl storage: pass encryption options to libroachccl. #20570
- add store key manager libroach: add key manager #20670
- add data key manager libroach: add data key manager #21150
- add file registry libroach: encrypt data at rest #21580
- add encrypted env libroach: encrypt data at rest #21580
- perform actual encryption libroach: encrypt data at rest #21580
- use encryption for all local disk usage (non-logs) encryption: add support for encryption to
writeFileSyncing. #25281 encryption: encrypt data written to disk by Distsql's temp engine. #25591 encryption: letdiskSideloadStorageread from RocksDB env. #25806 storage: use engine for os-level operations. #27075 - calculate/report encryption status (debug/status page/monitoring) libroach: add basic encryption stats. #25850 UI: add stores report page with CCL switching #26040
- calculate/report encryption progress (number and size of files) libroach: add basic file encryption stats. #26802 libroach: more accurate encryption file statistics. #27388 ui: encryption stats on stores report #26890
- live key rotation (not just at startup) libroach: rotate data encryption keys while running. #28148
- tooling support (debug commands) cli: support encrypted stores in debug commands. #26477, ccl: debug encryption-active-key command to show active store key ID. #35234
- metrics enterprise: Report per-store encryption type in metrics and telemetry. #35506
- benchmarks
- draft docs Draft: encryption at rest docs#3193
- QA: qa: Encryption at rest #21880
Tracking of misc improvements:
- test non-rocksdb file paths in file registry
- respect read-only flag (don't write file/key registries) libroach: Obey read-only option in FileRegistry and DataKeyManager. #26468
- cli command to generate keys encryption: add cli command
cockroach gen encryption-key#26167 - better logging of encryption status libroach: Log encryption actions #27880
- prevent key leakage on swap and core dump libroach: disable core file when encryption is requested. #27426
- ensure runtime AES-NI detection is available libroach: make CryptoPP build with runtime AES-NI detection. #26649
- enterprise gating for report pages
- debug tools for better visibility debug: add encryption-status command. #28582
- ensure encryption settings are specified if previously used libroach: ensure encryption flags are passed if previously used. #28710