Skip to content

Commit 9fd9255

Browse files
committed
docs(snapshot s3): document the metadata fingerprint source
Spell out what --fingerprint-source metadata does and, more importantly, what it does not do. The obvious assumption is that reading metadata instead of object content needs weaker permissions; AWS requires s3:GetObject either way, so the help says so plainly rather than letting readers infer a benefit that is not there. The three conditions that make a bucket unusable in this mode -- objects without a stored checksum, composite multipart checksums, and a root .kosli_ignore -- are documented alongside the fix for each, so a reader can tell before running whether their bucket qualifies.
1 parent 9d964ef commit 9fd9255

1 file changed

Lines changed: 17 additions & 0 deletions

File tree

cmd/kosli/snapshotS3.go

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,15 @@ const snapshotS3LongDesc = snapshotS3ShortDesc + awsAuthDesc + `
1717
You can report the entire bucket content, or filter some of the content using ^--include^ / ^--exclude^ (literal prefix match) or ^--include-regex^ / ^--exclude-regex^ (Go regular expressions matched against the full object key).
1818
In all cases, the content is reported as one artifact. If you wish to report separate files/dirs within the same bucket as separate artifacts, you need to run the command twice.
1919
20+
By default the bucket content is fingerprinted by downloading every matching object and hashing it. ^--fingerprint-source metadata^ reads the SHA256 checksum S3 stores for each object instead, which avoids the download, the temporary disk space and the local hashing. Both sources produce the same fingerprint, so a snapshot matches the artifact you attested either way.
21+
22+
Fingerprinting from metadata comes with three conditions:
23+
- Every matching object must carry a full-object SHA256 checksum. S3 only stores one when the upload asked for it, for example ^aws s3api put-object --checksum-algorithm SHA256^. Objects uploaded without one are reported, and cannot be fingerprinted this way.
24+
- A multipart upload gets a composite SHA256, which hashes the checksums of the individual parts rather than the object content, so it cannot be used as the object's fingerprint. Such an object can be collapsed into a single part in place with ^aws s3api copy-object --checksum-algorithm SHA256 --copy-source yourBucket/yourKey --bucket yourBucket --key yourKey^.
25+
- ^.kosli_ignore^ is not applied, because reading it would mean downloading it. A bucket with a ^.kosli_ignore^ at its root is reported rather than fingerprinted without its rules.
26+
27+
It does not reduce the permissions the command needs: AWS requires ^s3:GetObject^ to read an object's checksum, the same permission that downloading it needs. Reading the checksum of an SSE-KMS encrypted object additionally needs ^kms:GenerateDataKey^ and ^kms:Decrypt^.
28+
2029
` + kosliIgnoreDesc
2130

2231
const snapshotS3Example = `
@@ -67,6 +76,14 @@ kosli snapshot s3 yourEnvironmentName \
6776
--exclude-regex '.*\.png$' \
6877
--api-token yourAPIToken \
6978
--org yourOrgName
79+
80+
# report contents of an AWS S3 bucket without downloading the objects,
81+
# using the SHA256 checksums S3 stores for them:
82+
kosli snapshot s3 yourEnvironmentName \
83+
--bucket yourBucketName \
84+
--fingerprint-source metadata \
85+
--api-token yourAPIToken \
86+
--org yourOrgName
7087
`
7188

7289
// fingerprint sources accepted by --fingerprint-source

0 commit comments

Comments
 (0)