Request/Response Checksum Behavior Updates #3277
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## flexible-checksums-v2 #3277 +/- ##
========================================================
Coverage ? 93.04%
========================================================
Files ? 66
Lines ? 14405
Branches ? 0
========================================================
Hits ? 13403
Misses ? 1002
Partials ? 0 ☔ View full report in Codecov by Sentry. |
nateprewitt
requested changes
Oct 11, 2024
| # `payload_signing_enabled` config overrides this logic and forces the | ||
| # header. | ||
| def test_content_sha256_not_set_if_config_value_is_true(self): | ||
| # By default, put_object() provides a trailing checksum and includes the |
There was a problem hiding this comment.
Is this right? I wouldn't expect us to be using trailing checksums by default on put_object.
There was a problem hiding this comment.
Because CRC32 is the new default, put_object will use trailing checksums by default. This is because put_object has streaming input.
You can see this behavior in the current version of the SDK by running:
import boto3
boto3.set_stream_logger('')
s3 = boto3.client("s3")
response = s3.put_object(
Body=b"Example data.",
Bucket="<bucket_name>",
Key="example_key.txt",
ChecksumAlgorithm="CRC32",
)
jonathan343
force-pushed
the
flexible-checksums-v2
branch
from
2d66b37
to
d586cba
Compare
jonathan343
force-pushed
the
request-checksum-calculation
branch
from
ccbc9c0
to
17ef6f7
Compare
jonathan343
changed the title
jonathan343
changed the title
jonathan343
changed the title
nateprewitt
reviewed
Oct 28, 2024
nateprewitt
reviewed
Oct 29, 2024
SamRemis
reviewed
Oct 29, 2024
nateprewitt
reviewed
Oct 29, 2024
Add ``request_checksum_calculation`` and ``response_checksum_validation`` config options.
* Add support for CRC64NVME when the CRT is available. * Update crc64nvme priority
…ed checksums." This reverts commit 93a47eb.
Add ``request_checksum_calculation`` and ``response_checksum_validation`` config options.
* Add support for CRC64NVME when the CRT is available. * Update crc64nvme priority
jonathan343
force-pushed
the
flexible-checksums-v2
branch
from
d586cba
to
fac0754
Compare
jonathan343
force-pushed
the
request-checksum-calculation
branch
from
009ea3d
to
626c08a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes the following updates to the botocore flexible checksum behavior:
Request Checksum Calculation
request_checksum_calculationconfig is set towhen_supported(default value), a checksum will be generated whenever one of the following conditions is true:httpchecksumtrait and defines arequestAlgorithmMember.httpchecksumtrait and definesrequestChecksumRequired: true.request_checksum_calculationconfig is set towhen_required, a checksum will be generated only when:httpchecksumtrait and definesrequestChecksumRequired: true.S3 Customizations:
Response Checksum Validation
requestValidationModeMemberby the user will be used by the SDK.requestValidationModeMembervalue is not set by the user, the following behavior is experienced:response_checksum_validationconfig is set towhen_supported(default value), therequestValidationModeMemberwill be set toENABLED.response_checksum_validationconfig is set towhen_required, therequestValidationModeMemberwill be set toENABLED.