Repository navigation
fix(streaming): return empty bytes when reading at the end of an S3 object - #8511
Conversation
|
Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need. |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #8511 +/- ##
===========================================
+ Coverage 96.88% 96.92% +0.04%
===========================================
Files 317 317
Lines 15899 15910 +11
Branches 1395 1398 +3
===========================================
+ Hits 15403 15421 +18
+ Misses 361 353 -8
- Partials 135 136 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Thanks for catching this, @chancesong10! Good find! I pushed a small follow-up after noticing that a stream closed before its first read could reopen on a later read. It now raises |
|
Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience! |
|
Awesome work, congrats on your first merged pull request and thank you for helping improve everyone's experience! |
|
Thanks @leandrodamascena, glad I could help! If it's alright, I'd like to take on #8348 (dependency injection for middleware), starting with function-based middleware as you suggested. Would you be OK with me opening a PR for it? |



Issue number: closes #8510
Summary
Changes
_S3SeekableIOopens each stream withRange: bytes=<position>-, and S3 answers416 InvalidRangewhen that position is at or past the end of the object, which includes any range on an empty object.raw_streamcatchesClientErrorwith codeInvalidRangeand uses an emptyPowertoolsStreamingBody, so reads at the end returnb""like a regular file. Any other error is still raised. This keeps oneGetObjectper stream, with no extraHeadObjectto check the size first.close()only closes_raw_streamif a stream is already open. Before, it went through theraw_streamproperty, so closing an unread object made aGetObjectjust to close it, and closing at the end of the object raised.PowertoolsStreamingBodyis now imported at runtime rather than only underTYPE_CHECKING, since it's used to build the empty body.Tests added to
tests/functional/streaming/_boto3/test_s3_seekable_io.py:test_read_at_end_of_object_returns_empty_bytestest_read_empty_object_returns_empty_bytes, coveringread()and iterationtest_raw_stream_raises_other_client_errors(NoSuchKeystill raises)test_context_manager_at_end_of_objecttest_closedno longer stubs aget_object. It now checks that closing an unread object makes no request. The three new behaviour tests and the updatedtest_closedfail ondevelop.User experience
Before:
After, these return
b"",b"", and exit cleanly, the same asio.BytesIO. Closing an unread object no longer callsGetObject.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.