Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3.get_object(ChecksumMode='ENABLED') causes an exception #1125

Closed
4 of 6 tasks
roy-ht opened this issue Jul 16, 2024 · 4 comments · Fixed by #1126
Closed
4 of 6 tasks

s3.get_object(ChecksumMode='ENABLED') causes an exception #1125

roy-ht opened this issue Jul 16, 2024 · 4 comments · Fixed by #1126
Labels
bug Something isn't working

Comments

@roy-ht
Copy link

roy-ht commented Jul 16, 2024

Describe the bug

Sample code:

import asyncio

import aiobotocore.session


async def main():
    session = aiobotocore.session.get_session()
    async with session.create_client("s3") as client:
        content = b"abc"
        bucket = "<...>"
        key = "temp/a.txt"
        await client.put_object(
            Bucket=bucket, Key=key, Body=content, ChecksumAlgorithm="sha1"
        )
        r = await client.get_object(Bucket=bucket, Key=key, ChecksumMode="ENABLED")
        await r["Body"].read()


if __name__ == "__main__":
    asyncio.run(main())

Result:

Traceback (most recent call last):
  File "/home/jovyan/temp/a/test.py", line 20, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/home/jovyan/temp/a/test.py", line 16, in main
    await r["Body"].read()
  File "/home/jovyan/temp/a/.venv/lib/python3.10/site-packages/botocore/httpchecksum.py", line 222, in read
    chunk = super().read(amt=amt)
  File "/home/jovyan/temp/a/.venv/lib/python3.10/site-packages/botocore/response.py", line 99, in read
    chunk = self._raw_stream.read(amt)
TypeError: ClientResponse.read() takes 1 positional argument but 2 were given

botocore version code (Success):

def main():
    session = botocore.session.get_session()
    client = session.create_client("s3")
    content = b"abc"
    bucket = "<...>"
    key = "temp/a.txt"
    client.put_object(Bucket=bucket, Key=key, Body=content, ChecksumAlgorithm="sha1")
    r = client.get_object(Bucket=bucket, Key=key, ChecksumMode="ENABLED")
    print(r["Body"].read())

Checklist

  • I have reproduced in environment where pip check passes without errors
  • I have provided pip freeze results
  • I have provided sample code or detailed way to reproduce
  • I have tried the same code in botocore to ensure this is an aiobotocore specific issue
  • I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue
  • I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection

pip freeze results

aiobotocore==2.13.1
aiohttp==3.9.5
aioitertools==0.11.0
aiosignal==1.3.1
async-timeout==4.0.3
attrs==23.2.0
botocore==1.34.131
frozenlist==1.4.1
idna==3.7
jmespath==1.0.1
multidict==6.0.5
python-dateutil==2.9.0.post0
six==1.16.0
urllib3==2.2.2
wrapt==1.16.0
yarl==1.9.4

Environment:

  • Python Version: 3.10.12
  • OS name and version: Ubuntu 22.04

Additional context

I guess aiohttp's ClientResponse.read() doesn't have an option "amt" while urllib3's response has it.

@thehesiod
Copy link
Collaborator

oh, this is a bigger problem than that, we're missing a whole subclass!

@thehesiod
Copy link
Collaborator

hmm, actually maybe we just missed to patch, since we already have async def handle_checksum_body (digging)

@thehesiod
Copy link
Collaborator

no, we're missing an AioStreamingChecksumBody

@thehesiod thehesiod linked a pull request Jul 18, 2024 that will close this issue
@thehesiod
Copy link
Collaborator

ty for the report!

@jakob-keller jakob-keller added the bug Something isn't working label Jul 18, 2024
chemelli74 pushed a commit to chemelli74/aiobotocore that referenced this issue Aug 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants