how to turn off automatic decompression when download gzipped file

[imeya]

when use the following to download gzipped file, The download works but Azure or the SDK decompresses my gzipped files on the fly when the files are fetched.
how can I turn off this behavior? thanks

BLOB_SERVICE = BlockBlobService(account_name=AZURE_BLOB_SERVICE_ACCOUNT_NAME, account_key=AZURE_BLOB_SERVICE_ACCOUNT_KEY)

BLOB_SERVICE.get_blob_to_path(
            guid,
            name,
            path,
        )

[zezha-msft]

Hi @imeya, thanks for reaching out!

I've seen a similar issue in the past, I'll investigate and let you know. Thanks!

[imeya]

@zezha-msft , any feedback?

[zezha-msft]

Hi @imeya, sorry for the delay.

Could you please confirm with me what was the Content-type and Content-encoding of your blob?

[imeya]

how can I check that in azure portal -> blob storage? I have checked the blob property from portal, but no such property you mentioned.

[zezha-msft]

Hi @imeya, you could do a get blob properties call with any SDK. You could also use the Storage Explorer either on the portal or on the desktop; do a right click on your blob and then select properties; finally scroll to the bottom to see the Content-type and Content-encoding .

[zezha-msft]

Hi @imeya, I did some digging and here's an update:

When the Content-Encoding header is set to gzip (by the user who uploaded the blob, there is no default value), the requests module would decode it automatically, which is the expected behavior.

If you have control over the data, I think the easiest way to fix this problem is to not set the Content-Encoding in the blobs' properties. If you cannot do that, then it's a bit more work to prevent the body from being decoded.

One way to do that is to replace the _http_client that the BlockBlobService uses to perform HTTP requests. Currently, it reads the body by calling response.content which decodes the body. Here is an explanation on how to read the raw data instead.

Here is a an example:

from azure.storage.common._http.httpclient import (
    _HTTPClient,
    HTTPResponse,
)
from azure.storage.common._serialization import _get_data_bytes_or_stream_only
import requests


class ExampleRawBodyReadingClient(_HTTPClient):
    def perform_request(self, request):
        '''
        Sends an HTTPRequest to Azure Storage and returns an HTTPResponse. If
        the response code indicates an error, raise an HTTPError.

        :param HTTPRequest request:
            The request to serialize and send.
        :return: An HTTPResponse containing the parsed HTTP response.
        :rtype: :class:`~azure.storage.common._http.HTTPResponse`
        '''
        # Verify the body is in bytes or either a file-like/stream object
        if request.body:
            request.body = _get_data_bytes_or_stream_only('request.body', request.body)

        # Construct the URI
        uri = self.protocol.lower() + '://' + request.host + request.path

        # Send the request
        response = self.session.request(request.method,
                                        uri,
                                        params=request.query,
                                        headers=request.headers,
                                        data=request.body or None,
                                        timeout=self.timeout,
                                        proxies=self.proxies,
                                        stream=True)

        # Parse the response
        status = int(response.status_code)
        response_headers = {}
        for key, name in response.headers.items():
            # Preserve the case of metadata
            if key.lower().startswith('x-ms-meta-'):
                response_headers[key] = name
            else:
                response_headers[key.lower()] = name

        wrap = HTTPResponse(status, response.reason, response_headers, response.raw.read())
        response.close()

        return wrap

The important part to note is that I added stream=True to the call self.session.request, and also used response.raw.read() instead of response.content when creating a HTTPResponse.

I tested it with the following code:

def run():
    bs = BlockBlobService(account_name="", account_key="")
    bs._httpclient = ExampleRawBodyReadingClient(session=requests.session(), protocol="https", timeout=2000)
    bs.get_blob_to_path(container_name="test_container", blob_name="test_blob",
                        file_path="./test_file")

I verified that the file downloaded this way has the right content, by computing its MD5 and comparing against the gzipped file. Please let me know if you have any question.