Skip to content

HTTP client response not decompressed after redirect #9309

Closed
@smortex

Description

@smortex

Describe the Bug

In puppetlabs/puppetlabs-apt#1170, @beliys report an issue with a ASCII armored GPG key that appear as a binary file after download. The issue can be reproduced using this minimal example:

file { '/tmp/with-a-redirect':
  source => 'https://packagecloud.io/timescale/timescaledb/gpgkey',
}

file { '/tmp/no-redirect':
  # this URL is not stable, get a "fresh" one with:
  # curl -sI https://packagecloud.io/timescale/timescaledb/gpgkey | awk '$1 == "location:" { print $2 }'
  source => 'https://d3fo0g5hm7lbuv.cloudfront.net/9356/8817/gpg/timescale-timescaledb-E7391C94080429FF.pub.gpg?Expires=1712081971&Signature=Zsi2Zk5uchwGUGflDbFNncWN4Nj8tBhsgg2tho5CaEcAwJ3MpkphBxz9kSBAAr4zlyp45j8zPV8Cehc9HCWY0FwmrPiONjmibmCWXuF9DyluaH-qd2mL9WTQ3WOLMVoAEuEM8-u~JfBeANoxevR-hSGf7rMjgVH1HdseYkaRe8Tr729YE5GOkIkA~Xa93l~qLCuESTVw2-7P2BcWqd2c-4PAgdBBuqLzJfA9NUTOnLJqLGa76vn7rnGmhL2z9SSZSbrW7hOzRonMSbV3wgMfXflc98dXjq-qrRR4z7QWIAf6bPQf3OBcenGIXyk5Xo~2HsrhlGPP62gwANnvynNkWg__&Key-Pair-Id=K1ZMSF1EKC3AZL'
}

The URL in the first file source redirects to the URL in the second file source.

Both downloaded files should have the same content in the end, but it happens that the first one seems to save the gzip compressed response from the final request, while the second one save the actual decompressed file (as expected).

romain@zappy ~ % ls -l /tmp/with-a-redirect /tmp/no-redirect
-rw-r--r--  1 romain wheel 3949  2 avr.  08:34 /tmp/no-redirect
-rw-r--r--  1 romain wheel 3012  2 avr.  08:33 /tmp/with-a-redirect
romain@zappy ~ % file /tmp/with-a-redirect /tmp/no-redirect 
/tmp/with-a-redirect: gzip compressed data, from Unix, original size modulo 2^32 3949
/tmp/no-redirect:     PGP public key block Public-Key (old)
romain@zappy ~ % md5 /tmp/with-a-redirect /tmp/no-redirect
MD5 (/tmp/with-a-redirect) = 951e9f5f3c5b204750574dc8d93fa709
MD5 (/tmp/no-redirect) = fdd5cb16b7250bb41f2976215afb5ce0
romain@zappy ~ % gunzip < /tmp/with-a-redirect > /tmp/with-a-redirect-decompressed
romain@zappy ~ % md5 /tmp/with-a-redirect /tmp/no-redirect /tmp/with-a-redirect-decompressed
MD5 (/tmp/with-a-redirect) = 951e9f5f3c5b204750574dc8d93fa709
MD5 (/tmp/no-redirect) = fdd5cb16b7250bb41f2976215afb5ce0
MD5 (/tmp/with-a-redirect-decompressed) = fdd5cb16b7250bb41f2976215afb5ce0

Expected Behavior

The saved file should be the same, regardless of the presence of HTTP redirects.

Steps to Reproduce

See above example.

Environment

  • Version 8.5.1
  • Platform n/a

Additional Context

n/a

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingduplicateThe issue or pull request already exists

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions