Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker archive auto compression #481

Merged
merged 8 commits into from
Jul 17, 2018
Merged

Docker archive auto compression #481

merged 8 commits into from
Jul 17, 2018

Commits on Jul 17, 2018

  1. Close the stream returned by openTarComponent on Source.GetBlob error… 

    … paths

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    3f53b44 View commit details
    Browse the repository at this point in the history

  2. Call Close() on the return value of gzip.NewReader() 

    compression.DecompressionFunc now returns a ReadCloser; update all
its users.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    7091efa View commit details
    Browse the repository at this point in the history

  3. pkg: compression: support xz (lzma) decompression 

    For quite a while we were blocked on support xz decompression because it
effectively required shelling out to "unxz" (which is just bad in
general). However, there is now a library -- github.com/ulikunitz/xz --
which has implemented LZMA decompression in pure Go. It isn't as
featureful as liblzma (and only supports 1.0.4 of the specification) but
it is an improvement over not supporting xz at all. And since we aren't
using its writer implementation, we don't have to worry about sub-par
compression.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

(Updated to return io.ReadCloser)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    86cad8d View commit details
    Browse the repository at this point in the history

  4. Simplify TestDetectCompression 

    Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    20b48f2 View commit details
    Browse the repository at this point in the history

  5. pkg: compression: simplify DetectCompression boilerplate 

    Most users of DetectCompression will use the decompressorFunc
immediately afterwards (the only exception is the copy package). Wrap
up this boilerplate in AutoDecompress, so it can be used elsewhere.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

(Updated primarily to handle closing the compressed stream.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    ff55ab6 View commit details
    Browse the repository at this point in the history

  6. Add tests for compression.AutoDecompress 

    Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    969c094 View commit details
    Browse the repository at this point in the history

  7. docker tarfile: support auto-decompression of source 

    This matches how "docker load" deals with compressed images, as well as
being a general quality-of-life improvement over the previous error
messages we'd give. This also necessarily removes the previous
special-cased gzip handling, and adds support for auto-decompression for
streams as well.

Signed-off-by: Aleksa Sarai <asarai@suse.de>

(Updated primarily to handle closing the decompressed streams.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    3181923 View commit details
    Browse the repository at this point in the history

  8. docker tarfile: report correct size of compressed layers 

    Tools like umoci will always compress layers, and in order to work
around some *lovely* issues with DiffIDs, tarfile.GetBlob would always
decompress them. However the BlobInfo returned from tarfile.GetBlob
would incorrectly give the size of the *compressed* layer because the
size caching code didn't actually check the layer size, resulting in
"skopeo copy" failing whenever sourcing umoci images.

As an aside, for some reason the oci: transport doesn't report errors
when the size is wrong...

Signed-off-by: Aleksa Sarai <asarai@suse.de>

(Updated primarily to handle closing the decompressed stream.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @cyphar @mtrmac
    cyphar authored and mtrmac committed Jul 17, 2018
    Configuration menu
    Copy the full SHA
    6353fa5 View commit details
    Browse the repository at this point in the history