panic on unexpected EOF during pull

[codegod100]

thread 'tokio-runtime-worker' panicked at ostree-ext/src/container/unencapsulate.rs:217:24:
Decompressor::finish MUST be called: unexpected end of file
Fetching layers ██░░░░░░░░░░░░░░░░░░ 9/75 
 └ Fetching █████░░░░░░░░░░░░░░░ 20.57 MiB/69.38 MiB (0 B/s) ostree chunk a15970c0d9d743e9f5fbd                                                                                                                             
  zsh: IOT instruction  sudo bootc switch ghcr.io/codegod100/template:latest

[cgwalters]

Decompressor::finish MUST be called

So we got here

bootc/ostree-ext/src/container/unencapsulate.rs

Line 217 in c8e7a68

self._finish().expect("Decompressor::finish MUST be called")

unexpected end of file

If all that happened here was an unexpected EOF we should definitely not panic on that. I think we probably need to just drop this panic because on error paths exactly like this it may be hit. Ultimately the calling code should be verifying checksums and the like for integrity.

And eventually we should switch to containers/skopeo#2601

[jeckersb]

If I had to guess, there's probably an error case somewhere that we missed that should be updated to clear out the decompressor when we get EOF like that. I would be extremely wary to remove that panic, the comment above there sums up why.

[jeckersb]

Actually looking at it again, the unexpected end of file is coming from

bootc/ostree-ext/src/container/unencapsulate.rs

Line 267 in c8e7a68

let n = std::io::copy(&mut self.inner, &mut sink)?;

So I'm guessing something is wrong with the state of self.inner just based on the assumption that a std::io::sink() can't give unexpected EOF.

[jeckersb]

I think this is probably coming from the gzip stream

https://github.com/rust-lang/flate2-rs/blob/7145603d57b7795c51d8674824163d6d685c9733/src/gz/mod.rs#L243

(give or take)

Which is easily reproduced by just feeding it an empty src:

diff --git i/ostree-ext/src/container/unencapsulate.rs w/ostree-ext/src/container/unencapsulate.rs
index 81eb90c7..21f7c23d 100644
--- i/ostree-ext/src/container/unencapsulate.rs
+++ w/ostree-ext/src/container/unencapsulate.rs
@@ -362,4 +362,11 @@ mod tests {
         let d = Decompressor::new(&oci_image::MediaType::ImageLayer, empty).unwrap();
         drop(d)
     }
+
+    #[test]
+    fn test_drop_decompressor_with_incomplete_gzip_data() {
+        let empty = std::io::empty();
+        let d = Decompressor::new(&oci_image::MediaType::ImageLayerGzip, empty).unwrap();
+        drop(d)
+    }
 }

=>

---- container::unencapsulate::tests::test_drop_decompressor_with_incomplete_gzip_data stdout ----

thread 'container::unencapsulate::tests::test_drop_decompressor_with_incomplete_gzip_data' panicked at ostree-ext/src/container/unencapsulate.rs:217:24:
Decompressor::finish MUST be called: unexpected end of file
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So really the problem is that when we finish the decompressor, we want to empty out the underlying pipe and ignore anything happening with the actual decompression. But the way things are currently structured, that's not accessible. We'd need some way to into_inner() on the GzDecoder or similar.