Skip to content

Conversation

@qkaiser
Copy link
Contributor

@qkaiser qkaiser commented Feb 15, 2024

MultiFile handlers would collect files within a directory corresponding to a specific schema without checking if those files are actually present.

For example, a directory could contain dangling symlinks with a name corresponding to the glob search. This would lead to FileNotFoundError being thrown by the multi-file handlers.

Resolve #771

@qkaiser qkaiser self-assigned this Feb 15, 2024
@AndrewFasano
Copy link

With this patch (testing on the same FW from #771) I get a new error:

2024-02-15 19:36.16 [warning  ] Unhandled Exception during multi file calculation handler=multi-gzip path=FW_RT_N66U_C1_300438510000.zip_extract/Firmware_Release/RT-N66U_C1_3.0.0.4_385_10000-gd8ccd3c.trx_extract/part1_extract/0-40448000.squashfs_v4_le_extract/www/fb_data.tgz.gz.part.d pid=65 severity=<Severity.ERROR: 'ERROR'>
Traceback (most recent call last):
  File "/unblob/unblob/processing.py", line 377, in _calculate_multifile
    return dir_handler.calculate_multifile(path)
  File "/unblob/unblob/handlers/compression/gzip.py", line 176, in calculate_multifile
    if file != paths[0]:
IndexError: list index out of range

@AndrewFasano
Copy link

With these changes plus the following check to fix the new error, the test FW extracts correctly.

--- a/unblob/handlers/compression/gzip.py
+++ b/unblob/handlers/compression/gzip.py
@@ -170,6 +170,9 @@ class MultiVolumeGzipHandler(DirectoryHandler):
             [p for p in file.parent.glob(f"{file.stem}.*") if p.resolve().exists()]
         )

+        if not len(paths):
+            return None
+
         # we 'discard' paths that are not the first in the ordered list,
         # otherwise we will end up with colliding reports, one for every
         # path in the list.

@qkaiser
Copy link
Contributor Author

qkaiser commented Feb 16, 2024

Good catch, did not think of it.

@qkaiser qkaiser force-pushed the 771-multi-gzip-symlinks branch from 5aa5036 to 0f10bfc Compare February 16, 2024 08:53
@qkaiser
Copy link
Contributor Author

qkaiser commented Feb 16, 2024

Added integration tests files reproducing the situation described in #771

@qkaiser qkaiser force-pushed the 771-multi-gzip-symlinks branch 2 times, most recently from ae577fb to 18ea325 Compare February 16, 2024 10:45
@qkaiser qkaiser requested a review from nyuware February 16, 2024 10:46
MultiFile handlers would collect files within a directory corresponding
to a specific schema without checking if those files are actually
present.

For example, a directory could contain dangling symlinks with a name
corresponding to the glob search. This would lead to FileNotFoundError
being thrown by the multi-file handlers.
@qkaiser qkaiser force-pushed the 771-multi-gzip-symlinks branch from 18ea325 to 25c43e3 Compare February 16, 2024 12:59
@qkaiser qkaiser enabled auto-merge February 16, 2024 12:59
@qkaiser qkaiser merged commit 70f7185 into main Feb 16, 2024
@qkaiser qkaiser deleted the 771-multi-gzip-symlinks branch February 16, 2024 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unhandled Exception during multi file calculation for multi-gzip handler

4 participants