UEFIExtract fails to recursively extract files with very deep internal structure (file path for extracted item gets longer than MAX_PATH)

[platomav]

I encountered some slightly odd UEFI capsules which do include valid UEFI volumes but within areas which are shown as Non-UEFI:

UEFITool is able to detect/parse valid UEFI volumes within Non-UEFI data. However, UEFIExtract is having trouble extracting them. The old-style "unpack" command does work, but not "dump" and, by extension, "all":

I am attaching 14 sample capsule images, which should help to narrow down this UEFIExtract behavior.

Non_UEFI.zip

[platomav]

Ha, interesting, did not notice the vintage factor. And here I thought one of the oldest public implementation was Apple's iMac from early 2006 (attached for reference). Thanks for looking into it whenever time permits, Nikolaj!

IM41_88Z_0039_B00_2006-01-06_12-06_B483979D.zip

[NikolajSchlej]

Turns out the main reason for that "Error 35" issue is the dreaded MAX_PATH limitation for Windows.

[NikolajSchlej]

Should be partially fixed by 13f38aa
However, for the fix to start working, you need Windows 10 1067+ or Windows 11, and add the following to your registry:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem]
"LongPathsEnabled"=dword:00000001

More information on the Windows-side of the fix here: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

[NikolajSchlej]

[platomav]

I can confirm that it works now. Both Windows Explorer and Python work as expected. Long paths were enabled before on my OS, but it needed support from the program-side. Thanks Nikolaj!

These long paths work in "funny" ways. Once they get triggered, Windows Explorer context menu breaks a bit, and you cannot right-click to open files or even launch the terminal (from within those paths). You click the options, but nothing happens. However, you can modify the path at the top and remove the long path UNC prefix (i.e. "\\?\"), which causes it to auto-convert the path to the old-style DOS short names (with the tilde, ~) and then all normal context menu actions start working again. So weird. This is obviously not an UEFITool behavior but Windows stupidity. No one really explains these side effects anywhere, but I did find one person who saw the same weirdness here.

Anyway, something else to note: On Windows 10+, the MAX_PATH can indeed be increased but the limit for MAX_NAME (or whatever it is called) is still ~250 characters. So even if the whole path can now be extended, the final path component (i.e. file/link/folder name) cannot exceed that (small, by today's standards) hard-coded value.

[NikolajSchlej]

Those UNC (not actually, more like "transparent" as in "sent to device driver as is without processing") paths are the worst. I've tried to fix this bug without requiring registry fixups and Windows 10+, but bailed really hard because reimplementing all the path normalization just for Windows is too much, especially given that we do already have a special extraction mode that lays files in a flat structure anyway, so that problem is a bit less glaring.
Still, that limitation was and remains stupid, and I'm glad I haven't yet stumble upon anything like this in Linux or macOS.

[platomav]

especially given that we do already have a special extraction mode that lays files in a flat structure anyway, so that problem is a bit less glaring

You mentioned that the duplicates' problem was also fixed, but I forgot to check. Can indeed confirm that both "unpack" and "dump" produce the same files with a different structure. I see that an index is added to duplicates (001, 002 etc). Technically, though, the "all" mode does store some additional data (the raw body/header), which is not available in the flat (i.e. "unpack") mode. Not a big deal, but just something I noticed.

I'm glad I haven't yet stumble upon anything like this in Linux or macOS.

I actually managed to hit that limit on Linux (Ubuntu) a few weeks back (not with UEFITool). Turns out that technically there is no FS limit (https://en.wikipedia.org/wiki/Comparison_of_file_systems#Limits), but most Linux distros do impose the MAX_PATH constant, which happens to allow 4096 characters. Supposedly, it can be increased with a command but in my case it would still not allow paths longer than 4096 characters so who knows, haha. Still, much better than the crapy/old Windows default or semi-working extended behavior. And if I manage to hit the limit with UEFITool, I'll let you know, so you can curse me. 😁