Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for zstd compression #193

Open
probonopd opened this issue Sep 21, 2024 · 2 comments
Open

Add support for zstd compression #193

probonopd opened this issue Sep 21, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@probonopd
Copy link
Member

probonopd commented Sep 21, 2024

We are trying to standardize on zstd (no pun intended). So if libappiage doesn't support it yet, it would be highly appreciated if someone could add it.

@philmb3487 over at AppImage/appimagetool#68 (comment):

Some people are generating AppImages with zstd compression, and that is not supported out of the box by libappimage.
Now, that is fine for executing AppImages, but file thumbnailers such as KDE's Dolphin use libappimage.so directly to peek at the .DirIcon information, which fails for zstd images.

Would it be possible to add this support?

Arch Linux seems to enable fuse to get that support, squashfuse then needs to be compiled with zstd support turned on. That's a lot of switches that might or might not be enabled, depending on the distribution, and I think thumbnails should always be available in any case.

See my findings here,
ivan-hc/Steam-appimage#27 (comment)

So, I propose either,

  1. better zstd support out of the box,
  2. document what needs to be done if Thumbnails cannot be shown on some appimages (out of the box it would be fine with xz, zip images,
Squashfs image uses (null) compression, this version supports only xz, zlib.
ERROR: appimage_read_file_into_buffer_following_symlinks : sqfs_open_image error: /home/phil/Applications/Steam-1.0.0.81-2-3-x86_64.appimage
@probonopd
Copy link
Member Author

There is a pull request that did not get merged so far, maybe it can be reactivated:

@probonopd
Copy link
Member Author

probonopd commented Sep 22, 2024

Currently we are using vasi/squashfuse@1f98030 from 2016. We should probably use a more recent version.

The changes would need to be made around here:

ExternalProject_Add(
squashfuse-EXTERNAL
GIT_REPOSITORY https://github.com/vasi/squashfuse/
GIT_TAG 1f98030
UPDATE_COMMAND "" # make sure CMake won't try to fetch updates unnecessarily and hence rebuild the dependency every time
PATCH_COMMAND bash -xe ${CMAKE_CURRENT_BINARY_DIR}/patch-squashfuse.sh
CONFIGURE_COMMAND ${LIBTOOLIZE} --force
COMMAND env ACLOCAL_FLAGS="-I /usr/share/aclocal" aclocal
COMMAND ${AUTOHEADER}
COMMAND ${AUTOMAKE} --force-missing --add-missing
COMMAND ${AUTORECONF} -fi || true
COMMAND ${SED} -i "/PKG_CHECK_MODULES.*/,/,:./d" configure # https://github.com/vasi/squashfuse/issues/12
COMMAND ${SED} -i "s/typedef off_t sqfs_off_t/typedef int64_t sqfs_off_t/g" common.h # off_t's size might differ, see https://stackoverflow.com/a/9073762
COMMAND CC=${CC} CXX=${CXX} CFLAGS=${CFLAGS} LDFLAGS=${LDFLAGS} <SOURCE_DIR>/configure --disable-demo --disable-high-level --without-lzo --without-lz4 --prefix=<INSTALL_DIR> --libdir=<INSTALL_DIR>/lib --with-xz=${xz_PREFIX} ${EXTRA_CONFIGURE_FLAGS}
COMMAND ${SED} -i "s|XZ_LIBS = -llzma |XZ_LIBS = -Bstatic ${xz_LIBRARIES}/|g" Makefile
BUILD_COMMAND ${MAKE}
BUILD_IN_SOURCE ON
INSTALL_COMMAND ${MAKE} install
UPDATE_DISCONNECTED On
)

and

liblzma-dev

It seems that @TheAssassin had invested a lot of time to think about internal vs. external building (rather than my typical style of coding "linearly" to make it work in exactly one "known good" configuration), so I'll leave it up to him to add zstd support in order not to inadvertently break this flexibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant