Add support for IBM Z hardware-accelerated deflate

IBM Z mainframes starting from version z15 provide DFLTCC instruction,
which implements deflate algorithm in hardware with estimated
compression and decompression performance orders of magnitude faster
than the current zlib and ratio comparable with that of level 1.

This patch adds DFLTCC support to zlib. It can be enabled using the
following build commands:

    # via configure
    $ ./configure --dfltcc
    $ make

    # via cmake
    $ cmake -DZLIB_DFLTCC=on ..
    $ make

When built like this, zlib would compress in hardware on level 1, and
in software on all other levels. Decompression will always happen in
hardware. In order to enable DFLTCC compression for levels 1-6 (i.e.,
to make it used by default) one could either configure with
--dfltcc-level-mask=0x7e or export DFLTCC_LEVEL_MASK=0x7e at run
time.

Two DFLTCC compression calls produce the same results only when they
both are made on machines of the same generation, and when the
respective buffers have the same offset relative to the start of the
page. Therefore care should be taken when using hardware compression
when reproducible results are desired. One such use case - reproducible
software builds - is handled explicitly: when the SOURCE_DATE_EPOCH
environment variable is set, the hardware compression is disabled.

DFLTCC does not support every single zlib feature, in particular:

    * inflate(Z_BLOCK) and inflate(Z_TREES)
    * inflateMark()
    * inflatePrime()
    * inflateSyncPoint()

When used, these functions will either switch to software, or, in case
this is not possible, gracefully fail.

This patch tries to add DFLTCC support in the least intrusive way.
All SystemZ-specific code is placed into separate files, but
unfortunately there is still a noticeable amount of changes in the
main zlib code. Below is the summary of these changes.

DFLTCC takes as arguments a parameter block, an input buffer, an output
buffer and a window. Since DFLTCC requires parameter block to be
doubleword-aligned, and it's reasonable to allocate it alongside
deflate and inflate states, The ZALLOC_STATE(), ZFREE_STATE() and
ZCOPY_STATE() macros are introduced in order to encapsulate the
allocation details. The same is true for window, for which
the ZALLOC_WINDOW() and TRY_FREE_WINDOW() macros are introduced.

Software and hardware window formats do not match, therefore,
deflateSetDictionary(), deflateGetDictionary(),
inflateSetDictionary() and inflateGetDictionary() need special
handling, which is triggered using the new
DEFLATE_SET_DICTIONARY_HOOK(), DEFLATE_GET_DICTIONARY_HOOK(),
INFLATE_SET_DICTIONARY_HOOK() and INFLATE_GET_DICTIONARY_HOOK()
macros.

deflateResetKeep() and inflateResetKeep() now update the DFLTCC
parameter block, which is allocated alongside zlib state, using
the new DEFLATE_RESET_KEEP_HOOK() and INFLATE_RESET_KEEP_HOOK()
macros.

The new DEFLATE_PARAMS_HOOK() macro switches between the hardware
and the software deflate implementations when the deflateParams()
arguments demand this.

The new INFLATE_PRIME_HOOK(), INFLATE_MARK_HOOK() and
INFLATE_SYNC_POINT_HOOK() macros make the respective unsupported
calls gracefully fail.

The algorithm implemented in the hardware has different compression
ratio than the one implemented in software. In order for
deflateBound() to return the correct results for the hardware
implementation, the new DEFLATE_BOUND_ADJUST_COMPLEN() and
DEFLATE_NEED_CONSERVATIVE_BOUND() macros are introduced.

Actual compression and decompression are handled by the new
DEFLATE_HOOK() and INFLATE_TYPEDO_HOOK() macros. Since inflation
with DFLTCC manages the window on its own, calling updatewindow() is
suppressed using the new INFLATE_NEED_UPDATEWINDOW() macro.

In addition to the compression, DFLTCC computes the CRC-32 and Adler-32
checksums, therefore, whenever it's used, the software checksumming is
suppressed using the new DEFLATE_NEED_CHECKSUM() and
INFLATE_NEED_CHECKSUM() macros.

DFLTCC will refuse to write an End-of-block Symbol if there is no input
data, thus in some cases it is necessary to do this manually. In order
to achieve this, send_bits(), bi_reverse(), bi_windup() and
flush_pending() are promoted from local to ZLIB_INTERNAL.
Furthermore, since the block and the stream termination must be handled
in software as well, enum block_state is moved to deflate.h.

Since the first call to dfltcc_inflate() already needs the window,
and it might be not allocated yet, inflate_ensure_window() is
factored out of updatewindow() and made ZLIB_INTERNAL.

Co-authored-by: Eduard Stefes <eduard.stefes@ibm.com>

Author: iii-i

CMakeLists.txt

@@ -206,6 +206,7 @@ if(ZLIB_BUILD_SHARED)
         AND NOT APPLE
         AND NOT (CMAKE_SYSTEM_NAME STREQUAL AIX))
     target_link_libraries(zlib PRIVATE $<TARGET_NAME_IF_EXISTS:zlib_crc32_vx>)
+    target_link_libraries(zlib PRIVATE $<TARGET_NAME_IF_EXISTS:zlib_dfltcc>)
 endif(ZLIB_BUILD_SHARED)
 
 if(ZLIB_BUILD_STATIC)
@@ -229,6 +230,7 @@ if(ZLIB_BUILD_STATIC)
         zlibstatic PROPERTIES EXPORT_NAME ZLIBSTATIC OUTPUT_NAME
                                                      z${zlib_static_suffix})
     target_link_libraries(zlibstatic PRIVATE $<TARGET_NAME_IF_EXISTS:zlib_crc32_vx>)
+    target_link_libraries(zlibstatic PRIVATE $<TARGET_NAME_IF_EXISTS:zlib_dfltcc>)
 endif(ZLIB_BUILD_STATIC)
 
 if(ZLIB_INSTALL)

Makefile.in

@@ -171,6 +171,9 @@ crc32-vx.o: $(SRCDIR)contrib/s390x/crc32-vx.c
 s390x-functable.o: $(SRCDIR)contrib/s390x/s390x-functable.c
 	$(CC) $(CFLAGS) $(VGFMAFLAG) $(ZINC) -c -o $@ $(SRCDIR)contrib/s390x/s390x-functable.c
 
+dfltcc.o: $(SRCDIR)contrib/s390x/dfltcc.c
+	$(CC) $(CFLAGS) $(ZINC) -c -o $@ $(SRCDIR)contrib/s390x/dfltcc.c
+
 deflate.o: $(SRCDIR)deflate.c
 	$(CC) $(CFLAGS) $(ZINC) -c -o $@ $(SRCDIR)deflate.c
 
@@ -231,6 +234,11 @@ s390x-functable.lo: $(SRCDIR)contrib/s390x/s390x-functable.c
 	$(CC) $(SFLAGS) $(VGFMAFLAG) $(ZINC) -DPIC -c -o objs/s390x-functable.o $(SRCDIR)contrib/s390x/s390x-functable.c
 	-@mv objs/s390x-functable.o $@
 
+dfltcc.lo: $(SRCDIR)contrib/s390x/dfltcc.c
+	-@mkdir objs 2>/dev/null || test -d objs
+	$(CC) $(SFLAGS) $(ZINC) -DPIC -c -o objs/dfltcc.o $(SRCDIR)contrib/s390x/dfltcc.c
+	-@mv objs/dfltcc.o $@
+
 deflate.lo: $(SRCDIR)deflate.c
 	-@mkdir objs 2>/dev/null || test -d objs
 	$(CC) $(SFLAGS) $(ZINC) -DPIC -c -o objs/deflate.o $(SRCDIR)deflate.c
@@ -304,6 +312,9 @@ placebo $(SHAREDLIBV): $(PIC_OBJS) libz.a
 	ln -s $@ $(SHAREDLIBM)
 	-@rmdir objs
 
+crc32_test$(EXE): crc32_test.o $(STATICLIB)
+	$(CC) $(CFLAGS) -o $@ crc32_test.o $(TEST_LIBS)
+
 example$(EXE): example.o $(STATICLIB)
 	$(CC) $(CFLAGS) $(LDFLAGS) -o $@ example.o $(TEST_LIBS)
 
@@ -316,6 +327,9 @@ examplesh$(EXE): example.o $(SHAREDLIBV)
 minigzipsh$(EXE): minigzip.o $(SHAREDLIBV)
 	$(CC) $(CFLAGS) -o $@ minigzip.o $(LDFLAGS) -L. $(SHAREDLIBV)
 
+crc32_test64$(EXE): crc32_test64.o $(STATICLIB)
+	$(CC) $(CFLAGS) -o $@ crc32_test64.o $(TEST_LIBS)
+
 example64$(EXE): example64.o $(STATICLIB)
 	$(CC) $(CFLAGS) $(LDFLAGS) -o $@ example64.o $(TEST_LIBS)
 

configure

@@ -94,6 +94,7 @@ address=0
 memory=0
 unknown=0
 enable_crcvx=1
+enable_dfltcc=0
 old_cc="$CC"
 old_cflags="$CFLAGS"
 OBJC='$(OBJZ) $(OBJG)'
@@ -121,7 +122,7 @@ case "$1" in
       echo '  configure [--const] [--zprefix] [--prefix=PREFIX]  [--eprefix=EXPREFIX]' | tee -a configure.log
       echo '    [--static] [--64] [--libdir=LIBDIR] [--sharedlibdir=LIBDIR]' | tee -a configure.log
       echo '    [--includedir=INCLUDEDIR] [--archs="-arch i386 -arch x86_64"]' | tee -a configure.log
-      echo '    [--disable-crcvx]' | tee -a configure.log
+      echo '    [--disable-crcvx] [--dfltcc] [--dfltcc-level-mask=MASK]' | tee -a configure.log
         exit 0 ;;
     -p*=* | --prefix=*) prefix=`echo $1 | sed 's/.*=//'`; shift ;;
     -e*=* | --eprefix=*) exec_prefix=`echo $1 | sed 's/.*=//'`; shift ;;
@@ -149,6 +150,10 @@ case "$1" in
     --address) address=1; shift ;;
     --memory) memory=1; shift ;;
     --disable-crcvx) enable_crcvx=0; shift ;;
+    --dfltcc)enable_dfltcc=1; shift ;;
+    --dfltcc-level-mask=*)
+      CFLAGS="$CFLAGS -DDFLTCC_LEVEL_MASK=`echo $1 | sed 's/.*=//'`"
+      shift ;;
     *) unknown=1; echo "unknown option ignored: $1" | tee -a configure.log; shift;;
     esac
 done
@@ -937,6 +942,15 @@ EOF
     fi
 fi
 
+# enable ibm s390x dfltcc extension
+HAVE_S390X_DFLTCC=0
+if test $HAVE_S390X -eq 1 && test $enable_dfltcc -eq 1; then
+    echo "Enabeling s390x dfltcc extension ... Yes." | tee -a configure.log
+    CFLAGS="$CFLAGS -DHAVE_S390X_DFLTCC"
+    OBJC="$OBJC dfltcc.o s390x-functable.o"
+    PIC_OBJC="$PIC_OBJC dfltcc.lo"
+fi
+
 # show the results in the log
 echo >> configure.log
 echo ALL = $ALL >> configure.log

contrib/README.contrib

@@ -47,7 +47,9 @@ puff/       by Mark Adler <madler@alumni.caltech.edu>
         unambiguous description of the deflate format.
 
 s390x/      by Ilya Leoshkevich <iii@linux.ibm.com>
-        Hardware-accelerated CRC32 on IBM Z with Z13 VX extension.
+        Hardware-accelerated CRC32 with Z13 VX extension and 
+        Hardware-accelerated deflate on IBM Z with Z15 DEFLATE CONVERSION CALL
+        instruction.
 
 testzlib/   by Gilles Vollant <info@winimage.com>
         Example of the use of zlib

contrib/functable/hooks.h

@@ -1,6 +1,11 @@
 #ifndef Z_HOOKS_H__
 #define Z_HOOKS_H__
 
+#ifdef HAVE_S390X_DFLTCC
+#include "../s390x/s390x-hooks.h"
+#include "../s390x/dfltcc.h"
+#else
+
 /**
  * DEFLATE HOOKS
  */
@@ -39,3 +44,5 @@
 #define ZFREE_STATE ZFREE
 #define ZFREE_WINDOW ZFREE
 #endif
+
+#endif 

contrib/s390x/CMakeLists.txt

@@ -1,4 +1,5 @@
 option(ZLIB_CRC32VX "Enable building S390-CRC32VX implementation" ON)
+option(ZLIB_DFLTCC  "Enable building S390-DFLTCC deflate/inflate accelerator implementation" OFF)
 
 set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")
 
@@ -62,7 +63,47 @@ if(ZLIB_CRC32VX AND HAS_S390X_SUPPORT)
         set_source_files_properties(
             crc32-vx.c
             PROPERTIES COMPILE_OPTIONS "${VGFMAFLAG}")
-        target_compile_definitions(
-            zlib_crc32_vx PUBLIC HAVE_S390X_VX=1)
+        target_compile_definitions(zlib_crc32_vx PUBLIC HAVE_S390X_VX=1)
+        target_compile_definitions(zlib_s390x_functable PUBLIC HAVE_S390X_VX=1)
     endif(HAS_S390X_VX_SUPPORT OR HAS_Z13_S390X_VX_SUPPORT)
 endif(ZLIB_CRC32VX AND HAS_S390X_SUPPORT)
+
+#
+# Check for IBM S390X - DFLTCC extensions
+#
+if(ZLIB_DFLTCC AND HAS_S390X_SUPPORT)
+    # check if we have static_assert
+    check_c_source_compiles("
+        #include <assert.h>
+        static_assert(1==1,\"true\");
+        " HAS_STATIC_ASSERT)
+
+    # check if we have secure_getenv
+    set(CMAKE_REQUIRED_FLAGS -D_GNU_SOURCE=1)
+    check_c_source_compiles("
+        #include <stdlib.h>
+        int main() { char* _foo = secure_getenv(\"PWD\");return 0; }
+        " HAS_SECURE_GETENV)
+    unset(CMAKE_REQUIRED_FLAGS)
+
+    # check for specific headers
+    check_include_file(sys/sdt.h HAS_SYS_SDT_H)
+
+    # prepare compiling for s390x
+    add_library(zlib_dfltcc OBJECT
+        dfltcc.c
+        ../functable/hooks.h)
+    target_link_libraries(zlib_dfltcc PUBLIC $<TARGET_OBJECTS:zlib_s390x_functable>)
+    target_compile_definitions(zlib_dfltcc PUBLIC HAVE_S390X_DFLTCC=1)
+    target_compile_definitions(zlib_s390x_functable PUBLIC HAVE_S390X_DFLTCC=1)
+    if(HAS_STATIC_ASSERT)
+        target_compile_definitions(zlib_dfltcc PRIVATE HAVE_STATIC_ASSERT=1)
+    endif()
+    if(HAS_SECURE_GETENV)
+        target_compile_definitions(zlib_dfltcc PRIVATE
+                                   HAVE_SECURE_GETENV=1 _GNU_SOURCE=1)
+    endif()
+    if(HAS_SYS_SDT_H)
+        target_compile_definitions(zlib_dfltcc PRIVATE HAVE_SYS_SDT_H=1)
+    endif()
+endif(ZLIB_DFLTCC AND HAS_S390X_SUPPORT)

contrib/s390x/README

@@ -7,3 +7,22 @@ ibm s390x. However this extension can disabled if desired:
 
     # for cmake build
     $ cmake .. -DZLIB_CRC32VX=off
+
+
+IBM Z mainframes starting from version z15 provide DFLTCC instruction,
+which implements deflate algorithm in hardware with estimated
+compression and decompression performance orders of magnitude faster
+than the current zlib and ratio comparable with that of level 1.
+
+This directory adds DFLTCC support. In order to enable it, the following
+build commands should be used:
+
+    $ ./configure --dfltcc
+    $ make
+
+When built like this, zlib would compress in hardware on level 1, and in
+software on all other levels. Decompression will always happen in
+hardware. In order to enable DFLTCC compression for levels 1-6 (i.e. to
+make it used by default) one could either configure with
+--dfltcc-level-mask=0x7e or set the environment variable
+DFLTCC_LEVEL_MASK to 0x7e at run time.