don't limit outputTimestamp to zip (MS DOS) range

[hboutemy]

fixes #308
drops check added in #22 and documented in #255

there is no reason to limit the value to something related to zip format: the timestamp may also be used for tar or any other archive format

failing a plugin for Reproducible Builds default timestamp value just creates a bad experience, that is really tricky to understand (default value of a plugin parameter injected from a property): if value has been defined with a value like "10", it's because user does not care about the effective timestamp in his archives, he just wants Reproducible Builds

let's keep things simple and not mix concerns: Reproducible Builds is about getting stable output, not a contract on precise zip entries timestamp

[michael-o]

You mean the contract has to come from the caller and not by the producing lib? Shit in, shit out?

[hboutemy]

the contract is: "give a stable binary output"
if the timestamp hint has to be tweaked, that's a valid compromise (for example the compromise in zip due to timezones)

I don't give any judgement on the provided value: it's the desired hint, and the JDK does not fail when using that hint

[michael-o]

Here is yet another problem I have:

maven-archiver/src/main/java/org/apache/maven/archiver/MavenArchiver.java

Line 113 in 009c817

private JarArchiver archiver;

This class is designed for JARs only, no tar or alike.

[hboutemy]

This class is designed for JARs only, no tar or alike.

1. no
2. not a reason

[michael-o]

This class is designed for JARs only, no tar or alike.

1. no
2. not a reason

I am confused about the first statement. What did I miss?

[hboutemy]

jar, with their manifest, is one use case for MavenArchiver
but you can use the timestamp value for any other type of archive

[hboutemy]

let's stop discussing about theory and not understanding each other:
here is a small java program to create a zip file in Java with SOURCE_DATE_EPOCH = 0, = 1970:

import java.io.*;
import java.util.zip.*;

public class ZipFile {

    public static void main(String[] args) throws IOException {
        FileOutputStream fos = new FileOutputStream("compressed.zip");
        ZipOutputStream zipOut = new ZipOutputStream(fos);

        ZipEntry zipEntry = new ZipEntry("f.txt");
        zipEntry.setTime(0);
        zipOut.putNextEntry(zipEntry);
        zipOut.close();
        fos.close();
    }
}

once run:

$ unzip -v compressed.zip
Archive:  compressed.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
       0  Defl:N        2   0% 1970-01-01 01:00 00000000  f.txt
--------          -------  ---                            -------
       0                2   0%                            1 file

$ bsdtar tvfz compressed.zip
-rw-rw-r--  0 0      0           0 janv.  1  1970 f.txt

I see no issue

$  zipdetails compressed.zip

0000 LOCAL HEADER #1       04034B50
0004 Extract Zip Spec      14 '2.0'
0005 Extract OS            00 'MS-DOS'
0006 General Purpose Flag  0808
     [Bits 1-2]            0 'Normal Compression'
     [Bit  3]              1 'Streamed'
     [Bit 11]              1 'Language Encoding'
0008 Compression Method    0008 'Deflated'
000A Last Mod Time         00210000 'Tue Jan  1 01:00:00 1980'
000E CRC                   00000000
0012 Compressed Length     00000000
0016 Uncompressed Length   00000000
001A Filename Length       0005
001C Extra Length          0009
001E Filename              'f.txt'
0023 Extra ID #0001        5455 'UT: Extended Timestamp'
0025   Length              0005
0027   Flags               '01 mod'
0028   Mod Time            00000000 'Thu Jan  1 01:00:00 1970'
002C PAYLOAD               ..

....

everything looks fine: yes, a timestamp not valid in MS DOS time range is encoded as 0023 Extra ID #0001 5455 'UT: Extended Timestamp'

definitively, JDK does a great job: I don't see any reason to limit values for Reproducible Builds

[michael-o]

jar, with their manifest, is one use case for MavenArchiver but you can use the timestamp value for any other type of archive

This I understand, but this specific class is for ZIP files only.

[michael-o]

let's stop discussing about theory and not understanding each other: here is a small java program to create a zip file in Java with SOURCE_DATE_EPOCH = 0, = 1970:

import java.io.*;
import java.util.zip.*;

public class ZipFile {

    public static void main(String[] args) throws IOException {
        FileOutputStream fos = new FileOutputStream("compressed.zip");
        ZipOutputStream zipOut = new ZipOutputStream(fos);

        ZipEntry zipEntry = new ZipEntry("f.txt");
        zipEntry.setTime(0);
        zipOut.putNextEntry(zipEntry);
        zipOut.close();
        fos.close();
    }
}

once run:

$ unzip -v compressed.zip
Archive:  compressed.zip
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
       0  Defl:N        2   0% 1970-01-01 01:00 00000000  f.txt
--------          -------  ---                            -------
       0                2   0%                            1 file

$ bsdtar tvfz compressed.zip
-rw-rw-r--  0 0      0           0 janv.  1  1970 f.txt

I see no issue

$  zipdetails compressed.zip

0000 LOCAL HEADER #1       04034B50
0004 Extract Zip Spec      14 '2.0'
0005 Extract OS            00 'MS-DOS'
0006 General Purpose Flag  0808
     [Bits 1-2]            0 'Normal Compression'
     [Bit  3]              1 'Streamed'
     [Bit 11]              1 'Language Encoding'
0008 Compression Method    0008 'Deflated'
000A Last Mod Time         00210000 'Tue Jan  1 01:00:00 1980'
000E CRC                   00000000
0012 Compressed Length     00000000
0016 Uncompressed Length   00000000
001A Filename Length       0005
001C Extra Length          0009
001E Filename              'f.txt'
0023 Extra ID #0001        5455 'UT: Extended Timestamp'
0025   Length              0005
0027   Flags               '01 mod'
0028   Mod Time            00000000 'Thu Jan  1 01:00:00 1970'
002C PAYLOAD               ..

....

everything looks fine: yes, a timestamp not valid in MS DOS time range is encoded as 0023 Extra ID #0001 5455 'UT: Extended Timestamp'

definitively, JDK does a great job: I don't see any reason to limit values for Reproducible Builds

This is irrelevant because Maven Archiver uses Plexus Archiver which uses Commons Compress. You need to test with Commons Compress first: https://github.com/codehaus-plexus/plexus-archiver/blob/d88dfdcaa41704a7a3b2fcab4247d51f8ca6cbc2/src/main/java/org/codehaus/plexus/archiver/zip/AbstractZipArchiver.java#L36-L40

Here: https://github.com/apache/commons-compress/blob/master/src/main/java/org/apache/commons/compress/archivers/zip/ZipArchiveEntry.java

We need to make sure that Commons Compress does the right thing and does not modify the value beyond java.util.zip. Then this is fine.

[hboutemy]

again mixing concerns: if the zip creation tool using the timestamp does not use it the right way, we need to fix the zip creation tool

not block the configuration in an obscure way, that makes it tricky now just to prove that the zip tool is misusing the value: please help instead of just forcing me to prove a problem that was never proven before "solving" it

[michael-o]

Agreed.

[hboutemy]

merging, then we can work on creating a sample jar file and investigate how it differs from sample JDK zip

[jorsol]

@hboutemy Just for the record, I added the check to align with the behavior of the jar tool:
https://bugs.openjdk.org/browse/JDK-8276766
openjdk/jdk@db68a0c

Also, plexus-archiver uses --date in JarToolModularJarArchiver.java if available:
https://github.com/codehaus-plexus/plexus-archiver/blob/d88dfdcaa41704a7a3b2fcab4247d51f8ca6cbc2/src/main/java/org/codehaus/plexus/archiver/jar/JarToolModularJarArchiver.java#L274-L290
So compressing modular JARs will fail anyway unless the --date check is removed and the JAR is “recompressed” again in the fallback section.

My personal preference would be to clamp the date to DATE_MIN, as @bmarwell suggested in #308. From a Reproducible Builds perspective, the key requirement is not a specific minimum date, but that the build process is deterministic: given the same source, it must always produce bit-for-bit identical binaries, clamping the date fits in this scenario.

There has already been a lot of work to ensure things behave correctly. ZIP’s core spec enforces 1980-01-01 as the minimum date, and while extended timestamp fields can encode earlier dates, they’re not consistently portable or respected. That’s why the Reproducible Builds community typically normalizes JAR entry dates to 1980.