parsing of reproducible builds 'project.build.outputTimestamp' cannot handle default dateFormat

[Interessierter]

Describe the bug (required)

in https://maven.apache.org/guides/mini/guide-reproducible-builds.html it is mentioned that you could do <project.build.outputTimestamp>${git.commit.time}</project.build.outputTimestamp> and this is IMHO indeed a neat idea.

If you do so, however, you'll get

[ERROR] Failed to execute goal io.github.git-commit-id:git-commit-id-maven-plugin:6.0.0:revision (get-the-git-infos) on project mautstation: Invalid 'project.build.outputTimestamp' value '2023-11-30T09:17:06+0100': Unparseable date: "2023-11-30T09:17:06+0100" -> [Help 1]

This is because the defaullt dateFormat is yyyy-MM-dd'T'HH:mm:ssZ but parseOutputTimestamp only tries to parse the format yyyy-MM-dd'T'HH:mm:ssXXX

It can be work-arounded by specifying the configuration <dateFormat>yyyy-MM-dd'T'HH:mm:ssXXX</dateFormat> but this shouldnt be neccessary.

Tell us about your plugin configuration (required)

		<plugin>
			<groupId>io.github.git-commit-id</groupId>
			<artifactId>git-commit-id-maven-plugin</artifactId>
			<version>6.0.0</version>
			<executions>
				<execution>
					<id>get-the-git-infos</id>
					<goals>
						<goal>revision</goal>
					</goals>
				</execution>
			</executions>
			<configuration>
				<generateGitPropertiesFile>true</generateGitPropertiesFile>
			</configuration>
		</plugin>

Tell us about the Plugin version used (required)

6.0.0

Tell us about the Maven version used (required)

mvn --version
Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: C:\Users\chris\AppData\Local\Programs\IntelliJ IDEA Ultimate\plugins\maven\lib\maven3
Java version: 17.0.2, vendor: Oracle Corporation, runtime: C:\java\openjdk-17.0.2
Default locale: de_DE, platform encoding: UTF-8
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Steps to Reproduce (required)

just configure the property <project.build.outputTimestamp>${git.commit.time}</project.build.outputTimestamp> in the POM

Are there any stacktraces or any error messages? (required)

Caused by: pl.project13.core.GitCommitIdExecutionException: Invalid 'project.build.outputTimestamp' value '2023-11-30T09:17:06+0100'
    at pl.project13.maven.git.GitCommitIdMojo.parseOutputTimestamp (GitCommitIdMojo.java:1511)
    at pl.project13.maven.git.GitCommitIdMojo$2.getReproducibleBuildOutputTimestamp (GitCommitIdMojo.java:1336)
    at pl.project13.core.GitCommitIdPlugin.loadBuildData (GitCommitIdPlugin.java:335)
    at pl.project13.core.GitCommitIdPlugin.runPlugin (GitCommitIdPlugin.java:295)
    at pl.project13.maven.git.GitCommitIdMojo.execute (GitCommitIdMojo.java:1453)

Is there a (public) project where this issue can be reproduced? (optional)

No response

Your Environment (optional)

No response

Context (optional)

No response

[Interessierter]

I could do an MR but I am unsure how to proceed.... IMHO the parseOutputTimestamp method should use the configured dateFormat for parsing but maybe there were reasons to use a hardcoded format? A relatively safe option would be to leave the existing two attempts as is and do a 3rd attempt with the configured dateFormat - WDYT?

[TheSnoozer]

Mhh this is a weird one.

The page https://maven.apache.org/guides/mini/guide-reproducible-builds.html mentions the following should be supported:

   <properties>
     <project.build.outputTimestamp>2023-01-01T00:00:00Z</project.build.outputTimestamp>
   </properties>

Which aligns with https://maven.apache.org/ref/3.3.1/maven-model-builder/ that states that the maven.build.timestamp that will also be outputted as yyyy-MM-dd'T'HH:mm:ss'Z'. The format can apparently be overwritten by maven.build.timestamp.format.

When clicking on the somewhat hidden link in (Enable Reproducible Builds mode for plugins, by adding project.build.outputTimestamp property to the project's pom.xml) mentioned in https://maven.apache.org/guides/mini/guide-reproducible-builds.html one comes across the following:

 * Timestamp for reproducible output archive entries, either formatted as ISO 8601
 * <code>yyyy-MM-dd'T'HH:mm:ssXXX</code> or as an int representing seconds since the epoch (like
 * <a href="https://reproducible-builds.org/docs/source-date-epoch/">SOURCE_DATE_EPOCH</a>).

a For Maven 2.x and 3.x, the value can be defined as an equivalent property:

<project>
  ...
  <properties>
    <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
    ...
  </properties>
  ...
</project>

So I guess here is one source of this secret XXX-Format.

When now checking the code of the plugin:

git-commit-id-maven-plugin/src/main/java/pl/project13/maven/git/GitCommitIdMojo.java

Lines 1484 to 1496 in fd67b7e

	/**
	* Parse output timestamp configured for Reproducible Builds' archive entries
	* (https://maven.apache.org/guides/mini/guide-reproducible-builds.html).
	* The value from <code>${project.build.outputTimestamp}</code> is either formatted as ISO 8601
	* <code>yyyy-MM-dd'T'HH:mm:ssXXX</code> or as an int representing seconds since the epoch (like
	* <a href="https://reproducible-builds.org/docs/source-date-epoch/">SOURCE_DATE_EPOCH</a>.
	*
	* Inspired by https://github.com/apache/maven-archiver/blob/a3103d99396cd8d3440b907ef932a33563225265/src/main/java/org/apache/maven/archiver/MavenArchiver.java#L765
	*
	* @param outputTimestamp the value of <code>${project.build.outputTimestamp}</code> (may be <code>null</code>)
	* @return the parsed timestamp, may be <code>null</code> if <code>null</code> input or input contains only 1
	* character
	*/

I see in the commentary that this was inspired by https://github.com/apache/maven-archiver/blob/a3103d99396cd8d3440b907ef932a33563225265/src/main/java/org/apache/maven/archiver/MavenArchiver.java#L765.

Here we have again the same/similar pattern of yyyy-MM-dd'T'HH:mm:ssXXX

A more recent version got rid of this hard-coded string:
https://github.com/apache/maven-archiver/blob/cc2f6a219f6563f450b0c00e8ccd651520b67406/src/main/java/org/apache/maven/archiver/MavenArchiver.java#L755

So I would suggest to use a similar parsing mechanism and drop the pattern with yyyy-MM-dd'T'HH:mm:ssXXX. Thus the entire parsing should IMHO look like:

return Date.from(OffsetDateTime.parse(outputTimestamp)
                    .withOffsetSameInstant(ZoneOffset.UTC)
                    .truncatedTo(ChronoUnit.SECONDS)
                    .toInstant());

Of course I think it also makes sense to update the comments to mention an yyyy-MM-dd'T'HH:mm:ssXXX format to refer just to ISO 8601. For example:

git-commit-id-maven-plugin/src/main/java/pl/project13/maven/git/GitCommitIdMojo.java

Line 1125 in fd67b7e

* <code>yyyy-MM-dd'T'HH:mm:ssXXX</code> or as an int representing seconds since the epoch (like

.

Does this make sense?
Could you create a PR and add relevant tests?
Thanks :-)

[Interessierter]

I finally got some time to look into this again. Unfortunately your suggestion does not work as (also new to me) the new java date parsers are not able to parse the RFC 822 format?!

Caused by: java.time.format.DateTimeParseException: Text '2024-02-09T16:48:52+0100' could not be parsed, unparsed text found at index 22
    at java.time.format.DateTimeFormatter.parseResolved0 (DateTimeFormatter.java:2055)
    at java.time.format.DateTimeFormatter.parse (DateTimeFormatter.java:1954)
    at java.time.OffsetDateTime.parse (OffsetDateTime.java:404)
    at java.time.OffsetDateTime.parse (OffsetDateTime.java:389)
    at pl.project13.maven.git.GitCommitIdMojo.parseOutputTimestamp (GitCommitIdMojo.java:1467)
    at pl.project13.maven.git.GitCommitIdMojo$2.getReproducibleBuildOutputTimestamp (GitCommitIdMojo.java:1290)
    at pl.project13.core.GitCommitIdPlugin.loadBuildData (GitCommitIdPlugin.java:335)
    at pl.project13.core.GitCommitIdPlugin.runPlugin (GitCommitIdPlugin.java:295)
    at pl.project13.maven.git.GitCommitIdMojo.execute (GitCommitIdMojo.java:1409)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
...

I then went ahead and modified the last lines of https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/master/src/main/java/pl/project13/maven/git/GitCommitIdMojo.java#L1464

to be the following (i.e. just re-try parsing it using the format the timestamp is created with from git);

    DateFormat df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssXXX");
    try {
      return df.parse(outputTimestamp);
    } catch (ParseException pe) {
      getLog().debug("Invalid 'project.build.outputTimestamp' value '" + outputTimestamp + "', re-trying " +
              "with default format '" + dateFormat + "'", pe);
    }

    df = new SimpleDateFormat(dateFormat);
    try {
      return df.parse(outputTimestamp);
    } catch (ParseException pe) {
      throw new GitCommitIdExecutionException(
              "Invalid 'project.build.outputTimestamp' value '" + outputTimestamp + "'", pe);
    }

this worked for this plugin, unfortunately then later on in my project the spring-boot-maven-plugin fails because it also cannot handle the RFC822 timestamp https://github.com/spring-projects/spring-boot/blob/main/spring-boot-project/spring-boot-tools/spring-boot-maven-plugin/src/main/java/org/springframework/boot/maven/BuildInfoMojo.java#L157 ;(

Caused by: java.time.format.DateTimeParseException: Text '2024-02-09T16:48:52+0100' could not be parsed at index 19
    at java.time.format.DateTimeFormatter.parseResolved0 (DateTimeFormatter.java:2052)
    at java.time.format.DateTimeFormatter.parse (DateTimeFormatter.java:1954)
    at java.time.Instant.parse (Instant.java:397)
    at org.springframework.boot.maven.BuildInfoMojo.getBuildTime (BuildInfoMojo.java:157)
...

I thought that maybe using the format yyyy-MM-dd'T'HH:mm:ssXXX as default for the commit timestamp would be most useful way to continue and changed https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/master/src/main/java/pl/project13/maven/git/GitCommitIdMojo.java#L513 but to my surprise this did not work, I still got the error from the Spring Boot plugin above, only setting it explicitly in the git plugins configuration did work, is it possible that this default value isn't used but there is another default value in the project that is actually using the values (git-commit-id-plugin-core-6.0.0-rc.6.jar)?

Ultimately I am unsure how to proceed and will stick with the workaround I am currently configured (explicitly configuring a working format in the plugins configurion in pom.xml)

[TheSnoozer]

The time must be formatted as ISO 8601. Can you check if the code in https://github.com/git-commit-id/git-commit-id-maven-plugin/pull/699/files would cover your case?

[Interessierter]

Thank you very much for your response!

Unfortunately it does not, as (maybe this was not clear enough in my ramblings above) the plugin itself generates an invalid dateformat for further maven processing (as seen here: https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/master/src/main/java/pl/project13/maven/git/GitCommitIdMojo.java#L513). I cobbled together a quick reproducer project:

git-plugin-timestamp-problem-demo.zip

1. unzip it to some dir
2. run git init in that dir
3. add all the files to the empty repository, i.e. git add . in that dir
4. commit it, i.e. git commit -m "initial commit" in that dir
5. run mvn clean package to see the build fail

This is caused because I set the project.build.outputTimestamp to the plugins value (line 18 in pom.xml) - which is the heart of the reproducible build feature. If I explicitly set the output format to ISO8601 (commented out line 54 in the pom.xml) all is well. An like I said: just changing line 513 in GitCommitIdMojo is not enough, the plugin still generates an "invalid" format, it has to be set explicitly in the configuration.

[TheSnoozer]

Thanks for the additional explanations and the reproducer! Sorry maybe it was just me being slow in understanding what you really mean, but I think I got it now.
Essentially you would like to pass in any time-formatted values that are produced by the plugin like <project.build.outputTimestamp>${git.commit.time}</project.build.outputTimestamp>.

For that to happen (without any extra <dateFormat>yyyy-MM-dd'T'HH:mm:ssXXX</dateFormat>) the default value of the dateFormat would need to be changed (see https://github.com/git-commit-id/git-commit-id-maven-plugin/blob/master/src/main/java/pl/project13/maven/git/GitCommitIdMojo.java#L513).

Likely this would mean it's a breaking change, since as a plugin developer I don't know what weird features the plugin users rely on...

[Interessierter]

no worries :) As I said in my original message, doing <project.build.outputTimestamp>${git.commit.time}</project.build.outputTimestamp> is explicitly mentioned on Mavens reproducible build page (last item in FAQ) as this enables truly automatic reproducible builds but then this clashes with the "historic" dateformat of this plugin, which could not be consumed by the plugin itself (until your changes from #699 are applied) but also cause problems in other Maven plugins (like the not-so-seldomly-used spring-boot-maven-plugin`` and maybe others) because they too expect ISO8601 instead of RFC822 format.

So IMHO yes, you should do a breaking change as the "new" default is more sensible - this could clearly be advertised in the release notes and so on ("default format changed, if you need the old format add <dateFormat>yyyy-MM-dd'T'HH:mm:ssZ</dateFormat> as configuration").

Also, like I said above, I did exactly this locally (i.e. changed line 513 in GitCommitIdMojo to be @Parameter(defaultValue = "yyyy-MM-dd'T'HH:mm:ssXXX")) and installed SNAPSHOT version locally and used that and it had no effect, the timestamp was still in RFC822 format so there addtionally may be a bug somewhere (unfortunately I have no knowledge on how to debug a maven plugin and the "real work" is done in the other project (git-commit-id-plugin-core-6.0.0-rc.6) which I did not checkout)

[TheSnoozer]

No worries, just to make sure I don't overlook something I ran a quick test and setting the @Parameter(defaultValue = "yyyy-MM-dd'T'HH:mm:ssXXX"). It indeed does the trick, so maybe you still had/have a dateFormat encoded in your pom.xml (remember the plugin just suggests a default, everything can be overwritten by the user).

I have tested #699 with the demo profile that is encoded in the pom:

# First install the plugin so we can use unreleased changes
mvn clean install -Dmaven.test.skip=true
# Run it with demo profile encoded in pom
mvn clean package -Pdemo -Dmaven.test.skip=true

With a configuration of:

<!-- <dateFormat>yyyy-MM-dd'T'HH:mm:ssXXX</dateFormat> -->
<dateFormatTimeZone>GMT-08:00</dateFormatTimeZone>

this produces a 2024-02-14T10:06:03-08:00 which is a valid ISO 8601 representation.

[Interessierter]

It does not work on my machine :( I see that your changes should be good in theory but there's something wrong with it, I did the following, where is the mistake (could you try it on your environment):

1. I deleted the plugin from my maven repository (i.e. wiped out the whole %USERPROFILE%\.m2\repository\io\github\git-commit-id\git-commit-id-maven-plugin\ directory)
2. I pulled the 674 branch from the repo locally, checked that it contained your changes
3. I mvn clean installed the plugin, checked that it does appear again in the maven repo
4. I ran the reproducer I sent you earlier, it failes, this time in the maven-jar-plugin:

[INFO] --- maven-jar-plugin:3.3.0:jar (default-jar) @ demo ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.420 s
[INFO] Finished at: 2024-02-15T14:01:26+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.3.0:jar (default-jar) on project demo: Execution default-jar of goal org.apache.maven.plugins:maven-jar-plugin:3.3.0:jar failed: Invalid project.build.outputTimestamp value '2024-02-15T13:54:59+0100': Text '2024-02-15T13:54:59+0100' could not be parsed, unparsed text found at index 22 -> [Help 1]

I can see it in the plugins output that the wrong value is generated:

...
[INFO] --- git-commit-id-maven-plugin:7.0.1-SNAPSHOT:revision (default) @ demo ---
[INFO] dotGitDirectory 'C:\git\stuff\git-plugin-timestamp-problem-demo\.git'
[INFO] Collected git.build.user.name with value cXXXXXX mXXXXXXX
[INFO] Collected git.build.user.email with value cXXXXXX.mXXXXXXX@example.com
[INFO] Collected git.branch with value master
[INFO] Collected git.commit.id.describe with value 3c5db83
[INFO] Collected git.commit.id.describe-short with value 3c5db83
[INFO] Collected git.commit.id with value 3c5db8321936ee5ca766b584e67cd002ca4b76a4
[INFO] Collected git.commit.id.abbrev with value 3c5db83
[INFO] Collected git.dirty with value false
[INFO] Collected git.commit.user.name with value cXXXXXX mXXXXXXX
[INFO] Collected git.commit.user.email with value cXXXXXX.mXXXXXXX@example.com
[INFO] Collected git.commit.message.full with value initial commit
[INFO] Collected git.commit.message.short with value initial commit
[INFO] Collected git.commit.time with value 2024-02-15T13:54:59+0100
[INFO] Collected git.commit.author.time with value 2024-02-15T13:54:59+0100
[INFO] Collected git.commit.committer.time with value 2024-02-15T13:54:59+0100
[INFO] Collected git.remote.origin.url with value null
[INFO] Collected git.tags with value 
[INFO] Collected git.closest.tag.name with value 
[INFO] Collected git.closest.tag.commit.count with value 
[INFO] Collected git.total.commit.count with value 1
[INFO] Collected git.local.branch.ahead with value NO_REMOTE
...

What am I missing/doing wrong?? Again thank you for your investigations and effort, I really appreciate it 👍