Unable to obtain git.branch property with SonarQube build

[NicolaSpreafico]

Hi,
in my Maven project I have configured SonarQube analysis, and I want to use the name of the current git-branch in the analysis version.

Here is my configuration:

<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<sonar.projectVersion>${project.version}-${git.branch}</sonar.projectVersion>
	</properties>

And here is the result

The build is triggered using mvn clean verify sonar:sonar

I added an echo output to make sure the property is correctly created, and there is:

[INFO] --- maven-antrun-plugin:1.8:run (echo-properties) @ git-test-sonarqube ---
[INFO] Executing tasks

main:
     [echo] git.branch: feature/my_feature
[INFO] Executed tasks

But when the sonarqube plugin is triggered (after the echo, so I would expect to have the property value), on SonarQube dashboard I will find the property not resolved.

Here is my current plugin configuration

<build>
		<plugins>
			<plugin>
				<groupId>pl.project13.maven</groupId>
				<artifactId>git-commit-id-plugin</artifactId>
				<version>2.2.6</version>
				<configuration>
					<injectAllReactorProjects>true</injectAllReactorProjects>
				</configuration>
				<executions>
					<execution>
						<id>get-the-git-infos</id>
						<goals>
							<goal>revision</goal>
						</goals>
					</execution>
				</executions>
			</plugin>

			<plugin>
				<artifactId>maven-antrun-plugin</artifactId>
				<version>1.8</version>
				<executions>
					<execution>
						<id>echo-properties</id>
						<phase>compile</phase>
						<goals>
							<goal>run</goal>
						</goals>
						<configuration>
							<target>
								<echo>git.branch: ${git.branch}</echo>
							</target>
						</configuration>
					</execution>
				</executions>
			</plugin>

		</plugins>

		<pluginManagement>
			<plugins>
				<plugin>
					<groupId>org.sonarsource.scanner.maven</groupId>
					<artifactId>sonar-maven-plugin</artifactId>
					<version>3.6.0.1398</version>
				</plugin>
			</plugins>
		</pluginManagement>
	</build>

I enable full log with this command
mvn clean verify sonar:sonar -Dsonar.analysis.mode=preview -Dsonar.verbose=true -X -Dsonar.log.level=DEBUG > R:\log.txt

And found the section related to properties

[DEBUG] Initialize Maven Ant Tasks
parsing buildfile jar:file:/C:/Users/Nicola/.m2/repository/org/apache/maven/plugins/maven-antrun-plugin/1.8/maven-antrun-plugin-1.8.jar!/org/apache/maven/ant/tasks/antlib.xml with URI = jar:file:/C:/Users/Nicola/.m2/repository/org/apache/maven/plugins/maven-antrun-plugin/1.8/maven-antrun-plugin-1.8.jar!/org/apache/maven/ant/tasks/antlib.xml from a zip file
parsing buildfile jar:file:/C:/Users/Nicola/.m2/repository/org/apache/ant/ant/1.9.4/ant-1.9.4.jar!/org/apache/tools/ant/antlib.xml with URI = jar:file:/C:/Users/Nicola/.m2/repository/org/apache/ant/ant/1.9.4/ant-1.9.4.jar!/org/apache/tools/ant/antlib.xml from a zip file
Class org.apache.maven.ant.tasks.AttachArtifactTask loaded from parent loader (parentFirst)
 +Datatype attachartifact org.apache.maven.ant.tasks.AttachArtifactTask
Class org.apache.maven.ant.tasks.DependencyFilesetsTask loaded from parent loader (parentFirst)
 +Datatype dependencyfilesets org.apache.maven.ant.tasks.DependencyFilesetsTask
Setting project property: git.build.user.email -> XXXXXX
Setting project property: git.build.host -> XXXXXX
Setting project property: git.dirty -> true
Setting project property: git.remote.origin.url -> Unknown
Setting project property: git.closest.tag.name -> 
Setting project property: sonar.login -> XXXXXX
Setting project property: project.build.sourceEncoding -> UTF-8
Setting project property: sonar.host.url -> XXXXXX
Setting project property: git.total.commit.count -> 1
Setting project property: git.commit.id.describe-short -> be3e227-dirty
Setting project property: git.commit.user.email -> XXXXXX
Setting project property: git.commit.time -> 2019-04-24T08:29:19+0200
Setting project property: git.commit.message.full -> first commit
Setting project property: git.build.version -> 1.0.0
Setting project property: git.commit.message.short -> first commit
Setting project property: git.commit.id.abbrev -> be3e227
Setting project property: git.build.user.name -> XXXXXX
Setting project property: git.branch -> feature/my_feature
Setting project property: sonar.projectVersion -> 1.0.0-${git.branch}
Setting project property: git.closest.tag.commit.count -> 
Setting project property: git.commit.id.describe -> be3e227-dirty
Setting project property: git.commit.id -> XXXXXX
Setting project property: git.tags -> 
Setting project property: git.build.time -> 2019-04-24T08:43:05+0200
Setting project property: git.commit.user.name -> XXXXXX
Setting project property: ant.file -> R:\git-test\pom.xml
[DEBUG] Setting properties with prefix: 
Setting project property: project.groupId -> com
Setting project property: project.artifactId -> git-test-sonarqube
Setting project property: project.name -> git-test-sonarqube
Setting project property: project.version -> 1.0.0
Setting project property: project.packaging -> jar
Setting project property: project.build.directory -> R:\git-test\target
Setting project property: project.build.outputDirectory -> R:\git-test\target\classes
Setting project property: project.build.testOutputDirectory -> R:\git-test\target\test-classes
Setting project property: project.build.sourceDirectory -> R:\git-test\src\main\java
Setting project property: project.build.testSourceDirectory -> R:\git-test\src\test\java
Setting project property: localRepository ->       id: local
      url: file:///C:/Users/Nicola/.m2/repository/
   layout: default
snapshots: [enabled => true, update => always]
 releases: [enabled => true, update => always]

Setting project property: settings.localRepository -> C:\Users\Nicola\.m2\repository
Setting project property: maven.project.dependencies.versions -> 
[INFO] Executing tasks

As far I can see, the property are managed, but the sonar version is not resolved:

Setting project property: git.branch -> feature/my_feature
Setting project property: sonar.projectVersion -> 1.0.0-${git.branch}

Do you know similar cases or possible solutions?

[TheSnoozer]

Mhh in all honesty it wouldn't surprise me if this would be related to #287. Just out curiosity could you try mvn clean verify git-commit-id-plugin:revision sonar:sonar?

[NicolaSpreafico]

I had already tried before creating this issue, sorry I forgot to also mention this.

This is the command I tried after first failed attempts:
mvn clean verify git-commit-id:revision sonar:sonar

But at the end the result does not change:

Here is the output of
mvn clean verify git-commit-id:revision sonar:sonar -Dsonar.analysis.mode=preview -Dsonar.verbose=true -X -Dsonar.log.level=DEBUG > R:\log.txt

Setting project property: git.branch -> feature/my_feature
Setting project property: sonar.projectVersion -> 0.0.1-SNAPSHOT-${git.branch}

[TheSnoozer]

Let me be very clear from the start:
This is a ugly, ugly, ugly workaround and I provide it without any warranty whatsoever...

Ok so in summary I think I have finally found two workarounds, and one that is not directly usable and needs adjustments to the plugin. On a high level the workarounds can be summarized as following:

1. set some magic environment variables to do the 'right' thing
2. overwrite the 'sonar.projectVersion' with a third party plugin gmaven-plugin to the 'right' value
3. enable the plugin to replace any property (e.g. similar to two, but the plugin's replacementProperties can currently not overwrite foreign properties like 'sonar.projectVersion' )

Workaround 1

The sonar-scanner-maven extract's its properties from System.getenv() and happens to also to respect the magic environment variable SONARQUBE_SCANNER_PARAMS.
Setting this to a JSON Object makes sonar 'use' the right values:

export GIT_COMMIT=$(git rev-parse --short HEAD)
export GIT_BRANCH=$(git symbolic-ref --short -q HEAD)
export MVN_VERSION=$(echo '${project.version}' | mvn help:evaluate | grep -v '^[[]')

export SONARQUBE_SCANNER_PARAMS="{\"sonar.projectVersion\" : \"${MVN_VERSION}-${GIT_BRANCH}\"}"

mvn clean package sonar:sonar

Workaround 2

For this workaround we essentially generate the 'right' 'sonar.projectVersion' with an third party plugin. And because I couldn't get this approach to work with the build-helper-maven-plugin, properties-maven-plugin or maven-antrun-plugin I ended up going with gmaven-plugin (sorry this sounds for me so insane that we actually now need to use groovy in maven, but whatever):

<!-- make sure the git-commit-id-plugin generates a file via generateGitPropertiesFilename -->
<plugin>
   <groupId>pl.project13.maven</groupId>
   <artifactId>git-commit-id-plugin</artifactId>
   <version>2.6</version>
   <executions>
      <execution>
         <id>get-the-git-infos</id>
         <goals>
            <goal>revision</goal>
         </goals>
         <phase>initialize</phase>
      </execution>
   </executions>
   <configuration>
      <prefix>git</prefix>
      <verbose>true</verbose>
      <skipPoms>false</skipPoms>
      <!-- <runOnlyOnce>true</runOnlyOnce> -->
      <dotGitDirectory>${project.basedir}/../.git</dotGitDirectory>
      <injectAllReactorProjects>true</injectAllReactorProjects>
      <generateGitPropertiesFile>true</generateGitPropertiesFile>
      <evaluateOnCommit>HEAD</evaluateOnCommit>
      <generateGitPropertiesFilename>${project.build.outputDirectory}/git.properties</generateGitPropertiesFilename>
      <replacementProperties>
         <replacementProperty>
            <property>sonar.projectVersion</property>
            <token>^.*$</token>
            <value>${project.version}-${git.branch}</value>
            <regex>false</regex>
         </replacementProperty>
      </replacementProperties>
   </configuration>
</plugin>

<!-- now read the generated git file and create the sonar.projectVersion property -->
<plugin>
   <groupId>org.codehaus.gmaven</groupId>
   <artifactId>gmaven-plugin</artifactId>
   <version>1.5</version>
   <executions>
      <execution>
         <id>set-custom-property</id>
         <phase>compile</phase>
         <goals>
            <goal>execute</goal>
         </goals>
         <configuration>
            <source>Properties gitProperties = new Properties()
                new File('${project.build.outputDirectory}/git.properties').withInputStream {
                    gitProperties.load(it)
                }

                project.properties.setProperty('sonar.projectVersion', "${project.version}-" + gitProperties.getProperty("git.branch"))</source>
         </configuration>
      </execution>
   </executions>
</plugin>

Workaround 3 (Needs adjustment to the plugin)

For this approach I had in mind to simply use the replacementProperties to set sonar.projectVersion to the right thing (similar to 2). However the replacementProperties currently only work with properties generated by the plugin 8not sure if I want to change that). The idea pretty much boils down to this:

<plugin>
   <groupId>pl.project13.maven</groupId>
   <artifactId>git-commit-id-plugin</artifactId>
   <version>${git-commit-id-version}</version>
   <executions>
      <execution>
         <id>get-the-git-infos</id>
         <goals>
            <goal>revision</goal>
         </goals>
         <phase>initialize</phase>
      </execution>
      <execution>
         <id>validate-the-git-infos</id>
         <goals>
            <goal>validateRevision</goal>
         </goals>
         <phase>compile</phase>
      </execution>
   </executions>
   <configuration>
      <prefix>git</prefix>
      <verbose>true</verbose>
      <skipPoms>false</skipPoms>
      <!-- <runOnlyOnce>true</runOnlyOnce> -->
      <dotGitDirectory>${project.basedir}/../.git</dotGitDirectory>
      <injectAllReactorProjects>true</injectAllReactorProjects>
      <generateGitPropertiesFile>true</generateGitPropertiesFile>
      <evaluateOnCommit>HEAD</evaluateOnCommit>
      <generateGitPropertiesFilename>${project.build.outputDirectory}/git.properties</generateGitPropertiesFilename>
      <replacementProperties>
         <replacementProperty>
            <property>sonar.projectVersion</property>
            <token>^.*$</token>
            <value>${project.version}-${git.branch}</value>
            <regex>false</regex>
         </replacementProperty>
      </replacementProperties>
   </configuration>
</plugin>

maybe something even simpler would be possible. Like a new setting <forceResolutionOfProperties> (or something) that goes through all the current project properties and checks if they contain any properties generated by the plugin that haven't been expanded like 0.0.1-SNAPSHOT-${git.branch}. If this setting is enabled we would replace the values appropriately - Note after honering the include and exclude properties of course...

TLDR

Thank you for reminding me why i stopped using maven :-)

Hope this helps and sorry for the nastiness that is involved here - it seems i con't do anything about it.

---

In case I ever need this again: https://docs.docker.com/samples/library/sonarqube/

[NicolaSpreafico]

Thank you for your very detailed asnwer. I did not understood the whole of it (I do not know this side of Maven), but it is clear which is the problem and possible workarounds.

I do not know if you get a notification but I can see that there is another reply from Maven issue.

[TheSnoozer]

Thanks for the alert - and no I don't get e-Mail notifications from Maven issues.

In summary I'd claim we have slightly distinct issues - let me summarize how I understand those two issues:

Execute Plugin with it's Plugin's Prefix and pass variable through the direct config (#287)

For #287 we essentially use the plugin configuration to pass dynamic properties and use maven's prefix resolution.
In a nutshell what we want to have is a plugin execution/configuration that looks this:

<plugin>
  <groupId>com.test.plugins</groupId>
  <artifactId>testPlugin</artifactId>
  <version>1.0.0</version>
  <configuration>
    <project>${git.branch}</project>
  </configuration>
  <executions>
    <execution>
      <id>deploy</id>
      <phase>install</phase>
      <goals>
        <goal>deploy</goal>
      </goals>
    </execution>
  </executions>
</plugin>

and want to run it with it's plugin prefix (aka. mvn com.test.plugins:testPlugin:deployMojo). That was the the original issue I filed on maven. As per response what is happening is that the prefix resolution initializes the configuration of the plugin before we had the chance to get the variable git.branch. As a result the plugin is essentially configured with an uninitialized variable (e.g. ${git.branch}). The person who commented on the maven issue claims that some user may want the value without any modifications (so the uninitialized value). A supporting factor for this argument is that someone could expect some sort of isolated execution when running a plugin with it's prefix.

If a user wants to tell the plugin about the updated value you either need to integrate the plugin in a normal life-cycle (mvn clean install), or run the plugin who generates the property beforehand (mvn pl.project13.maven:git-commit-id-plugin:revision com.test.plugins:testPlugin:deployMojo) - explained here in more detail. Alternatively the plugin who consumes the property provides some sort of support for such dynamic variable (as the sonar project does).

Execute Plugin with it's Plugin's Prefix and pass variable through session / project properties (#413)

This is somewhat close to the alternative that is suggested in the maven issue. Since the configuration is initialized statically we could pass variables as System environment variables if the plugin supports such behaviour.
The sonar-scanner-maven plugin happens to supports such dynamic properties and as mentioned in one of the workarounds extract's its properties from System.getenv() and happens to also to respect the magic environment variable SONARQUBE_SCANNER_PARAMS. Using this environment variable is essentially the first suggested workaround.

The second workaround is what I assume a similar issue to the property get's initialized when we don't know the right git.branch property. I assume what is happening is that during initialization of maven (before any plugin is execution) the property sonar.projectVersion is resolved. As mention in your issue you had set the variable to ${project.version}-${git.branch}. However at that initialization stage the variable git.branch is unresolved leading to the property sonar.projectVersion being set to 1.0.0-${git.branch}. The second workaround essentially overwrite the property sonar.projectVersion at a stage where we happen to know the right value of the git.branch and thus resulting in the right variable.

This is also essentially the idea for the third workaround that leverages that the sonar plugin also checks the project properties on a project level. Essentially I want to use the functionality of the replacementProperties to set the sonar.projectVersion at a stage where we happen to know the right value of the git.branch. So in a nutshell it's the same as the second workaround without using an additional third party plugin. However as of now this plugin only allows replacementProperties that had been generated by the plugin itself (and thus fails for the external sonar.projectVersion property) and hence this workaround would need adjustments to this plugin itself (not sure about those).

TLDR

I hope this makes sense and gives a better insight on what is going on. For this issue I have hope that I could simplify the problem slightly for you, however for the #287 issue as commented in the maven issue it is not expected that this will change at any time (I feel I just go ahead and close the issues as Won't fix). Relying on system.properties certainly doesn't make it easier since it's a pain to modify those with java.

Outstanding work for this plugin

Make it possible to set or overwrite external properties with replacementProperties.

[TheSnoozer]

I have removed the 3.0 milestone again, since the outstanding work (e.g. Make it possible to set or overwrite external properties with replacementProperties) is more complicated to achieve the desired behaviour.

Essentially the suggested configuration

<replacementProperties>
         <replacementProperty>
            <property>sonar.projectVersion</property>
            <token>^.*$</token>
            <value>${project.version}-${git.branch}</value>
            <regex>false</regex>
         </replacementProperty>
      </replacementProperties>

results in the fact that during initialization the of the plugin the git.branch property is unresolved and thus resulting in the fact that the plugin only sees 0.0.3-SNAPSHOT-${git.branch}. that is certainly not desired.

What would be needed here is that the plugin also attempts to resolve the variables correctly.

Memo for myself (and how to to it properly):
The maven-resources-plugin uses the filtering logic from a shared module. This particular module seems to relies on plexus' string interpolation. This assumption is also confirmed by the flatten-maven-plugin. So in a nutshell in order to have the plugin produce the right values we would need to perform this particular interpolation ourself. I can't find any method to do it any way smarter....(thanks maven).

See this for possible inspiration.

== EDIT:
Check if the PluginParameterExpressionEvaluator might be the answer to this problem

import org.apache.maven.plugin.PluginParameterExpressionEvaluator;

private PluginParameterExpressionEvaluator expressionEvaluator;

expressionEvaluator = new PluginParameterExpressionEvaluator(session, execution);

// https://github.com/spotify/docker-maven-plugin/blob/c5bbc1c4993f5dc37c96457fa5de5af8308f7829/src/main/java/com/spotify/docker/BuildMojo.java#L574
// https://maven.apache.org/ref/3.6.1/maven-core/apidocs/org/apache/maven/plugin/PluginParameterExpressionEvaluator.html
private String expand(final String raw) throws MojoExecutionException {
    final Object value;
    try {
      value = expressionEvaluator.evaluate(raw);
    } catch (ExpressionEvaluationException e) {
      throw new MojoExecutionException("Expression evaluation failed: " + raw, e);
    }

    if (value == null) {
      throw new MojoExecutionException("Undefined expression: " + raw);
    }

    return value.toString();
}