apply replace on property xxx: original value 'null' with ''

[mmoossen]

Description

I have the same use case than #413, i want to use git.branch in sonar.projektKey.
for that i am trying to use:

                    <replacementProperties>
                        <replacementProperty>
                            <property>sonar.projectKey</property>
                            <token>^.*$</token>
                            <value>${project.groupId}:${project.artifactId}:${git.branch}</value>
                            <regex>false</regex>
                        </replacementProperty>
                    </replacementProperties>

but i get:

[INFO] --- git-commit-id-plugin:4.0.0:revision (default-cli) @ test ---
[INFO] Current project: 'test', first project to execute based on dependency graph: 'test'
[INFO] dotGitDirectory K:\repos\git\auv-fachanwendung\.git
[INFO] Collected git.branch with value develop
[INFO] apply replace on property sonar.projectKey: original value 'null' with ''
[INFO] including property git.branch in results
[INFO] including property sonar.projectKey in results

Expected behavior

i expect that git.branch gets replaced in sonar.projectKey

Additional context

• the plugin version is being used: 4.0.0
• the Java-Version is being used: Oracle JDK "1.8.0_172"
• the Maven-Version is being used: Apache Maven 3.6.3
• on what Operating System you experience the bug: Windows 10 Enterprise Build 17763
• in what context maven is being executed: Git-Bash 2.24.02 or IntelliJ 2019.3
• how maven is being executed: mvn git-commit-id:revision

here the whole pom.xml to reproduce the problem:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <name>test</name>
    <groupId>de.test</groupId>
    <artifactId>test</artifactId>
    <version>1-SNAPSHOT</version>
    <packaging>pom</packaging>
    <properties>
        <sonar.projectKey>${project.groupId}:${project.artifactId}:${git.branch}</sonar.projectKey>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>pl.project13.maven</groupId>
                <artifactId>git-commit-id-plugin</artifactId>
                <version>4.0.0</version>
                <executions>
                    <execution>
                        <id>get-the-git-infos</id>
                        <goals>
                            <goal>revision</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <injectAllReactorProjects>true</injectAllReactorProjects>
                    <runOnlyOnce>true</runOnlyOnce>
                    <skipPoms>false</skipPoms>
                    <includeOnlyProperties>
                        <includeOnlyProperty>^git.branch$</includeOnlyProperty>
                    </includeOnlyProperties>
                    <verbose>true</verbose>
                    <!-- see: https://github.com/git-commit-id/maven-git-commit-id-plugin/issues/413 -->
                    <replacementProperties>
                        <replacementProperty>
                            <property>sonar.projectKey</property>
                            <token>^.*$</token>
                            <value>${project.groupId}:${project.artifactId}:${git.branch}</value>
                            <regex>false</regex>
                        </replacementProperty>
                    </replacementProperties>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

and the Output of mvn git-commit-id:revision:

$ mvn git-commit-id:revision
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------------< de.test:test >----------------------------
[INFO] Building test 1-SNAPSHOT
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- git-commit-id-plugin:4.0.0:revision (default-cli) @ test ---
[INFO] Current project: 'test', first project to execute based on dependency graph: 'test'
[INFO] dotGitDirectory C:\repos\git\test\.git
[INFO] Collected git.branch with value develop
[INFO] apply replace on property sonar.projectKey: original value 'null' with ''
[INFO] including property git.branch in results
[INFO] including property sonar.projectKey in results
[INFO] Added properties to 1 projects
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.922 s
[INFO] Finished at: 2019-12-03T17:16:31+01:00
[INFO] ------------------------------------------------------------------------

what can be wrong?

many thanks

[TheSnoozer]

Thank you for reporting this!
After some investigation i think this might be a bug, but I wonder why it appears to work.
In specific when I add some ant-run echoing the property seem to look alright:
e.g.:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <name>test</name>
    <groupId>de.test</groupId>
    <artifactId>test</artifactId>
    <version>1-SNAPSHOT</version>
    <packaging>pom</packaging>
    <properties>
        <sonar.projectKey>${project.groupId}:${project.artifactId}:${git.branch}</sonar.projectKey>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>pl.project13.maven</groupId>
                <artifactId>git-commit-id-plugin</artifactId>
                <version>4.0.1-SNAPSHOT</version>
                <executions>
                    <execution>
                        <id>get-the-git-infos</id>
                        <goals>
                            <goal>revision</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <injectAllReactorProjects>true</injectAllReactorProjects>
                    <runOnlyOnce>true</runOnlyOnce>
                    <skipPoms>false</skipPoms>
                    <includeOnlyProperties>
                        <includeOnlyProperty>^git.branch$</includeOnlyProperty>
                    </includeOnlyProperties>
                    <verbose>true</verbose>
                    <!-- see: https://github.com/git-commit-id/maven-git-commit-id-plugin/issues/413 -->
                    <replacementProperties>
                        <replacementProperty>
                            <property>sonar.projectKey</property>
                            <token>.*</token>
                            <value>${project.groupId}:${project.artifactId}:${git.branch}</value>
                            <regex>false</regex>
                        </replacementProperty>
                    </replacementProperties>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.8</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <configuration>
                            <target>
                                <echo>===========================================================================</echo>
                                <echo>${sonar.projectKey}</echo>
                            </target>
                        </configuration>
                        <goals>
                            <goal>run</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</project>

Can you double check what ends up in sonar?
is it the right property or some yibberish?

Regardless what shows up, I still wonder why this works....the bug can be traced down inside the PropertiesReplacer where such specific replacements results in the following call-chain:
performReplacement -> performReplacementOnSingleProperty -> performReplacement -> replaceNonRegex

For the last replaceNonRegex the content is the empty-string mainly due to the fact that the content variable will become null when we get the property from the available properties in performReplacementOnSingleProperty. All properties are the properties that are generated by the plugin (hence the property collection does not include sonar.projectKey and will result in the null value). The null will then be shadowed by an empty string caused by the orElse in the expressionEvaluator expression.

If I'm not mistaken to fix this the replaceRegex and replaceNonRegex would need an additional condition like

if (content == null or content..isEmpty()) { return value; }

[mmoossen]

according to my tests, the property gets set wrongly:

ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.7.0.1746:sonar (default-cli) on project auv-parent: Validation of project reactor failed:
[ERROR]   o "" is not a valid project or module key. It cannot be empty nor contain whitespaces.

this means that the explicitly set value of the property is overwritten with the empty string.

[TheSnoozer]

I ran some debugging locally and I think I fixed the general issue already.
However as of now one can not use <value>${project.groupId}:${project.artifactId}:${git.branch}</value> in a multi-reactor build with sonar.
Sonar forces to have unique module names (which makes sense) and currently how my idea works is that the plugin will set the value sonar.projectKey when it's empty (which will only be the case for the root-project but not for any sub-modules). Once the reactor root executed the sonar.projectKey will appear to be set. This essentially results in the fact that any sub-module will reuse the same identifier as the reactor-root.

What might work in this case is an ability to reset the property or to force an overwrite. Not sure if we would get the ${project.artifactId} from the reactor-root (where the plugin execution is defined) or from the actual child.

Perhaps this idea could look something along the lines of:

<replacementProperty>
    <property>sonar.projectKey</property>
    <token>.*</token>
    <value>${project.groupId}:${project.artifactId}:${git.branch}</value>
    <regex>false</regex>
    <transformationRules>
      <transformationRule>
        <apply>BEFORE</apply>
        <action>SET_TO_EMPTY_STRING</action>
      </transformationRule>
    </transformationRules>
</replacementProperty>

or

<replacementProperty>
    <property>sonar.projectKey</property>
    <token>.*</token>
    <value>${project.groupId}:${project.artifactId}:${git.branch}</value>
    <regex>false</regex>
    <fullOverwrite>true</fullOverwrite> # perhaps we can call it 'forceValueEvaluation' too
</replacementProperty>

To some extend I would claim this goes against the ideology to have the ability to run the project only once. Not sure what the best way would be to solve this...

Could you perhaps elaborate on your use-case why it would make sense to store the git-commit on the sonar.projectKey and not the sonar.projectVersion?

====
Commands to run sonar locally (in case I need them again)

docker run -d --name sonarqube -p 9000:9000 sonarqube
http://localhost:9000
(By default you can login as admin with password admin)

mvn clean verify pl.project13.maven:git-commit-id-plugin:revision sonar:sonar -Dsonar.verbose=true -X -Dsonar.log.level=DEBUG -Pgit-with-parent

or just
mvn clean verify pl.project13.maven:git-commit-id-plugin:revision sonar:sonar -Pgit-with-parent

[mmoossen]

Hi,

btw, many thanks for the great and fast support.

we are migrating an old Sonar 5 server to Sonar 8, and we had always had the GIt-Branch name in the sonar.branch property, which we were maintaining by hand. now we the new Sonar server that does not support the sonar.branch parameter any more.
In the Developer Edition of Sonar8 it is still possible to use some sonar.branch.xx Properties, but not in the Community Edition. the official recommended way to deal with it, it is as we are doing, composing the sonar.projektKey property.
we could still maintain it by hand, but i thought to give this plugin a try to reduce the manual maintenance costs.
btw, to use the Branch in the Version instead of the Key does not make any sense. since we want to track changes in the same long living branches and not in between different branches. for instance,
having release/v1 and release/v2 branches and versions version 1.2.3 and 2.3.4 respectively , you do not want to put the issues together, but rather to really treat them as separate projects.

many thanks for your help so far
Michael

[tjaronen]

We have exactly the same problem.
We would like to manipulate sonar.projectKey since we have separate analysis for each branch.
We are now maintaining projectKey manually in the project properties with values:
${project.groupId}:${project.artifactId}:master
${project.groupId}:${project.artifactId}:develop
etc....

Of course, there is a way around this by obtaining commercial version of Sonar which has branch analysis.

[TheSnoozer]

IMHO pointing to a 'commercial' version is the last thing I want to do.

The plugin has now a forceValueEvaluation-switch (locally) and produces for every sub-project a unique sonarProjectKey...however if it would end here, I would perhaps just create a MR and call it a day....this is unfortunately not the end, since sonar still refuses with :

[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project git-commit-id-plugin-debugging: Project 'pl.project13.maven:submodule-two:master' can't have 2 modules with the following key: pl.project13.maven:submodule-two:master -> [Help 1]

Essentially what this means is that I now need to clone sonar code and understand exactly what the plugin is doing 'wrong'...or what sonar does not like in the way the plugin operates....a big OOF.

May I also point to sonar's analysis-parameters that indicate at least two interesting ones: sonar.scm.revision and sonar.analysis.[yourKey] (e.g. sonar.analysis.branch). Are those properties available in the Community Edition? How exactly would you do the compare (sorry not a user of sonar)?

And apologies to post all the questions here, but I noticed that my docker image 'only' has the 7.9.1.27448 version and it's unclear what version it comes with...so I need to find time to dig into deeper into sonar code and install sonar myself....

(eta when I find time is on the weekend)

Edit:
likely the sonar code that throws the 'Exception'

Reading the available docker images (https://hub.docker.com/_/sonarqube/) for sonarqube helps:
docker run -d --name sonarqube -p 9000:9000 sonarqube
-> sonarqube 7.X

docker run -d --name sonarqube -p 9000:9000 sonarqube:8.0-community-beta
-> sonarqube 8.X-community-beta

Perhaps is https://github.com/mc1arke/sonarqube-community-branch-plugin what you need here??

[tjaronen]

Thank You really much for pointing out sonarqube-community-branch-plugin. I was not aware of that and it's exactly what we need. I installed it and our problem is now resolved.
I thought that the only way we can get branch analysis is to get Sonar Developer Edition and we don't want to do that. Previously we had separate independent projects for each branch in Sonar, but now we have one project with branches.
We have quite long living branches and we need to analyze each branch before we merge those to master.

By referencing to 'commercial' version I tried to be sarcastic.

[TheSnoozer]

Thanks again for the report. IMHO fiddling with sonar.projectKey on a per project basis seems a bit tricky and for now I think its a non-goal of the plugin to generate unique properties for each project (it somewhat would contradict the run-only-once terminology in muli-reactor builds).

As per sonar's analysis-parameters I would suggest to attempt to use sonar.scm.revision and sonar.analysis.[yourKey] (e.g. sonar.analysis.branch).
What is needed for the actual comparison within the community-edition is still quite unclear for me and perhaps only possible with the sonarqube-community-branch-plugin

So from the perspective of the plugin it is 'fixed' because there was indeed a display and generation issue.

[mmoossen]

sorry for the late reply.
many thanks for your help and pointing out that plugin.
wishing you nice and peaceful holidays

[mmoossen]

at the end, i am setting the branch name from the shell to sonars plugin:

mvn -P sonar -Dsonar.branch.name=$(git rev-parse --abbrev-ref HEAD) clean verify sonar:sonar

easy and robust, if you do not forget to set the parameter...

[TheSnoozer]

Hello,
thanks for the feedback! Its sad that the plugin can#t help you here, but I think your solution is honestly the best and the maven environment is somewhat not really supportive when it boils down to such requests....