extractVersion with postProcessRelease (maven) causes rejected release

[rarkins]

Discussed in #31886

^{Originally posted by remal October 10, 2024}

What would you like help with?

I would like help with my configuration

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

No response

Please tell us more about your question or problem

Miniumal reproduction repository: https://github.com/remal/renovate-reproduction-intellij-ideaIC-build-range-eap

Context

I'm developing an IntelliJ IDEA plugin and would like to configure Renovate to update pluginUntilBuild property in the gradle.properties file.

Chosen approach to configure Renovate

1. Configure a RegEx custom manager that processes the gradle.properties file.
2. Renovate is supposed to replace a value for the pluginUntilBuild property.
3. An updated value should be the last XXX-EAP-SNAPSHOT version of com.jetbrains.intellij.idea:ideaIC dependency in https://www.jetbrains.com/intellij-repository/snapshots/
4. XXX-EAP-SNAPSHOT version should be converted to XXX (see extractVersion)

Issue

The property is not updated. An unexpected message can be found in the logs: DEBUG: Rejected release

What's wrong? What would you suggest me to change?

Logs (if relevant)

Full logs: https://developer.mend.io/github/remal/renovate-reproduction-intellij-ideaIC-build-range-eap/-/job/d8b53513-063b-42cc-9d75-400b0665247c

Relevant logs

DEBUG: Looking up com.jetbrains.intellij.idea:ideaIC in repository https://www.jetbrains.com/intellij-repository/snapshots/
DEBUG: hostRules: no authentication for www.jetbrains.com
DEBUG: Using queue: host=www.jetbrains.com, concurrency=16
DEBUG: Found 38 new releases for com.jetbrains.intellij.idea:ideaIC in repository https://www.jetbrains.com/intellij-repository/snapshots/
DEBUG: GET https://www.jetbrains.com/intellij-repository/snapshots/com/jetbrains/intellij/idea/ideaIC/243.19420.21-EAP-SNAPSHOT/maven-metadata.xml = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=404 retryCount=0, duration=294)
DEBUG: Content is not found for Maven url
{
  "url": "https://www.jetbrains.com/intellij-repository/snapshots/com/jetbrains/intellij/idea/ideaIC/243.19420.21-EAP-SNAPSHOT/maven-metadata.xml"
}

DEBUG: Rejected release
{
  "datasource": "maven"
  "packageName": "com.jetbrains.intellij.idea:ideaIC"
  "registryUrl": "https://www.jetbrains.com/intellij-repository/snapshots"
  "version": "243"
}

DEBUG: PackageFiles.add() - Package file saved for base branch
{
  "baseBranch": "main"
}

DEBUG: Package releases lookups complete
{
  "baseBranch": "main"
}

DEBUG: branchifyUpgrades
DEBUG: detectSemanticCommits()
DEBUG: getCommitMessages
DEBUG: semanticCommits: detected "unknown"
DEBUG: semanticCommits: disabled
DEBUG: 0 flattened updates found: 
DEBUG: Returning 0 branch(es)
DEBUG: config.repoIsOnboarded=true
DEBUG: packageFiles with updates
{
  "baseBranch": "main"
  "config": {
    "regex": [
      {
        "deps": [
          {
            "depName": "intellij-ideaIC-build-range-eap",
            "packageName": "com.jetbrains.intellij.idea:ideaIC",
            "currentValue": "242",
            "datasource": "maven",
            "registryUrls": [
              "https://www.jetbrains.com/intellij-repository/snapshots/"
            ],
            "replaceString": "\nplatformType = IC\nplatformVersion = 2023.3.8\n\n# Supported build number ranges and IntelliJ Platform versions -> https://plugins.jetbrains.com/docs/intellij/build-number-ranges.html\n#pluginSinceBuild = XXX - loaded automatically from `platformVersion`\npluginUntilBuild = 242.*\n\n",
            "updates": [],
            "versioning": "maven",
            "warnings": [],
            "registryUrl": "https://www.jetbrains.com/intellij-repository/snapshots",
            "packageScope": "com.jetbrains.intellij.idea",
            "currentVersion": "242"
          }
        ],
        "matchStrings": [
          "(^|\\n)\\s*platformType\\s*=\\s*IC\\s*(\\n+|$)[\\s\\S]*?(^|\\n)\\s*pluginUntilBuild\\s*=\\s*(?<currentValue>\\d+)\\.\\*\\s*(\\n+|$)",
          "(^|\\n)\\s*pluginUntilBuild\\s*=\\s*(?<currentValue>\\d+)\\.\\*\\s*(\\n+|$)[\\s\\S]*?(^|\\n)\\s*platformType\\s*=\\s*IC\\s*(\\n+|$)"
        ],
        "depNameTemplate": "intellij-ideaIC-build-range-eap",
        "packageNameTemplate": "com.jetbrains.intellij.idea:ideaIC",
        "datasourceTemplate": "maven",
        "registryUrlTemplate": "https://www.jetbrains.com/intellij-repository/snapshots/",
        "packageFile": "gradle.properties"
      }
    ]
  }
}

[Shegox]

I created as well https://github.com/Shegox/spring-boot-major as a minimal reproduction repository.

Effectively the use case is tracking a major release of spring-boot in the spring-boot-major-file. So I use the maven package org.springframework.boot:spring-boot and use extractVersionTemplate in the regex manager to extract just the major:

{
  "customManagers": [
    {
      "customType": "regex",
      "description": "Regex manager to update the spring major version in `spring-boot-major`.",
      "fileMatch": [
        "spring-boot-major$"
      ],
      "matchStrings": [
        "^(?<currentValue>\\d+)"
      ],
      "datasourceTemplate": "maven",
      "depNameTemplate": "org.springframework.boot:spring-boot",
      "extractVersionTemplate": "^(?<version>\\d+)"
    }
  ]
}

The logs show then, that the postProcessing tries to lookup the extracted version (in this case just 3):

...
DEBUG: Rejected release
{
  "datasource": "maven"
  "packageName": "org.springframework.boot:spring-boot"
  "registryUrl": "https://repo.maven.apache.org/maven2"
  "version": "3"
}
...
DEBUG: packageFiles with updates
{
  "baseBranch": "main"
  "config": {
    "regex": [
      {
        "deps": [
          {
            "depName": "org.springframework.boot:spring-boot",
            "currentValue": "2",
            "datasource": "maven",
            "replaceString": "2",
            "updates": [],
            "packageName": "org.springframework.boot:spring-boot",
            "versioning": "maven",
            "warnings": [],
            "sourceUrl": "https://github.com/spring-projects/spring-boot",
            "registryUrl": "https://repo.maven.apache.org/maven2",
            "homepage": "https://spring.io/projects/spring-boot",
            "packageScope": "org.springframework.boot",
            "currentVersion": "2"
          }
        ],
        "matchStrings": [
          "^(?<currentValue>\\d+)"
        ],
        "depNameTemplate": "org.springframework.boot:spring-boot",
        "datasourceTemplate": "maven",
        "packageFile": "spring-boot-major"
      }
    ]
  }
}
...

My expectation/assumption would be that we need to lookup the non-extracted version (3.3.5) and perform the version extraction only after the maven post-processing.

[rarkins]

@zharinov can we save a field like rawDatasourceVersion and use it instead?

[Shegox]

I just saw it working in my test repository (Shegox/spring-boot-major#2). Looks like a fix got implemented? Maybe #32376

The logs now show this:

"renovateVersion": "39.9.5"
...
DEBUG: Rejected release combined with `extractVersion`: preserving the release
{
  "datasource": "maven"
  "packageName": "org.springframework.boot:spring-boot"
  "registryUrl": "https://repo.maven.apache.org/maven2"
  "version": "3"
  "extractVersion": "^(?<version>\\d+)"
}

Edit: Can confirm in my original/internal scenario it is working now again as well 👍 🚀

[rarkins]

Yes that was the fix/workaround

[renovate-release]

🎉 This issue has been resolved in version 39.18.1 🎉

The release is available on:

• GitHub release
• npm package (@latest dist-tag)
• 39.18.1

Your semantic-release bot 📦🚀