Skip to content

Commit

Permalink
Add FindStagingRepository task to find staging repository by its desc…
Browse files Browse the repository at this point in the history
…ription (#201)

* Add FindStagingRepository task to find staging repository by its description

Fixes #19

* [#19] Adjust e2e tests for findStagingRepository

* [#19] Minor tweak in README

* [#19] Mark FindStagingRepository task as @Incubating

---------

Co-authored-by: Vladimir Sitnikov <[email protected]>
  • Loading branch information
szpak and vlsi authored Mar 3, 2023
1 parent 160fe01 commit a670b8d
Show file tree
Hide file tree
Showing 6 changed files with 306 additions and 5 deletions.
25 changes: 23 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,10 +118,31 @@ publishing {

Finally, call `publishToSonatype closeAndReleaseSonatypeStagingRepository` to publish all publications to Sonatype's OSSRH Nexus and subsequently close and release the corresponding staging repository, effectively making the artifacts available in Maven Central (usually after a few minutes).

Note that until [#19](https://github.com/gradle-nexus/publish-plugin/issues/19) is done, the `publishToSonatype closeAndReleaseSonatypeStagingRepository` tasks have to be executed in the same Gradle invocation because `closeAndRelease` relies on information that is not persisted between calls to Gradle. Failing to do so will result in an error like `No staging repository with name sonatype created`.

Please bear in mind that - especially on the initial project publishing to Maven Central - it might be wise to call just `publishToSonatype closeSonatypeStagingRepository` and manually verify that the artifacts placed in the closed staging repository in Nexus looks ok. After that, the staging repository might be dropped (if needed) or manually released from the Nexus UI.

#### Publishing and closing in different Gradle invocations

You might want to publish and close in different Gradle invocations. For example, you might want to publish from CI
and close and release from your local machine.
An alternative use case is to publish and close the repository and let others review and preview the publication before
the release.

The use case is possible by using `find${repository.name.capitalize()}StagingRepository` (e.g. `findSonatypeStagingRepository`) task.
By default, `initialize${repository.name.capitalize()}StagingRepository` task adds a description to the repository which defaults to
`$group:$module:$version` of the root project, so the repository can be found later using the same description.

The description can be customized via:
* `io.github.gradlenexus.publishplugin.NexusPublishExtension.getRepositoryDescription` property (default: `$group:$module:$version` of the root project)
* `io.github.gradlenexus.publishplugin.InitializeNexusStagingRepository.repositoryDescription` property
* `io.github.gradlenexus.publishplugin.FindStagingRepository.getDescriptionRegex` property (regex, default: `"\\b" + Regex.escape(repositoryDescription) + "(\\s|$)"`)

So the steps to publish and release in different Gradle invocations are:
1. Publish the artifacts to the staging repository: `./gradlew publishToSonatype`
2. Close the staging repository: `./gradlew findSonatypeStagingRepository closeSonatypeStagingRepository`
3. Release the staging repository: `./gradlew findSonatypeStagingRepository releaseSonatypeStagingRepository`

(in the above example, steps 1 and 2 could be also combined into `./gradlew publishToSonatype closeSonatypeStagingRepository`, to make only the releasing done in a separate step)

### Full example

#### Groovy DSL
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -889,6 +889,70 @@ class NexusPublishPluginTests {
)
}

@Test
fun `should find staging repository by description`() {
// given
writeDefaultSingleProjectConfiguration()
writeMockedSonatypeNexusPublishingConfiguration()
// and
val stagingRepository = StagingRepository(STAGED_REPOSITORY_ID, StagingRepository.State.OPEN, false)
val responseBody = getStagingReposWithOneStagingRepoWithGivenIdJsonResponseAsString(stagingRepository)
stubGetStagingReposForStagingProfileIdWithResponseStatusCodeAndResponseBody(STAGING_PROFILE_ID, 200, responseBody)

val result = run("findSonatypeStagingRepository")

assertSuccess(result, ":findSonatypeStagingRepository")
assertThat(result.output).containsPattern(Regex("Staging repository for .* '$STAGED_REPOSITORY_ID'").toPattern())
// and
assertGetStagingRepositoriesForStatingProfile(STAGING_PROFILE_ID)
}

@Test
fun `should not find staging repository by wrong description`() {
// given
writeDefaultSingleProjectConfiguration()
buildGradle.append("version='2.3.4-so staging repository is not found'")
writeMockedSonatypeNexusPublishingConfiguration()
// and
val stagingRepository = StagingRepository(STAGED_REPOSITORY_ID, StagingRepository.State.OPEN, false)
val responseBody = getStagingReposWithOneStagingRepoWithGivenIdJsonResponseAsString(stagingRepository)
stubGetStagingReposForStagingProfileIdWithResponseStatusCodeAndResponseBody(STAGING_PROFILE_ID, 200, responseBody)

val result = runAndFail("findSonatypeStagingRepository")

assertFailure(result, ":findSonatypeStagingRepository")
assertThat(result.output).contains("No staging repositories found for stagingProfileId: someProfileId, descriptionRegex: \\b\\Qorg.example:sample:2.3.4-so staging repository is not found\\E(\\s|\$). Here are all the repositories: [ReadStagingRepository(repositoryId=orgexample-42, type=open, transitioning=false, description=org.example:sample:0.0.1)]")
}

@Test
fun `should fail when multiple repositories exist`() {
// given
writeDefaultSingleProjectConfiguration()
writeMockedSonatypeNexusPublishingConfiguration()
// and
val stagingRepository = StagingRepository(STAGED_REPOSITORY_ID, StagingRepository.State.OPEN, false)
val stagingRepository2 = StagingRepository(OVERRIDDEN_STAGED_REPOSITORY_ID, StagingRepository.State.OPEN, false)
// Return two repositories with the same description, so the find call would get both, and it should fail
val responseBody = """
{
"data": [
${getOneStagingRepoWithGivenIdJsonResponseAsString(stagingRepository)},
${getOneStagingRepoWithGivenIdJsonResponseAsString(stagingRepository2)}
]
}
""".trimIndent()
stubGetStagingReposForStagingProfileIdWithResponseStatusCodeAndResponseBody(
STAGING_PROFILE_ID,
200,
responseBody
)

val result = runAndFail("findSonatypeStagingRepository")

assertFailure(result, ":findSonatypeStagingRepository")
assertThat(result.output).contains("Too many repositories found for stagingProfileId: someProfileId, descriptionRegex: \\b\\Qorg.example:sample:0.0.1\\E(\\s|\$). If some of the repositories are not needed, consider deleting them manually. Here are the repositories matching the regular expression: [ReadStagingRepository(repositoryId=orgexample-42, type=open, transitioning=false, description=org.example:sample:0.0.1), ReadStagingRepository(repositoryId=orgexample-42o, type=open, transitioning=false, description=org.example:sample:0.0.1)]")
}

// TODO: To be used also in other tests
private fun writeDefaultSingleProjectConfiguration() {
projectDir.resolve("settings.gradle").write(
Expand Down Expand Up @@ -1062,6 +1126,23 @@ class NexusPublishPluginTests {
)
}

private fun stubGetStagingReposForStagingProfileIdWithResponseStatusCodeAndResponseBody(
stagingProfileId: String,
statusCode: Int,
responseBody: String
) {
server.stubFor(
get(urlEqualTo("/staging/profile_repositories/$stagingProfileId"))
.withHeader("Accept", containing("application/json"))
.willReturn(
aResponse()
.withStatus(statusCode)
.withHeader("Content-Type", "application/json")
.withBody(responseBody)
)
)
}

private fun expectArtifactUploads(prefix: String, wireMockServer: WireMockServer = server) {
wireMockServer.stubFor(
put(urlMatching("$prefix/.+"))
Expand Down Expand Up @@ -1127,6 +1208,10 @@ class NexusPublishPluginTests {
server.verify(count, getRequestedFor(urlMatching("/staging/repository/$stagingRepositoryId")))
}

private fun assertGetStagingRepositoriesForStatingProfile(stagingProfileId: String = STAGING_PROFILE_ID, count: Int = 1) {
server.verify(count, getRequestedFor(urlMatching("/staging/profile_repositories/$stagingProfileId")))
}

private fun getOneStagingProfileWithGivenIdShrunkJsonResponseAsString(stagingProfileId: String): String {
return """
{
Expand All @@ -1148,6 +1233,19 @@ class NexusPublishPluginTests {
""".trimIndent()
}

private fun getStagingReposWithOneStagingRepoWithGivenIdJsonResponseAsString(
stagingRepository: StagingRepository,
stagingProfileId: String = STAGING_PROFILE_ID
): String {
return """
{
"data": [
${getOneStagingRepoWithGivenIdJsonResponseAsString(stagingRepository, stagingProfileId)}
]
}
""".trimIndent()
}

private fun getOneStagingRepoWithGivenIdJsonResponseAsString(
stagingRepository: StagingRepository,
stagingProfileId: String = STAGING_PROFILE_ID
Expand All @@ -1170,7 +1268,7 @@ class NexusPublishPluginTests {
"updated": "2020-01-28T10:23:49.616Z",
"updatedDate": "Tue Jan 28 10:23:49 UTC 2020",
"updatedTimestamp": 1580207029616,
"description": "Closed by io.github.gradle-nexus.publish-plugin Gradle plugin",
"description": "org.example:sample:0.0.1",
"provider": "maven2",
"releaseRepositoryId": "no-sync-releases",
"releaseRepositoryName": "No Sync Releases",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ import io.github.gradlenexus.publishplugin.BaseGradleTest
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.ValueSource
import java.io.File
import java.text.SimpleDateFormat
import java.util.*

@Suppress("FunctionName")
class NexusPublishE2ETests : BaseGradleTest() {
Expand Down Expand Up @@ -47,4 +49,50 @@ class NexusPublishE2ETests : BaseGradleTest() {
assertSuccess(":releaseSonatypeStagingRepository")
}
}

@ParameterizedTest(name = "{0}")
// nexus-publish-e2e-multi-project disabled due to: https://github.com/gradle-nexus/publish-plugin/issues/200
@ValueSource(strings = ["nexus-publish-e2e-minimal"/*, "nexus-publish-e2e-multi-project"*/])
fun `release project to real Sonatype Nexus in two executions`(projectName: String) {
File("src/e2eTest/resources/$projectName").copyRecursively(projectDir)
// Even though published e2e package is effectively dropped, Sonatype Nexus rules requires unique versions - https://issues.sonatype.org/browse/OSSRH-86532
// On the other hand, findSonatypeStagingRepository mechanism assumes constant project version across calls to find proper repository
val uniqueVersion = "0.0.2-unique-${SimpleDateFormat("yyyyMMdd-HHmmss").format(Date())}"

// when
val buildResult = run(
"build",
"-PoverriddenVersion=$uniqueVersion"
)
// then
buildResult.assertSuccess { it.path.substringAfterLast(':').matches("build".toRegex()) }

// when
// Publish artifacts to staging repository, close it == prepare artifacts for the review
val result = run(
"publishToSonatype",
"closeSonatypeStagingRepository",
"--info",
"-PoverriddenVersion=$uniqueVersion"
)

result.apply {
assertSuccess { it.path.substringAfterLast(':').matches("publish.+PublicationToSonatypeRepository".toRegex()) }
assertSuccess(":closeSonatypeStagingRepository")
}

// Release artifacts after the review
val closeResult = run(
"findSonatypeStagingRepository",
"releaseSonatypeStagingRepository",
"--info",
"-PoverriddenVersion=$uniqueVersion"
)

// then
closeResult.apply {
assertSuccess(":findSonatypeStagingRepository")
assertSuccess(":releaseSonatypeStagingRepository")
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/*
* Copyright 2021 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package io.github.gradlenexus.publishplugin

import io.github.gradlenexus.publishplugin.internal.NexusClient
import io.github.gradlenexus.publishplugin.internal.StagingRepositoryDescriptorRegistry
import org.gradle.api.GradleException
import org.gradle.api.Incubating
import org.gradle.api.model.ObjectFactory
import org.gradle.api.provider.Provider
import org.gradle.api.tasks.Input
import org.gradle.api.tasks.Internal
import org.gradle.api.tasks.Optional
import org.gradle.api.tasks.TaskAction
import org.gradle.kotlin.dsl.property
import javax.inject.Inject

@Incubating
open class FindStagingRepository @Inject constructor(
objects: ObjectFactory,
extension: NexusPublishExtension,
repository: NexusRepository,
private val registry: Provider<StagingRepositoryDescriptorRegistry>
) : AbstractNexusStagingRepositoryTask(objects, extension, repository) {

@Optional
@Input
val packageGroup = objects.property<String>().apply {
set(extension.packageGroup)
}

@Input
val descriptionRegex = objects.property<String>().apply {
set(extension.repositoryDescription.map { "\\b" + Regex.escape(it) + "(\\s|$)" })
}

@Internal
val stagingRepositoryId = objects.property<String>()

init {
outputs.cacheIf("the task requests data from the external repository, so we don't want to cache it") {
false
}
}

@TaskAction
fun findStagingRepository() {
val repository = repository.get()
val serverUrl = repository.nexusUrl.get()
val client = NexusClient(serverUrl, repository.username.orNull, repository.password.orNull, clientTimeout.orNull, connectTimeout.orNull)
val stagingProfileId = determineStagingProfileId(repository, client)
logger.info("Fetching staging repositories for {} at {}, stagingProfileId '{}'", repository.name, serverUrl, stagingProfileId)
val descriptionRegex = descriptionRegex.get()
val descriptor = client.findStagingRepository(stagingProfileId, Regex(descriptionRegex))
logger.lifecycle("Staging repository for {} at {}, stagingProfileId '{}', descriptionRegex '{}' is '{}'", repository.name, serverUrl, stagingProfileId, descriptionRegex, descriptor.stagingRepositoryId)
stagingRepositoryId.set(descriptor.stagingRepositoryId)
registry.get()[repository.name] = descriptor
}

// TODO: Duplication with InitializeNexusStagingRepository
private fun determineStagingProfileId(repository: NexusRepository, client: NexusClient): String {
var stagingProfileId = repository.stagingProfileId.orNull
if (stagingProfileId == null) {
val packageGroup = packageGroup.get()
logger.info("No stagingProfileId set, querying for packageGroup '{}'", packageGroup)
stagingProfileId = client.findStagingProfileId(packageGroup)
?: throw GradleException("Failed to find staging profile for package group: $packageGroup")
}
return stagingProfileId
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,16 @@ class NexusPublishPlugin : Plugin<Project> {
repository,
registry
)
val findStagingRepository = rootProject.tasks.register<FindStagingRepository>(
"find${capitalizedName}StagingRepository",
rootProject.objects,
extension,
repository,
registry
)
findStagingRepository {
description = "Finds the staging repository for ${repository.name}"
}
val closeTask = rootProject.tasks.register<CloseNexusStagingRepository>(
"close${capitalizedName}StagingRepository",
rootProject.objects,
Expand All @@ -100,11 +110,13 @@ class NexusPublishPlugin : Plugin<Project> {
description = "Closes open staging repository in '${repository.name}' Nexus instance."
group = PublishingPlugin.PUBLISH_TASK_GROUP
mustRunAfter(initializeTask)
mustRunAfter(findStagingRepository)
}
releaseTask {
description = "Releases closed staging repository in '${repository.name}' Nexus instance."
group = PublishingPlugin.PUBLISH_TASK_GROUP
mustRunAfter(initializeTask)
mustRunAfter(findStagingRepository)
mustRunAfter(closeTask)
}
closeAndReleaseTask {
Expand All @@ -117,6 +129,9 @@ class NexusPublishPlugin : Plugin<Project> {
rootProject.tasks.named("initialize${capitalizedName}StagingRepository").configure {
enabled = false
}
rootProject.tasks.named("find${capitalizedName}StagingRepository").configure {
enabled = false
}
rootProject.tasks.named("close${capitalizedName}StagingRepository").configure {
enabled = false
}
Expand All @@ -142,6 +157,7 @@ class NexusPublishPlugin : Plugin<Project> {
val nexusRepositories = addMavenRepositories(publishingProject, extension, registry)
nexusRepositories.forEach { (nexusRepo, mavenRepo) ->
val initializeTask = rootProject.tasks.named<InitializeNexusStagingRepository>("initialize${nexusRepo.capitalizedName}StagingRepository")
val findStagingRepositoryTask = rootProject.tasks.named<FindStagingRepository>("find${nexusRepo.capitalizedName}StagingRepository")
val closeTask = rootProject.tasks.named<CloseNexusStagingRepository>("close${nexusRepo.capitalizedName}StagingRepository")
val releaseTask = rootProject.tasks.named<ReleaseNexusStagingRepository>("release${nexusRepo.capitalizedName}StagingRepository")
val publishAllTask = publishingProject.tasks.register("publishTo${nexusRepo.capitalizedName}") {
Expand All @@ -154,7 +170,7 @@ class NexusPublishPlugin : Plugin<Project> {
releaseTask {
mustRunAfter(publishAllTask)
}
configureTaskDependencies(publishingProject, initializeTask, publishAllTask, closeTask, releaseTask, mavenRepo)
configureTaskDependencies(publishingProject, initializeTask, findStagingRepositoryTask, publishAllTask, closeTask, releaseTask, mavenRepo)
}
}
}
Expand Down Expand Up @@ -190,6 +206,7 @@ class NexusPublishPlugin : Plugin<Project> {
private fun configureTaskDependencies(
project: Project,
initializeTask: TaskProvider<InitializeNexusStagingRepository>,
findStagingRepositoryTask: TaskProvider<FindStagingRepository>,
publishAllTask: TaskProvider<Task>,
closeTask: TaskProvider<CloseNexusStagingRepository>,
releaseTask: TaskProvider<ReleaseNexusStagingRepository>,
Expand All @@ -203,6 +220,7 @@ class NexusPublishPlugin : Plugin<Project> {
)
publishTask {
dependsOn(initializeTask)
mustRunAfter(findStagingRepositoryTask)
doFirst { logger.info("Uploading to {}", repository.url) }
}
publishAllTask {
Expand Down
Loading

0 comments on commit a670b8d

Please sign in to comment.