Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jenkins workflows to release -SNAPSHOT builds and to stage releases in maven #187

Merged
merged 3 commits into from
Jul 28, 2022
Merged

Jenkins workflows to release -SNAPSHOT builds and to stage releases in maven #187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a42e97b
Publish snapshot to maven.
dblock Jul 28, 2022
144b722
Release to maven.
dblock Jul 28, 2022
4d9e13c
Updated credentials and signing.
dblock Jul 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions java-client/build.gradle.kts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,8 +248,15 @@ tasks.withType<Jar> {

publishing {
repositories{
maven {
url = uri("${rootProject.buildDir}/repository")
if (version.toString().endsWith("SNAPSHOT")) {
maven("https://aws.oss.sonatype.org/content/repositories/snapshots/") {
name = "snapshotRepo"
credentials(PasswordCredentials::class)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my learning, how are these creds passed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above, this asks for a specific environment variable that gets set by the wrapper.

}
} else {
maven("${rootProject.buildDir}/repository") {
name = "localRepo"
}
}
}
publications {
Expand Down
25 changes: 25 additions & 0 deletions jenkins/publish-snapshot.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
pipeline {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the Jenkins fleet pick this job/pipeline ?
Is there a corresponding file in opensearch-ci?

Also I see for every component, jenkins files are defined in opensearch-build. Why is this different?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaiksaya @peterzhuamazon ?

i think the job needs to be added manually once, but I could be wrong (I hope so)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could move these to OpenSearch-build, I have no strong preference, I was replacing a file that was already here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah for now we to manually added files, for releasing opensearch-java it already exists. @saratvemulapalli we are planning to move respective jenkins file for clients in their repo. Would be easier to manage. However, need to add regression tests for all jenkins file to see the behavior is as expected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks that helps :).
One concern having these files in individual repositories is that the responsibility is delegated to the maintainers of the individual projects and make sure the infrastructure is not mis-used.

Copy link
Member

@gaiksaya gaiksaya Jul 28, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the maintenance of these files will be delegated to respective maintainers of the project rather than @opensearch-project/engineering-effectiveness being the bottle neck for it. Regarding the mis-use of the infrastructure, we have to be careful what triggers the workflows. For example, if a commit push is gonna trigger the workflow no matter where the workflow lies it will be executed.

agent {
docker {
label 'Jenkins-Agent-AL2-X64-C54xlarge-Docker-Host'
image 'opensearchstaging/ci-runner:ci-runner-centos7-opensearch-build-v2'
args '-e JAVA_HOME=/opt/java/openjdk-11'
alwaysPull true
}
}
stages {
stage('Publish to Sonatype Snapshots Repo') {
steps {
git url: 'https://github.com/opensearch-project/opensearch-java.git', branch: 'main'
withCredentials([usernamePassword(credentialsId: 'jenkins-sonatype-creds', usernameVariable: 'ORG_GRADLE_PROJECT_snapshotsUsername', passwordVariable: 'ORG_GRADLE_PROJECT_snapshotsPassword')]) {
sh './gradlew --no-daemon publishPublishMavenPublicationToSnapshotRepoRepository'
}
}
post {
always {
cleanWs disableDeferredWipeout: true, deleteDirs: true
}
}
}
}
}
85 changes: 85 additions & 0 deletions jenkins/stage-maven-release.jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
lib = library(identifier: 'jenkins@main', retriever: modernSCM([
$class: 'GitSCMSource',
remote: 'https://github.com/opensearch-project/opensearch-build.git',
]))

pipeline {
gaiksaya marked this conversation as resolved.
Show resolved Hide resolved
agent {
docker {
label 'Jenkins-Agent-AL2-X64-C54xlarge-Docker-Host'
image 'opensearchstaging/ci-runner:ci-runner-centos7-opensearch-build-v2'
args '-e JAVA_HOME=/opt/java/openjdk-11'
alwaysPull true
}
}
environment {
VERSION = "${params.VERSION}"
ARTIFACT_PATH = "$WORKSPACE/build/repository/org/opensearch/client/opensearch-java/${VERSION}"
}
stages {
stage('parameters') {
steps {
script {
properties([
parameters([
string(
name: 'REF',
trim: true
),
string(
name: 'VERSION',
trim: true
)
])
])
if (params.REF.isEmpty() || params.VERSION.isEmpty()) {
currentBuild.result = 'ABORTED'
error('Missing REF and/or VERSION.')
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you want to add triggers now or I can create a follow up PR. I have the code ready. Needs few PRs to add creds to jenkins infrastructure.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do a followup.

}
stage('Publish to Maven Local') {
steps {
// checkout the commit
git url: 'https://github.com/opensearch-project/opensearch-java.git', branch: 'main'
sh('git checkout ${REF}')
Comment on lines +44 to +46
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can be

                checkout([$class: 'GitSCM', userRemoteConfigs: [[url: 'https://github.com/gaiksaya/opensearch-java.git']],
                        branches: [[name: "$ref"]]])


// publish maven artifacts
sh('./gradlew --no-daemon publishPublishMavenPublicationToLocalRepoRepository')
}
}
stage('Sign') {
gaiksaya marked this conversation as resolved.
Show resolved Hide resolved
steps {
script {
signArtifacts(
artifactPath: "${ARTIFACT_PATH}",
type: 'maven',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad. Please do type: 'maven',

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. That's what I figured.

platform: 'linux'
)
}
}
}
stage('Stage Maven Artifacts') {
environment {
REPO_URL = 'https://aws.oss.sonatype.org/'
STAGING_PROFILE_ID = "${SONATYPE_STAGING_PROFILE_ID}"
BUILD_ID = "${BUILD_NUMBER}"
}
steps {
// checkout the build repo
git url: 'https://github.com/opensearch-project/opensearch-build.git', branch: 'main'

// stage artifacts for release with Sonatype
withCredentials([usernamePassword(credentialsId: 'jenkins-sonatype-creds', usernameVariable: 'SONATYPE_USERNAME', passwordVariable: 'SONATYPE_PASSWORD')]) {
sh('$WORKSPACE/publish/stage-maven-release.sh $ARTIFACT_PATH')
}
}
}
}
post {
always {
cleanWs disableDeferredWipeout: true, deleteDirs: true
}
}
}
138 changes: 0 additions & 138 deletions jenkins/stage-maven-release/JenkinsFile
View file
Open in desktop

This file was deleted.