Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Artifacts path update at right location #376

Closed
wants to merge 3 commits into from

Conversation

vibrantvarun
Copy link
Member

@vibrantvarun vibrantvarun commented May 3, 2023

Description

The PR fixes the artifact publishing issue with job scheduler. I have added a new publication called mavenJava in the build .gradle which does packaging in Jar to the right publishing path.

Steps I followed to test:

  1. I have changed repositories path from sonatype URL to my local machine
  2. Added a new publishing end point mavenJava in build.gradle
  3. Ran below command
./gradlew opensearch-job-scheduler-spi:publishAllPublicationsToSnapshotsRepository && ./gradlew publishPluginZipPublicationToSnapshotsRepository && ./gradlew publishPluginJarToSnapshotsRepository

Please find the test output below.

.
└── org
    └── opensearch
        ├── opensearch-job-scheduler
        │   ├── 3.0.0.0-SNAPSHOT
        │   │   ├── maven-metadata.xml
        │   │   ├── maven-metadata.xml.md5
        │   │   ├── maven-metadata.xml.sha1
        │   │   ├── maven-metadata.xml.sha256
        │   │   ├── maven-metadata.xml.sha512
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-javadoc.jar
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-javadoc.jar.md5
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-javadoc.jar.sha1
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-javadoc.jar.sha256
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-javadoc.jar.sha512
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-sources.jar
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-sources.jar.md5
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-sources.jar.sha1
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-sources.jar.sha256
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1-sources.jar.sha512
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1.pom
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1.pom.md5
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1.pom.sha1
        │   │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203646-1.pom.sha256
        │   │   └── opensearch-job-scheduler-3.0.0.0-20230503.203646-1.pom.sha512
        │   ├── maven-metadata.xml
        │   ├── maven-metadata.xml.md5
        │   ├── maven-metadata.xml.sha1
        │   ├── maven-metadata.xml.sha256
        │   └── maven-metadata.xml.sha512
        ├── opensearch-job-scheduler-spi
        │   ├── 3.0.0.0-SNAPSHOT
        │   │   ├── maven-metadata.xml
        │   │   ├── maven-metadata.xml.md5
        │   │   ├── maven-metadata.xml.sha1
        │   │   ├── maven-metadata.xml.sha256
        │   │   ├── maven-metadata.xml.sha512
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-javadoc.jar
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-javadoc.jar.md5
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-javadoc.jar.sha1
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-javadoc.jar.sha256
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-javadoc.jar.sha512
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-sources.jar
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-sources.jar.md5
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-sources.jar.sha1
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-sources.jar.sha256
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1-sources.jar.sha512
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.jar
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.jar.md5
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.jar.sha1
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.jar.sha256
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.jar.sha512
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.pom
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.pom.md5
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.pom.sha1
        │   │   ├── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.pom.sha256
        │   │   └── opensearch-job-scheduler-spi-3.0.0.0-20230503.203639-1.pom.sha512
        │   ├── maven-metadata.xml
        │   ├── maven-metadata.xml.md5
        │   ├── maven-metadata.xml.sha1
        │   ├── maven-metadata.xml.sha256
        │   └── maven-metadata.xml.sha512
        └── plugin
            └── opensearch-job-scheduler
                ├── 3.0.0.0-SNAPSHOT
                │   ├── maven-metadata.xml
                │   ├── maven-metadata.xml.md5
                │   ├── maven-metadata.xml.sha1
                │   ├── maven-metadata.xml.sha256
                │   ├── maven-metadata.xml.sha512
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.pom
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.pom.md5
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.pom.sha1
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.pom.sha256
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.pom.sha512
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.zip
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.zip.md5
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.zip.sha1
                │   ├── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.zip.sha256
                │   └── opensearch-job-scheduler-3.0.0.0-20230503.203644-1.zip.sha512
                ├── maven-metadata.xml
                ├── maven-metadata.xml.md5
                ├── maven-metadata.xml.sha1
                ├── maven-metadata.xml.sha256
                └── maven-metadata.xml.sha512

Issues Resolved

#374

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Member

@cwperks cwperks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job figuring this out @vibrantvarun! I have checked out this branch and verified the changes locally.

build.gradle Outdated
}
repositories {
maven {
name = "Snapshots" // optional target repository name
url = "https://aws.oss.sonatype.org/content/repositories/snapshots"
url = "https://aws.oss.sonatype.org/content/repositories/snapshots/snapshots"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this line changed?

Copy link
Member Author

@vibrantvarun vibrantvarun May 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted it. Changed by mistake. Thanks for calling it out

developer.appendNode('url', 'https://github.com/opensearch-project/job-scheduler')
}
}
// all {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove the commented out lines?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took care off

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still seeing commented out lines in the PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 lm still seeing commented code.

build.gradle Outdated
@@ -136,11 +136,31 @@ publishing {
groupId = "org.opensearch.plugin"
}
}
mavenJava(MavenPublication){ publication ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: with the other publication called pluginZip, do you think we should call this pluginJar?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

@saratvemulapalli saratvemulapalli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good to me. Thanks @vibrantvarun .
Could you take a look at the CI failures?

@prudhvigodithi @gaiksaya how do we fix this for 2.7 published release after this change is merged?

developer.appendNode('url', 'https://github.com/opensearch-project/job-scheduler')
}
}
// all {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 lm still seeing commented code.

@gaiksaya
Copy link
Member

gaiksaya commented May 3, 2023

Hi @vibrantvarun,

Can you check if it is publishing as expected on maven local too?
The reason being this script is called https://github.com/opensearch-project/job-scheduler/blob/main/scripts/build.sh when we build JS into distribution. AFAIK the components are logged into build manifest as below:

Details

  - name: job-scheduler
    repository: https://github.com/opensearch-project/job-scheduler.git
    ref: e63a2e2ce9a55524bbc9eaef7c924cac7f3cbeea
    commit_id: e63a2e2ce9a55524bbc9eaef7c924cac7f3cbeea
    artifacts:
      maven:
        - maven/org/opensearch/plugin/opensearch-job-scheduler/maven-metadata.xml
        - maven/org/opensearch/plugin/opensearch-job-scheduler/maven-metadata.xml.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/maven-metadata.xml.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/maven-metadata.xml.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/maven-metadata.xml.sha512
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.jar
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.jar.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.jar.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.jar.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.jar.sha512
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.pom
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.pom.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.pom.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.pom.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.pom.sha512
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.module
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.module.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.module.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.module.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.module.sha512
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-javadoc.jar
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-javadoc.jar.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-javadoc.jar.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-javadoc.jar.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-javadoc.jar.sha512
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-sources.jar
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-sources.jar.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-sources.jar.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-sources.jar.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0-sources.jar.sha512
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.zip
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.zip.sha1
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.zip.md5
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.zip.sha256
        - maven/org/opensearch/plugin/opensearch-job-scheduler/2.7.0.0/opensearch-job-scheduler-2.7.0.0.zip.sha512
        - maven/org/opensearch/opensearch-job-scheduler-spi/maven-metadata.xml
        - maven/org/opensearch/opensearch-job-scheduler-spi/maven-metadata.xml.sha1
        - maven/org/opensearch/opensearch-job-scheduler-spi/maven-metadata.xml.md5
        - maven/org/opensearch/opensearch-job-scheduler-spi/maven-metadata.xml.sha256
        - maven/org/opensearch/opensearch-job-scheduler-spi/maven-metadata.xml.sha512
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.jar
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.jar.sha1
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.jar.md5
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.jar.sha256
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.jar.sha512
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.pom
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.pom.sha1
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.pom.md5
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.pom.sha256
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0.pom.sha512
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-sources.jar
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-sources.jar.sha1
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-sources.jar.md5
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-sources.jar.sha256
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-sources.jar.sha512
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-javadoc.jar
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-javadoc.jar.sha1
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-javadoc.jar.md5
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-javadoc.jar.sha256
        - maven/org/opensearch/opensearch-job-scheduler-spi/2.7.0.0/opensearch-job-scheduler-spi-2.7.0.0-javadoc.jar.sha512
      plugins:
        - plugins/opensearch-job-scheduler-2.7.0.0.zip
    version: 2.7.0.0

Which are then downloaded from S3 and published using maven cli. Workflow: https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch-maven-release/publish-to-maven.jenkinsfile#L39-L44

@saratvemulapalli @prudhvigodithi I believe we might need to document the process step by step of manual checking out the job scheduler repo at tag 2.7.0.0, cherry-pick this commit, get maven credentials and only run publish task for this URL: https://aws.oss.sonatype.org/content/repositories/releases/org/opensearch/opensearch-job-scheduler/

@vibrantvarun vibrantvarun deleted the artifact branch May 5, 2023 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants