-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added opensearch.pluginzip plugin #31
Conversation
Signed-off-by: pgodithi <[email protected]>
Signed-off-by: pgodithi <[email protected]>
.github/workflows/CI.yml
Outdated
@@ -30,3 +30,4 @@ jobs: | |||
- name: Build and Run Tests | |||
run: | | |||
./gradlew build | |||
./gradlew help --task publishPluginZipPublicationToZipStagingRepository |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like we just need ./gradlew publishPluginZipPublicationToZipStagingRepository
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@reta, I wanted to check opinions here, yes even this ./gradlew publishPluginZipPublicationToZipStagingRepository
can be added, but do we need to, I see there are multiple custom plugins already added to the template, but we dont run a plugin specific task, just ./gradlew build
exists, do we even need to add publishPluginZipPublicationToZipStagingRepository
to test the CI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah I see, I think for the sake of checking that publishing works, we could publish to local repo: publishPluginZipPublicationToZipLocalRepository
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes we can add ./gradlew publishPluginZipPublicationToZipStagingRepository
, also moving forward its better we add some validation by running the custom plugin tasks, so that way we can some evidence that the custom plugins task actually work, I will add ./gradlew publishPluginZipPublicationToZipStagingRepository
to the CI.yaml.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
settings.gradle
Outdated
@@ -8,3 +8,4 @@ | |||
*/ | |||
|
|||
rootProject.name = 'plugin-template' | |||
startParameter.excludedTaskNames=["validatePluginZipPom"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, where this setting comes from?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@reta This is part of the plugin setup, usually all plugins managed build scripts or default build scripts run assemble
task and publish
tasks (specific to repo), example job-scheduler, but not direct build
task, if called we need to exclude this validatePluginZipPom
as general build
task will include this, if this task called it will fail with POM validation with the error
> Task :validatePluginZipPom FAILED
Execution optimizations have been disabled for task ':validatePluginZipPom' to ensure correctness due to the following reasons:
- Gradle detected a problem with the following location: '/usr/share/opensearch/git/opensearch-plugin-template-java/build/distributions/rename-unspecified.pom'. Reason: Task ':validatePluginZipPom' uses this output of task ':generatePomFileForNebulaPublication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.4.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
name is missing in [/usr/share/opensearch/git/opensearch-plugin-template-java/build/distributions/rename-unspecified.pom]
description is missing in [/usr/share/opensearch/git/opensearch-plugin-template-java/build/distributions/rename-unspecified.pom]
licenses is empty in [/usr/share/opensearch/git/opensearch-plugin-template-java/build/distributions/rename-unspecified.pom]
developers is empty in [/usr/share/opensearch/git/opensearch-plugin-template-java/build/distributions/rename-unspecified.pom]
Ideally we dont need to run this task as we define the POM values during actual publish.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just to add I have also added this point to the plugin usage document (PR under review)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This still looks wrong to me, shouldn't we be fixing the POM by adding these fields? Open a new issue?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It was fixed after adding
allprojects {
project.ext.licenseName = 'The Apache Software License, Version 2.0'
project.ext.licenseUrl = 'http://www.apache.org/licenses/LICENSE-2.0.txt'
publishing {
publications {
all {
pom.withXml { XmlProvider xml ->
Node node = xml.asNode()
node.appendNode('inceptionYear', '2022')
node.appendNode('name', 'rename')
node.appendNode('description', 'Custom plugin')
Node license = node.appendNode('licenses').appendNode('license')
license.appendNode('name', project.licenseName)
license.appendNode('url', project.licenseUrl)
Node developer = node.appendNode('developers').appendNode('developer')
developer.appendNode('name', 'OpenSearch')
developer.appendNode('url', 'https://github.com/opensearch-project/opensearch-plugin-template-java')
}
}
}
}
}
Then we dont need to exclude, I would say we can add the above template to the build.gradle
and it's up-to the plugin team, if they want to exclude the the task validatePluginZipPom
or add the POM fields in right way, I can update the info in the document as well, thoughts? @dblock
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
excluding is better, rather than forcing user to add these POM fields as then it allow user to run ./gradlew build
, without any restrictions, @dblock @reta @bbarani @peterzhuamazon
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think I understand. Plugins should all look the same and have all the right/same info in the POM. Why wouldn't this be in all plugins?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
some plugins do not have all of these settings @dblock, but in our case to publish zips we dont need this task validatePluginZipPom
.
This is coming from PomValidationTask (org.opensearch.gradle.precommit.PomValidationTask)
Signed-off-by: pgodithi <[email protected]>
Signed-off-by: pgodithi <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@prudhvigodithi so I pulled the pull request locally and it seems like more work is needed:
> Task :publishPluginZipPublicationToZipStagingRepository
Execution optimizations have been disabled for task ':publishPluginZipPublicationToZipStagingRepository' to ensure correctness due to the following reasons:
- Gradle detected a problem with the following location: 'opensearch-plugin-template-java/build/distributions/rename-unspecified.pom'. Reason: Task ':publishPluginZipPublicationToZipStagingRepository' uses this output of task ':generatePomFileForNebulaPublication' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.4.2/userguide/validation_problems.html#implicit_dependency for more details about this problem.
Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.
Those warnings should not be there, we need to find the way to properly address POM file exclusion or/and generation
Hey @reta, thanks for testing,
however as we dont need this task
Also I can see I have also tested this with job-scheduler and AD plugin, does not throw any errors, thoughts @reta ? |
Dont want to go with hacks, but looks like
We might to add this way https://github.com/opensearch-project/OpenSearch/blob/896b97e54df4e192a15994a1076a9d1d04312f74/modules/lang-painless/build.gradle#L80-L83 |
@prudhvigodithi looking, once sec |
I have tried this way @reta, still get that warning :) |
Signed-off-by: pgodithi <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM!
@reta re-review? |
Hey @dblock, since the fix was pushed yesterday, the plugin exists but I dont think the fixed code will get into |
@dblock @prudhvigodithi opened this one opensearch-project/OpenSearch#3252 so we could just do:
|
Hey @reta, thanks for the PR, but I see some existing plugins already use the syntax as of which I have created the PR example for job-scheduler, this is used for the jar's to get those POM fields, now with PR looks like the plugins team should add |
I think we are dealing with different publications here: for JAR and for ZIP, which essentially generate different POM files (at least, it should), but thanks for the hit, I will make sure we could use this model as well. |
I think @reta we can go with this PR for now and add this fix to both JAR and ZIP's as next new enhancement, thoughts ? @reta |
I am fine with merging it as-is, thanks @prudhvigodithi |
Signed-off-by: pgodithi <[email protected]>
Signed-off-by: pgodithi <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd be happy to merge on green against any version of OpenSearch.
build.gradle
Outdated
@@ -37,7 +60,7 @@ validateNebulaPom.enabled = false | |||
|
|||
buildscript { | |||
ext { | |||
opensearch_version = "2.0.0-SNAPSHOT" | |||
opensearch_version = "3.0.0-SNAPSHOT" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@prudhvigodithi this is right change in general, but for plugins we should target 2.0.0 at most - we expect most of the development happening here.
Signed-off-by: pgodithi <[email protected]>
Hey @dblock and @reta we have a |
Thanks @reta |
Signed-off-by: pgodithi [email protected]
Description
Add
opensearch.pluginzip
custom gradle plugin to the template.Issues Resolved
opensearch-project/opensearch-build#1916
opensearch-project/opensearch-plugins#144
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.