Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add details for gradle custom plugin. #144

Merged
merged 16 commits into from
May 10, 2022
100 changes: 100 additions & 0 deletions BUILDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
- [Always Build SNAPSHOT Artifacts](#always-build-snapshot-artifacts)
- [Consume Maven Artifacts in Order](#consume-maven-artifacts-in-order)
- [Include Checksums in Maven Publications](#include-checksums-in-maven-publications)
- [Custom Gradle Plugins](#custom-gradle-plugins)
- [opensearch.pluginzip](#opensearchpluginzip)
- [Plugin Design](#plugin-design)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would get rid of sub-sections and make it one entry (opensearch.pluginzip).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would it not be easy for a user to read sub-sections rather than a bit fat single section about the plugin? :)

- [Plugin Usage](#plugin-usage)

## Developing Plugins for OpenSearch

Expand Down Expand Up @@ -162,3 +166,99 @@ publishing {
...
```
Use `./gradlew publishShadowPublicationToStagingRepository` to produce maven artifacts. When building as part of the monolithic distribution customize `scripts/build.sh` to collect maven artifacts. See [job-scheduler#71](https://github.com/opensearch-project/job-scheduler/pull/82/files) for an example.

## Custom Gradle Plugins

Automations, Testing other OpenSearch funtionalities can be acheived using Custom Gradle Plugins, there are several custom gradle plugins developed depending on desired use case. The custom gradle plugins are added in [OpenSearch](https://github.com/opensearch-project/OpenSearch/tree/main/buildSrc) repo.
prudhvigodithi marked this conversation as resolved.
Show resolved Hide resolved

### opensearch.pluginzip

This plugin will identify the generated zip file distribution which is the ouput of `bundlePlugin` task and publish to local maven repo with right maven coordinates.
prudhvigodithi marked this conversation as resolved.
Show resolved Hide resolved
[Plugin Code](https://github.com/opensearch-project/OpenSearch/tree/main/buildSrc/src/main/java/org/opensearch/gradle/pluginzip), [Plugin Tests](https://github.com/opensearch-project/OpenSearch/tree/main/buildSrc/src/test/java/org/opensearch/gradle/pluginzip), [Plugin META-INF](https://github.com/opensearch-project/OpenSearch/blob/main/buildSrc/src/main/resources/META-INF/gradle-plugins/opensearch.pluginzip.properties)


#### Plugin Design
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove this title.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wont it be easy to check the plugin code location, if a user wants to know more details about this ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation for the plugin belongs with the plugin in its code. You can add these things as comments in the plugin itself. This doc is just about how to use it.


* `opensearch.pluginzip` plugin is java based code added to existing build-tools framework.
* This plugin requires build-tools framework as dependancy.
prudhvigodithi marked this conversation as resolved.
Show resolved Hide resolved
* The maven coordinates groupID is fixed as `org.openserach.plugin`, `version` and `artifcatID` will be inferred from gradle project properties.
* User can also pass custom POM extensions, that will add desired xml to the zip maven POM file geneaterd during runtime.
* Once the plugin is added to the `build.gradle` as `apply plugin: 'opensearch.pluginzip'`, this will add a new custom publish task `publishPluginZipPublicationToZipStagingRepository`, this task will publish the zip distribution to the maven local (file system).
* The maven local artifacts can then be published to maven central/nexus.
* The plugin will not add jars generated by tasks `sourcesJar` and `javadocJar`, this is done in purpose to exclude `jars` for `zip` publications.

#### Plugin Usage

* The Key requirement for this plugin to work is that the zip should be generated by `bundlePlugin` task that comes from existing [PublishPlugin](https://github.com/opensearch-project/OpenSearch/blob/main/buildSrc/src/main/java/org/opensearch/gradle/PublishPlugin.java).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make the steps numbered, 1.2.3

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dblock so instead of * you suggest add number ?

Copy link
Member

@dblock dblock May 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not an enumeration, I'd just write the text as is without any *.

`./gradlew tasks ` should list `bundlePlugin`.

* Add the build-script dependency to get build-tools framework to the current gradle project. (If already exists, dont need to re-add again)
```
buildscript {
ext {
opensearch_version = System.getProperty("opensearch.version", "2.0.0-rc1-SNAPSHOT")
}
repositories {
mavenLocal()
}
dependencies {
classpath "org.opensearch.gradle:build-tools:${opensearch_version}"
}
}
```

* Add `apply plugin: 'opensearch.pluginzip'` to the build.gradle file.
Once added, this should list the new task `publishPluginZipPublicationToZipStagingRepository`.

* Run the task `publishPluginZipPublicationToZipStagingRepository` (add it to managed build script build.sh)
```./gradlew publishPluginZipPublicationToZipStagingRepository -Dopensearch.version=$VERSION -Dbuild.snapshot=$SNAPSHOT -Dbuild.version_qualifier=$QUALIFIER```

Note: The gradle assemble task `./gradlew assemble` should be called first before calling `publishPluginZipPublicationToZipStagingRepository`, as zip file need to be generated first before publishing.

* To add custom POM extensions:
Example:
```
allprojects {
project.ext.licenseName = 'The Apache Software License, Version 2.0'
project.ext.licenseUrl = 'http://www.apache.org/licenses/LICENSE-2.0.txt'
publishing {
repositories {
maven {
name = 'staging'
url = "${rootProject.buildDir}/local-staging-repo"
}
}
publications {
all {
pom.withXml { XmlProvider xml ->
Node node = xml.asNode()
node.appendNode('inceptionYear', '2021')

Node license = node.appendNode('licenses').appendNode('license')
license.appendNode('name', project.licenseName)
license.appendNode('url', project.licenseUrl)

Node developer = node.appendNode('developers').appendNode('developer')
developer.appendNode('name', 'OpenSearch')
developer.appendNode('url', 'https://github.com/opensearch-project/security')
}
}
}
}
}
```
* To run `build` task using `./gradlew build` for the entire project instead of running specific tasks, add the line ```startParameter.excludedTaskNames=["validatePluginZipPom"]``` in `settings.gradle` file, this task `validatePluginZipPom` is not required for this plugin.

Note: If gradle project is executed by specific task, then its not required to add ```startParameter.excludedTaskNames=["validatePluginZipPom"]```, only required when ran `./gradlew build`.

* Exclude the following tasks in `settings.gradle` file, if the build script exists for the plugin and has the tasks `publishToMavenLocal` and `publishAllPublicationsToStagingRepository`, these will also include the tasks `publishPluginZipPublicationToMavenLocal` and `publishPluginZipPublicationToStagingRepository` which is not required to be called with this plugin.
To exclude add the following in `settings.gradle` file ```startParameter.excludedTaskNames=["publishPluginZipPublicationToMavenLocal", "publishPluginZipPublicationToStagingRepository"]```

Note: If there is a managed `build.sh` file and do not have any publish tasks, then its not required to exlcude these tasks, only required if it is calling publish tasks that targets ALL repos and ALL publications.

* Quick Example:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Point to a PR instead of lines of code, these links will stop working when those files are edited for whatever reason.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dblock the plugins have not yet started to use to this gradle plugin, so we dont yet have a PR, once this document is merged and I have a PR as an example, then I can update the doc back again.

Job-scheduler:
[build.gradle](https://github.com/prudhvigodithi/job-scheduler/blob/gradleplugin/build.gradle#L33)
[settings.gradle](https://github.com/prudhvigodithi/job-scheduler/blob/gradleplugin/settings.gradle#L13)
[build.sh](https://github.com/prudhvigodithi/job-scheduler/blob/gradleplugin/scripts/build.sh#L80)

5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
- [Building Plugins with OpenSearch](#building-plugins-with-opensearch)
- [Upgrading Plugins to work with OpenSearch](#upgrading-plugins-to-work-with-opensearch)
- [Installing Plugins](#installing-plugins)
- [Gradle custom plugins](#gradle-custom-plugins)
- [Developing Plugins for OpenSearch](#developing-plugins-for-opensearch)
- [Workflows](#workflows)
- [Plugin Release Notes](#plugin-release-notes)
Expand Down Expand Up @@ -41,6 +42,10 @@ To upgrade your existing plugins to work with OpenSearch, see [UPGRADING](./UPGR

See [INSTALLING](INSTALLING.md) for details.

### Gradle custom plugins
prudhvigodithi marked this conversation as resolved.
Show resolved Hide resolved

See [Custom Gradle Plugins](/BUILDING.md#custom-gradle-plugins) for details.

### Developing Plugins for OpenSearch

See [DEVELOPING_PLUGINS](BUILDING.md#developing-new-plugins-for-opensearch) for details.
Expand Down