Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration test infrastructure set up #230

Merged
merged 4 commits into from
Nov 30, 2023
Merged

Integration test infrastructure set up #230

merged 4 commits into from
Nov 30, 2023

Conversation

joshpalis
Copy link
Member

Description

This PR sets up the initial integration test cluster infrastructure. We iterate through all the zipArchive dependencies and install them into the cluster prior to installing the flow framework plugin. As we add support for additional plugin APIs, to add them to the test cluster, all we need to do is to add the related zipArchive under dependencies.

Additionally this PR fixes our ./gradlew run task which uses the integration test cluster. Now when you ./gradlew run both ML-commons and Flow Framework will be automatically installed into a local cluster

Issues Resolved

Part of #88

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@joshpalis joshpalis added the backport 2.x backport PRs to 2.x branch label Nov 30, 2023
@joshpalis
spotless
23744ae 
Signed-off-by: Joshua Palis <[email protected]>
@amitgalitz
Copy link
Member

Thanks for adding this! Could you change our DEVELOPER_GUIDE.md for the ./gradlew :run section to mention it also includes flow-framework and ml-commons installed

amitgalitz
amitgalitz approved these changes Nov 30, 2023
View reviewed changes
Copy link
Member

@amitgalitz amitgalitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! just one minor comment to update the developer guide

@joshpalis
Updating DEVELOPER_GUIDE
f7448a9 
Signed-off-by: Joshua Palis <[email protected]>
@joshpalis
Copy link
Member Author

@owaiskazi19 I'm opting not to backport this to the feature/agent_framework branch since these dev branches are not published, therefore we cannot pull them in as zipArchive dependencies to install into our test clusters.

@ohltyler
Copy link
Member

@owaiskazi19 I'm opting not to backport this to the feature/agent_framework branch since these dev branches are not published, therefore we cannot pull them in as zipArchive dependencies to install into our test clusters.

+1, there is no clean way to pull and build custom zips based on branches without extra boilerplate and workarounds that would need to be inevitably reverted.

@joshpalis joshpalis merged commit aba7dea into opensearch-project:main Nov 30, 2023
19 checks passed
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Nov 30, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 30, 2023
@github-actions
Integration test infrastructure set up (#230)
8fc719c 
* Initial integ test framework modification, sets up integration test cluster and fixes ./gradlew run

Signed-off-by: Joshua Palis <[email protected]>

* spotless

Signed-off-by: Joshua Palis <[email protected]>

* Updating DEVELOPER_GUIDE

Signed-off-by: Joshua Palis <[email protected]>

---------

Signed-off-by: Joshua Palis <[email protected]>
(cherry picked from commit aba7dea)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
joshpalis pushed a commit that referenced this pull request Nov 30, 2023
@opensearch-trigger-bot @github-actions
[Backport 2.x] Integration test infrastructure set up (#232)
c4e8d50 
Integration test infrastructure set up (#230)

* Initial integ test framework modification, sets up integration test cluster and fixes ./gradlew run



* spotless



* Updating DEVELOPER_GUIDE



---------


(cherry picked from commit aba7dea)

Signed-off-by: Joshua Palis <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@joshpalis joshpalis deleted the integ branch December 4, 2023 18:08
dbwiddis pushed a commit to dbwiddis/flow-framework that referenced this pull request Dec 11, 2023
@joshpalis @dbwiddis
Integration test infrastructure set up (opensearch-project#230)
5c2726b 
* Initial integ test framework modification, sets up integration test cluster and fixes ./gradlew run

Signed-off-by: Joshua Palis <[email protected]>

* spotless

Signed-off-by: Joshua Palis <[email protected]>

* Updating DEVELOPER_GUIDE

Signed-off-by: Joshua Palis <[email protected]>

---------

Signed-off-by: Joshua Palis <[email protected]>
dbwiddis added a commit that referenced this pull request Dec 12, 2023
@dbwiddis @owaiskazi19
@mend-for-github.aaakk.us.kg @reta @joshpalis
Add setting to limit max workflow steps (#266)
7bec6e8 
* Rename repo name in code files and CI (#182)

Signed-off-by: owaiskazi19 <[email protected]>

* Update actions/setup-java action to v4 (#219)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Use only pluginZip publication of Apache Maven artifacts (#226)

* Use only pluginZip publication of Apache Maven artifacts

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>

* Integration test infrastructure set up (#230)

* Initial integ test framework modification, sets up integration test cluster and fixes ./gradlew run

Signed-off-by: Joshua Palis <[email protected]>

* spotless

Signed-off-by: Joshua Palis <[email protected]>

* Updating DEVELOPER_GUIDE

Signed-off-by: Joshua Palis <[email protected]>

---------

Signed-off-by: Joshua Palis <[email protected]>

* Update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.23.2 (#229)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Update to Gradle 8.5 (#227)

Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: Owais Kazi <[email protected]>

* Update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.23.3 (#252)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Update dependency org.eclipse.platform:org.eclipse.core.runtime to v3.30.0 (#255)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Add setting to limit max workflow steps

Signed-off-by: Daniel Widdis <[email protected]>

---------

Signed-off-by: owaiskazi19 <[email protected]>
Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Joshua Palis <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Co-authored-by: Owais Kazi <[email protected]>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: Joshua Palis <[email protected]>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Dec 12, 2023
@github-actions @owaiskazi19
@mend-for-github.aaakk.us.kg @reta @joshpalis
Add setting to limit max workflow steps (#266)
caa42e7 
* Rename repo name in code files and CI (#182)

Signed-off-by: owaiskazi19 <[email protected]>

* Update actions/setup-java action to v4 (#219)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Use only pluginZip publication of Apache Maven artifacts (#226)

* Use only pluginZip publication of Apache Maven artifacts

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

* Address code review comments

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Andriy Redko <[email protected]>

* Integration test infrastructure set up (#230)

* Initial integ test framework modification, sets up integration test cluster and fixes ./gradlew run

Signed-off-by: Joshua Palis <[email protected]>

* spotless

Signed-off-by: Joshua Palis <[email protected]>

* Updating DEVELOPER_GUIDE

Signed-off-by: Joshua Palis <[email protected]>

---------

Signed-off-by: Joshua Palis <[email protected]>

* Update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.23.2 (#229)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Update to Gradle 8.5 (#227)

Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: Owais Kazi <[email protected]>

* Update dependency com.diffplug.spotless:spotless-plugin-gradle to v6.23.3 (#252)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Update dependency org.eclipse.platform:org.eclipse.core.runtime to v3.30.0 (#255)

Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>

* Add setting to limit max workflow steps

Signed-off-by: Daniel Widdis <[email protected]>

---------

Signed-off-by: owaiskazi19 <[email protected]>
Signed-off-by: mend-for-github.aaakk.us.kg[bot] <mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <[email protected]>
Signed-off-by: Joshua Palis <[email protected]>
Signed-off-by: Daniel Widdis <[email protected]>
Co-authored-by: Owais Kazi <[email protected]>
Co-authored-by: mend-for-github.aaakk.us.kg[bot] <50673670+mend-for-github.aaakk.us.kg[bot]@users.noreply.github.com>
Co-authored-by: Andriy Redko <[email protected]>
Co-authored-by: Joshua Palis <[email protected]>
(cherry picked from commit 7bec6e8)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amitgalitz amitgalitz amitgalitz approved these changes

@ohltyler ohltyler ohltyler approved these changes

@dbwiddis dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

@owaiskazi19 owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

@jackiehanyang jackiehanyang Awaiting requested review from jackiehanyang jackiehanyang is a code owner

Assignees
No one assigned
Labels
backport 2.x backport PRs to 2.x branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@joshpalis @amitgalitz @ohltyler