Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

removing job-scheduler zip and replacing with distribution build #487

Merged
merged 2 commits into from
Apr 4, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,5 @@ out/
.classpath
.vscode
bin/
._.DS_Store
._.DS_Store
src/test/resources/job-scheduler/
21 changes: 19 additions & 2 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,16 @@ buildscript {
// 2.0.0-alpha1-SNAPSHOT -> 2.0.0.0-alpha1-SNAPSHOT
version_tokens = opensearch_version.tokenize('-')
opensearch_build = version_tokens[0] + '.0'
job_scheduler_no_snapshot = opensearch_build
if (buildVersionQualifier) {
opensearch_build += "-${buildVersionQualifier}"
job_scheduler_no_snapshot += "-${buildVersionQualifier}"
}
if (isSnapshot) {
opensearch_build += "-SNAPSHOT"
}
opensearch_no_snapshot = opensearch_version.substring(0, opensearch_version.indexOf("-", opensearch_version.indexOf("-") + 1))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is just to remove the -SNAPSHOT, can't we just persist the opensearch_build var as a separate var before the optional -SNAPSHOT is added on lines 30-32?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, what's the difference between job_scheduler_no_snapshot and opensearch_no_snapshot?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just replace -SNAPSHOT with should be enough.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

job_scheduler_no_snapshot = 2.0.0.0-alpha1
opensearch_no_snapshot = 2.0.0-alpha1
The optional -SNAPSHOT is added after adding the .0
@peterzhuamazon good call, I'll change to opensearch_version.replace("-SNAPSHOT, "")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ohltyler since one has the .0 and one doesn't, I changed it to .replace from the opensearch_version. Might have misunderstood what you meant

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, missed that. Your change looks good to me!

js_resource_folder = "src/test/resources/job-scheduler"
common_utils_version = System.getProperty("common_utils.version", opensearch_build)
job_scheduler_version = System.getProperty("job_scheduler.version", opensearch_build)
}
Expand Down Expand Up @@ -254,7 +258,14 @@ testClusters.integTest {
return new RegularFile() {
@Override
File getAsFile() {
return fileTree("src/test/resources/job-scheduler").getSingleFile()
project.mkdir js_resource_folder
ant.get(src: 'https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/'
+ opensearch_no_snapshot
+ '/latest/linux/x64/builds/opensearch/plugins/opensearch-job-scheduler-'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see we use hard code x64 here, not sure if it will impact CI/IT workflow for other platform, @peterzhuamazon any suggestion?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since github actions offer default runner are x64 I see no problem running on hardcoded x64.
Jenkins are testing both x64 and arm64 tho after each build.
If you really want arm64 to be on github action you need to manage self-hosted runners yourself tho.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will eventually become a problem, but not just because of x64. For example, windows. For now this is fine.

+ job_scheduler_no_snapshot + '.zip',
ohltyler marked this conversation as resolved.
Show resolved Hide resolved
dest: js_resource_folder,
httpusecaches: false)
return fileTree(js_resource_folder).getSingleFile()
}
}
}
Expand Down Expand Up @@ -326,7 +337,13 @@ String bwcFilePath = "src/test/resources/org/opensearch/ad/bwc/"
return new RegularFile() {
@Override
File getAsFile() {
return fileTree(bwcFilePath + "job-scheduler/" + bwcVersion).getSingleFile()
ant.get(src: 'https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/'
+ opensearch_no_snapshot
+ '/latest/linux/x64/builds/opensearch/plugins/opensearch-job-scheduler-'
+ job_scheduler_no_snapshot + '.zip',
dest: bwcFilePath + "job-scheduler/" + opensearch_version,
httpusecaches: false)
return fileTree(bwcFilePath + "job-scheduler/" + opensearch_version).getSingleFile()
}
}
}
Expand Down
Binary file not shown.