Rename parameters for Distributed Workload Generation - Issue 258 #407
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
IanHoang
reviewed
Nov 6, 2023
Hi @IanHoang , Here is the screenshot of the log from the coordinator node. It shows a warning as it cannot reach the worker ips by GET.
|
|
@dosa-chammandi That makes sense, can you clear the data in the worker ips by deleting the corpus in |
|
Hi @IanHoang ,
|
IanHoang
approved these changes
Nov 7, 2023
AkshathRaghav
pushed a commit
to AkshathRaghav/opensearch-benchmark
that referenced
this pull request
Nov 8, 2023
…ensearch-project#407) Signed-off-by: vivek palakkat <[email protected]>
AkshathRaghav
pushed a commit
to AkshathRaghav/opensearch-benchmark
that referenced
this pull request
Nov 8, 2023
…ensearch-project#407) Signed-off-by: vivek palakkat <[email protected]> Signed-off-by: AkshathRaghav <[email protected]>
IanHoang
pushed a commit
to IanHoang/opensearch-benchmark
that referenced
this pull request
Jan 9, 2024
… 258 (opensearch-project#407)" This reverts commit 3adacc0.
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The terminology used in Distributed Workload Generation is updated.
Once all the hosts have had the daemon successfully start on them, we run the following command on the host we designated as --coordinator-ip.
Run test command on coordinator
opensearch-benchmark execute_test --pipeline=benchmark-only --workload= --load-worker-coordinator-hosts=<IP2,IP3> --target-hosts=
The parameter "--load-worker-coordinator-hosts" is renamed to "worker-ips" to be consistent with --node-ip and --coordinator-ip.
Issues Resolved
[#258]
Testing
To use Distributed Workload Generation, users have to first start the opensearch-benchmark daemon on all hosts they plan to distribute load from. One of the hosts --referred to as the load coordinator host -- coordinates requests while the others -- referred to as worker-ips -- will drive the requests / load. For example, if we have three hosts -- denoted by IP1, IP2, and IP3 --, we would do run the corresponding command on each host:
Run on IP1 Host
opensearch-benchmarkd start --node-ip=<IP1> --coordinator-ip=<IP1>Run on IP2 Host
opensearch-benchmarkd start --node-ip=<IP2> --coordinator-ip=<IP1>Run on IP3 Host
opensearch-benchmarkd start --node-ip=<IP3> --coordinator-ip=<IP1>Once all the hosts have had the daemon successfully start on them, run the following command on the host we designated as --coordinator-ip in the commands above (in this case, IP1):
Run test command on IP1 Host
opensearch-benchmark execute-test --pipeline=benchmark-only --workload=<workload> --worker-ips=<IP2,IP3> --target-hosts=<endpoint>This starts the test. IP1 is the coordinator and delegates a portion of ingestion and search requests for IP2 and IP3 to send to the target cluster.
It is expected to run the tests successfully. The following screenshot is an output from a test.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.