Auto-generated / custom datasets for workloads

[achitojha]

OpenSearch Benchmark workloads currently use an existing dataset to ingest records into OpenSearch/ElasticSearch. The goal for this task is to build support for automatically generated datasets for a workload. This would enable workloads to have large auto-generated datasets without requiring any specific data.

Acceptance Criteria

• We have generated 3 to 5 datasets for custom workloads based on real world use cases
• The generated workloads can be used by OpenSearch benchmark to measure specific performance stats for OpenSearch cluster
• The custom workloads are specific to the needs (Ex: smaller workloads for executing against each PR, medium workloads for more frequent runs, complex workloads to capture specific issues / regression for longevity tests etc..)

[dblock]

Hopefully those datasets aren't "random", but maybe "configurable or auto-generated data sets"?

[achitojha]

@dblock : Agreed - updated the issue.

[bbarani]

@dblock @achitojha Is the plan here to create automated workloads (Ex: using create-workload --workload) using an existing active cluster and use that workload for performance testing?

[achitojha]

We don’t have a finalized approach for this. The above suggestion could be a possible step forward. Another option here could be to specify certain attributes and have OpenSearch benchmark general a data-set based on the specified attributes. There may still be more alternate approaches

[gkamat]

Initial focus is on providing a capability to increase the data corpus size for a workload.

[dtaivpp]

One thing that I wanted to add is it would be handy if in this or future versions we could add some random-ness that could be used to demonstrate anomaly detection.

[dblock]

I'd like to be able to generate such data as "1B IP addresses skewed towards US-based IPs" (or other data that flows some statistical distribution). Maybe there are existing tools that can do that well?

[dblock]

Is this a dup/subset of #253?

[IanHoang]

@dblock yes, this is technically a duplicate/subset of RFC as this issue was created before. The RFC dives deeper.

[dblock]

Let's close!