Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump corpus generator v0.6.0 #1491

Closed
wants to merge 23 commits into from
Closed

Bump corpus generator v0.6.0 #1491

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
1e71c71
bump github.com/elastic/elastic-integration-corpus-generator-tool v0.6.0
Oct 6, 2023
6354f8b
use new tot_events instead of size
Oct 6, 2023
db17f01
update docs
Oct 6, 2023
437dcf0
update tests
Oct 6, 2023
f098d4b
init generator
Oct 6, 2023
a734849
fix descriptions to reference events instead of size
Oct 6, 2023
274f70e
check static
Oct 6, 2023
846a481
format_version: 2.8.1-next\?
Oct 6, 2023
3cae2a2
2.8.1 was released
Oct 6, 2023
512a3a6
fix test system_benchmark package for format_version 2.8.1
Oct 6, 2023
5377dff
format_version: 3.0.1-next
Oct 6, 2023
c64ceaf
bump to v0.7.0
Oct 10, 2023
ff25aa7
refactor for v0.7.0
Oct 10, 2023
77603ca
Merge branch 'main' into bump-corpus-generator-v0.6.0
Oct 10, 2023
a51fe55
go mod tidy
Oct 10, 2023
e2f09ff
Merge branch 'main' into bump-corpus-generator-v0.6.0
Oct 15, 2023
8903c20
package-spec 3.0.0
Oct 15, 2023
4b664a2
rename to total_events
Oct 16, 2023
06143af
add owner.type in manifest
Oct 16, 2023
43942ba
Merge branch 'main' into bump-corpus-generator-v0.6.0
Oct 17, 2023
de55eb1
Merge branch 'main' into bump-corpus-generator-v0.6.0
Oct 18, 2023
85116cf
is it an yaml formatting issue?
Oct 18, 2023
f154b83
is it an yaml formatting issue? it seems so
Oct 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/howto/sample_metric.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,7 +630,7 @@
},
"parameter": {
"package": "system_benchmarks",
"description": "Benchmark 20MiB of data ingested",
"description": "Benchmark 20000 events ingested",
"version": "999.999.999",
"policy_template": "testpo",
"input": "filestream",
Expand All @@ -648,7 +648,7 @@
"wait_for_data_timeout": 60000000000,
"corpora": {
"generator": {
"size": "20MiB",
"total_events": "20000",
"template": {
"raw": "",
"path": "./logs-benchmark/template.log",
Expand Down
54 changes: 27 additions & 27 deletions docs/howto/system_benchmarking.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,43 +227,43 @@ want to benchmark. There can be multiple scenarios defined for the same package.
The `<scenario>.yml` files allow you to define various settings for the benchmark scenario
along with values for package and data stream-level variables. These are the available configuration options for system benchmarks.

| Option | Type | Required | Description |
|---|---|---|---|
| package | string | | The name of the package. If omitted will pick the current package, this is to allow for future definition of benchmarks outside of the packages folders. |
| description | string | | A description for the scenario. |
| version | string | | The version of the package to benchmark. If omitted will pick the current version of the package. |
| policy_template | string | | The policy template to test. If omitted will pick the first one. |
| input | string | yes | Input type to test (e.g. logfile, httpjson, etc). Defaults to the input used by the first stream in the data stream manifest. |
| vars | dictionary | | Package level variables to set (i.e. declared in `$package_root/manifest.yml`). If not specified the defaults from the manifest are used. |
| data_stream.name | string | yes | The data stream to benchmark. |
| data_stream.vars | dictionary | | Data stream level variables to set (i.e. declared in `package_root/data_stream/$data_stream/manifest.yml`). If not specified the defaults from the manifest are used. |
| warmup_time_period | duration | | Warmup time period. All data prior to this period will be ignored in the benchmark results. |
| benchmark_time_period | duration | | Amount of time the benchmark needs to run for. If set the benchmark will stop after this period even though more data is still pending to be ingested. |
| wait_for_data_timeout | duration | | Amount of time to wait for data to be present in Elasticsearch. Defaults to 10m. |
| corpora.generator.size | string | | String describing the amount of data to generate. Example: `20MiB` |
| corpora.generator.template.raw | string | | Raw template for the corpus generator. |
| corpora.generator.template.path | string | | Path to the template for the corpus generator. If a `path` is defined, it will override any `raw` template definition. |
| corpora.generator.template.type | string | | Type of the template for the corpus generator. Default `placeholder`. |
| corpora.generator.config.raw | dictionary | | Raw config for the corpus generator. |
| corpora.generator.config.path | string | | Path to the config for the corpus generator. If a `path` is defined, it will override any `raw` config definition. |
| corpora.generator.fields.raw | dictionary | | Raw fields for the corpus generator. |
| corpora.generator.fields.path | string | | Path to the fields for the corpus generator. If a `path` is defined, it will override any `raw` fields definition. |
| corpora.input_service.name | string | | Name of the input service to use (defined in the `deploy` folder). |
| corpora.input_service.signal | string | | Signal to send to the input service once the benchmark is ready to start. |
| Option | Type | Required | Description |
|---------------------------------|------------|---|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| package | string | | The name of the package. If omitted will pick the current package, this is to allow for future definition of benchmarks outside of the packages folders. |
| description | string | | A description for the scenario. |
| version | string | | The version of the package to benchmark. If omitted will pick the current version of the package. |
| policy_template | string | | The policy template to test. If omitted will pick the first one. |
| input | string | yes | Input type to test (e.g. logfile, httpjson, etc). Defaults to the input used by the first stream in the data stream manifest. |
| vars | dictionary | | Package level variables to set (i.e. declared in `$package_root/manifest.yml`). If not specified the defaults from the manifest are used. |
| data_stream.name | string | yes | The data stream to benchmark. |
| data_stream.vars | dictionary | | Data stream level variables to set (i.e. declared in `package_root/data_stream/$data_stream/manifest.yml`). If not specified the defaults from the manifest are used. |
| warmup_time_period | duration | | Warmup time period. All data prior to this period will be ignored in the benchmark results. |
| benchmark_time_period | duration | | Amount of time the benchmark needs to run for. If set the benchmark will stop after this period even though more data is still pending to be ingested. |
| wait_for_data_timeout | duration | | Amount of time to wait for data to be present in Elasticsearch. Defaults to 10m. |
| corpora.generator.total_events | uint64 | | Number of total events to generate. Example: `20000` |
| corpora.generator.template.raw | string | | Raw template for the corpus generator. |
| corpora.generator.template.path | string | | Path to the template for the corpus generator. If a `path` is defined, it will override any `raw` template definition. |
| corpora.generator.template.type | string | | Type of the template for the corpus generator. Default `placeholder`. |
| corpora.generator.config.raw | dictionary | | Raw config for the corpus generator. |
| corpora.generator.config.path | string | | Path to the config for the corpus generator. If a `path` is defined, it will override any `raw` config definition. |
| corpora.generator.fields.raw | dictionary | | Raw fields for the corpus generator. |
| corpora.generator.fields.path | string | | Path to the fields for the corpus generator. If a `path` is defined, it will override any `raw` fields definition. |
| corpora.input_service.name | string | | Name of the input service to use (defined in the `deploy` folder). |
| corpora.input_service.signal | string | | Signal to send to the input service once the benchmark is ready to start. |

Example:

`logs-benchmark.yml`
```yaml
---
description: Benchmark 100MiB of data ingested
description: Benchmark 100000 events ingested
input: filestream
vars: ~
data_stream.name: test
data_stream.vars.paths:
- "{{SERVICE_LOGS_DIR}}/corpus-*"
warmup_time_period: 10s
corpora.generator.size: 100MiB
corpora.generator.total_events: 100000
corpora.generator.template.path: ./logs-benchmark/template.log
corpora.generator.config.path: ./logs-benchmark/config.yml
corpora.generator.fields.path: ./logs-benchmark/fields.yml
Expand Down Expand Up @@ -322,7 +322,7 @@ elastic-package benchmark system --benchmark logs-benchmark -v
│ info │
├──────────────┬──────────────────────────────────────┤
│ benchmark │ logs-benchmark │
│ description │ Benchmark 100MiB of data ingested │
│ description │ Benchmark 100000 events ingested │
│ run ID │ d2960c04-0028-42c9-bafc-35e599563cb1 │
│ package │ system_benchmarks │
│ start ts (s) │ 1682320355 │
Expand All @@ -339,7 +339,7 @@ elastic-package benchmark system --benchmark logs-benchmark -v
│ warmup time period │ 10s │
│ benchmark time period │ 0s │
│ wait for data timeout │ 0s │
│ corpora.generator.size │ 100MiB │
│ corpora.generator.total_events │ 100000 │
│ corpora.generator.template.path │ ./logs-benchmark/template.log │
│ corpora.generator.template.raw │ │
│ corpora.generator.template.type │ │
Expand Down
19 changes: 10 additions & 9 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ require (
github.com/cbroglie/mustache v1.4.0
github.com/cespare/xxhash/v2 v2.2.0
github.com/dustin/go-humanize v1.0.1
github.com/elastic/elastic-integration-corpus-generator-tool v0.5.0
github.com/elastic/elastic-integration-corpus-generator-tool v0.7.0
github.com/elastic/go-elasticsearch/v7 v7.17.10
github.com/elastic/go-licenser v0.4.1
github.com/elastic/go-resource v0.1.1
Expand All @@ -20,15 +20,15 @@ require (
github.com/fatih/color v1.15.0
github.com/go-git/go-billy/v5 v5.5.0
github.com/go-git/go-git/v5 v5.9.0
github.com/google/go-cmp v0.6.0
github.com/google/go-cmp v0.5.9
github.com/google/go-github/v32 v32.1.0
github.com/google/go-querystring v1.1.0
github.com/google/uuid v1.3.1
github.com/jedib0t/go-pretty v4.3.0+incompatible
github.com/magefile/mage v1.15.0
github.com/mholt/archiver/v3 v3.5.1
github.com/olekukonko/tablewriter v0.0.5
github.com/pmezard/go-difflib v1.0.0
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
github.com/shirou/gopsutil/v3 v3.23.9
github.com/spf13/cobra v1.7.0
github.com/stretchr/testify v1.8.4
Expand Down Expand Up @@ -63,7 +63,7 @@ require (
github.com/cloudflare/circl v1.3.3 // indirect
github.com/creasty/defaults v1.7.0 // indirect
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dnephin/pflag v1.0.7 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/elastic/gojsonschema v1.2.1 // indirect
Expand Down Expand Up @@ -103,7 +103,7 @@ require (
github.com/json-iterator/go v1.1.12 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.16.0 // indirect
github.com/klauspost/compress v1.17.0 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de // indirect
github.com/lithammer/shortuuid/v3 v3.0.7 // indirect
Expand Down Expand Up @@ -136,7 +136,8 @@ require (
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/shopspring/decimal v1.3.1 // indirect
github.com/skeema/knownhosts v1.2.0 // indirect
github.com/spf13/cast v1.5.0 // indirect
github.com/spf13/afero v1.10.0 // indirect
github.com/spf13/cast v1.5.1 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/stretchr/objx v0.5.1 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
Expand All @@ -153,15 +154,15 @@ require (
golang.org/x/crypto v0.14.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20221208152030-732eee02a75a // indirect
golang.org/x/mod v0.13.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.8.0 // indirect
golang.org/x/net v0.16.0 // indirect
golang.org/x/oauth2 v0.12.0 // indirect
golang.org/x/sync v0.4.0 // indirect
golang.org/x/sys v0.13.0 // indirect
golang.org/x/term v0.13.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/time v0.3.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/protobuf v1.30.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
Expand Down
Loading