-
Notifications
You must be signed in to change notification settings - Fork 419
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Usage improvements #884
Usage improvements #884
Conversation
@jonathan-buttner @marshallmain Would you take a pass and let me know if you have any feedback on the usage docs for some of the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great readme! Thanks for creating it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fantastic, thanks for turning my infamous comment on #746 into proper documentation 😃
I'm providing a few thoughts here for a follow-up PR. That seemed the simplest way to provide this feedback 🙂
Prior to installing dependencies or running the tools, it's recommended to check out the `git` branch for the ECS version being targeted. | ||
|
||
**Example**: For ECS `1.5.0`: | ||
|
||
``` | ||
$ git checkout v1.5.0 | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Now that we have --ref
we don't need this step anymore :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great point, we should definitely promote --ref
instead.
I ran up against an issue (as I was going to update the docs 😄 ) trying to run the generator with --ref
(#892), so I may hold off removing this until we squash that bug.
* The script will need to be executed from the top-level of the ECS repo | ||
* The `version` displayed when running `generator.py` is based on the current value of the [version](version) file in the top-level of the repo | ||
|
||
The following options add functionality beyond the defaults. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could add here that the ECS team maintains the official ECS artifacts by running the script in this way.
Then we can segue from that to explain "and now here's how you add your own fields, or remove the fields you don't need"
@@ -0,0 +1,274 @@ | |||
# ECS Tooling Usage | |||
|
|||
In addition to the published schema and generated artifacts, the ECS repo also contains tools to generate artifacts based on the current published and custom schemas. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should expand more here.
Q. Why do people care?
A.
- ECS has too many fields
- ECS doesn't have their custom fields.
=> Users can use ECS tools to fix both of these things in their project. They can maintain only their customizations and use these tools to produce relevant artifacts for their various data sources.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In addition to this, we could specifically call out the two artifacts that I expect users will find most useful. The Elasticsearch template is expected to be most useful for users in general. But Beats contributors may instead be interested in the Beats-formatted YAML field definition files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These are great additions. Maybe another nice, future addition is tutorial-like walkthroughs showing end-to-end how to define your own schema customizations, generate artifacts, and then application in Beats or ES.
$ python scripts/generator.py --out ../myproject/ecs/out/ | ||
``` | ||
|
||
Inside the directory passed in as the target dir to the `--out` flag, two directories, `generated` and `docs`, will be created. `docs` will contain three asciidoc files based on the contents of the provided schema. `generated` will contain the various artifacts laid out as in the published repo (`beats`, `csv`, `ecs`, `elasticsearch`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As soon as either --include
or --subset
are used, the asciidoc is not generated, though.
$ python scripts/generator.py --ref v1.5.0 | ||
``` | ||
> Note: `--ref` does have a dependency on `git` being installed and all expected commits/tags fetched from the ECS upstream repo. This will unlikely be an issue unless you downloaded the ECS as a zip archive from GitHub vs. cloning it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good to mention 👍
|
||
#### Intermediate-Only | ||
|
||
The `--intermediate-only` argument is used for debugging purposes. It only generates the "intermediate files" of ECS without generating the rest of the artifacts. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could mention them explicitly here, ecs_flat.yml
and ecs_nested.yml
Thanks @webmat for the feedback! I hope this helps clear up some of the common confusion with using the tooling. I'll work on including your feedback/suggestions in a follow-up PR 😄 . I need to update the documented file format for |
Adds
USAGE.md
covering the getting started experience and usage of the ECS tooling. The currentgenerator.py
arguments are documented along with example usage.The aim is to simplify the experience for users generating their own artifacts (based off the published or custom schema definitions).
USAGE.md
is linked fromREADME.md
Caveats
--subset
usage will need to be updated to reflect the modifiedsubset
file format and improvementsAddresses #746