Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Stack Monitoring] working with packages doc #149067

Merged
merged 3 commits into from
Jan 26, 2023

Conversation

klacabane
Copy link
Contributor

Summary

Added howto work with packages documentation

@klacabane klacabane added the Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services label Jan 17, 2023
@klacabane klacabane self-assigned this Jan 17, 2023
@klacabane klacabane marked this pull request as ready for review January 18, 2023 14:56
@klacabane klacabane requested a review from a team as a code owner January 18, 2023 14:56
@elasticmachine
Copy link
Contributor

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

@@ -0,0 +1,179 @@
# Stack Monitoring integration packages

The Stack Monitoring packages provide a similar experience as their beats modules counterpart, both for metrics and logs ingestion. Feature wise these two collection modes are used to provide the Stack Monitoring UI the data it needs to power each product views. There is a however a difference in how the data is stored: while standalone beats modules will store all their data in a single data stream, for example `.monitoring-elasticsearch-8-mb`, the integrations will create one data stream for each metricset or filestream, for example `metrics-elasticsearch.stack-monitoring.node-default` for the node metricset. This means the integrations mappings are more specialized because they're split to only define the relevant properties for each metricset but the merged version of an integration data streams should be equivalent to its metricbeat mapping.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The Stack Monitoring packages provide a similar experience as their beats modules counterpart, both for metrics and logs ingestion. Feature wise these two collection modes are used to provide the Stack Monitoring UI the data it needs to power each product views. There is a however a difference in how the data is stored: while standalone beats modules will store all their data in a single data stream, for example `.monitoring-elasticsearch-8-mb`, the integrations will create one data stream for each metricset or filestream, for example `metrics-elasticsearch.stack-monitoring.node-default` for the node metricset. This means the integrations mappings are more specialized because they're split to only define the relevant properties for each metricset but the merged version of an integration data streams should be equivalent to its metricbeat mapping.
The Stack Monitoring packages provide a similar experience as their beats modules counterpart, both for metrics and logs ingestion. Feature wise these two collection modes are used to provide the Stack Monitoring UI with the data it needs to power each product view. There is however a difference in how the data is stored: while standalone beats modules will store all their data in a single data stream, for example `.monitoring-elasticsearch-8-mb`, the integrations will create one data stream for each metricset or filestream, for example `metrics-elasticsearch.stack-monitoring.node-default` for the node metricset. This means the integrations mappings are more specialized because they're split to only define the relevant properties for each metricset but the merged version of an integration data stream should be equivalent to its metricbeat mapping.

Comment on lines +60 to +62
A package can define the services it needs to monitor for development and automated testing purposes, for example the mysql package needs a running database to fetch and validate metrics or logs. While we could use the elasticsearch and kibana services started by `elastic-package stack up` we don't want the stack packages to be special cases and just like mysql, the packages should be responsible of defining their dependencies.

This can be done by creating a `docker-compose` file under the package `_dev/deploy` directory, then running `elastic-package service up -v` in the package folder. An example is the [elasticsearch package](https://github.com/elastic/integrations/tree/main/packages/elasticsearch/_dev/deploy/docker) that starts a service which generates every types of logs with the help of a script executing queries.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A package can define the services it needs to monitor for development and automated testing purposes, for example the mysql package needs a running database to fetch and validate metrics or logs. While we could use the elasticsearch and kibana services started by `elastic-package stack up` we don't want the stack packages to be special cases and just like mysql, the packages should be responsible of defining their dependencies.
This can be done by creating a `docker-compose` file under the package `_dev/deploy` directory, then running `elastic-package service up -v` in the package folder. An example is the [elasticsearch package](https://github.com/elastic/integrations/tree/main/packages/elasticsearch/_dev/deploy/docker) that starts a service which generates every types of logs with the help of a script executing queries.
A package can define the services it needs to monitor for development and automated testing purposes, for example, the mysql package needs a running database to fetch and validate metrics or logs. While we could use the elasticsearch and kibana services started by `elastic-package stack up` we don't want the stack packages to be exceptional cases and just like mysql, the packages should be responsible for defining their dependencies.
This can be done by creating a `docker-compose` file under the package `_dev/deploy` directory and then running `elastic-package service up -v` in the package folder. An example is the [elasticsearch package](https://github.com/elastic/integrations/tree/main/packages/elasticsearch/_dev/deploy/docker) that starts a service which generates every type of log with the help of a script executing queries.

Copy link
Contributor

@tonyghiani tonyghiani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great explanation here 👏
My browser just passed the text with the Grammarly extension and found a couple of typos, I marked them as suggestions, but overall the content LGTM 👌

@klacabane klacabane added the release_note:skip Skip the PR/issue when compiling release notes label Jan 25, 2023
@klacabane klacabane merged commit de7eb48 into elastic:main Jan 26, 2023
@kibanamachine kibanamachine added v8.7.0 backport:skip This commit does not require backporting labels Jan 26, 2023
kqualters-elastic pushed a commit to kqualters-elastic/kibana that referenced this pull request Feb 6, 2023
## Summary

Added howto work with packages documentation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Stack Monitoring release_note:skip Skip the PR/issue when compiling release notes Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services v8.7.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants