Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Istio metrics and dashboards #4678

Closed
12 of 14 tasks
gsantoro opened this issue Nov 18, 2022 · 11 comments
Closed
12 of 14 tasks

Istio metrics and dashboards #4678

gsantoro opened this issue Nov 18, 2022 · 11 comments
Assignees
Labels
Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring]

Comments

@gsantoro
Copy link
Contributor

gsantoro commented Nov 18, 2022

Review the implementation of Istiod_metrics datastream under the Istio Integration according to this comment.

Related PRs

Related issues

Make sure that the following Criteria are met before releasing to GA:

Checklist for GA:

  • Supported versions are documented
  • Supported operating systems are documented (if applicable)
  • Integration tests exist
  • System tests exist
  • Automated checks that all fields are documented
  • Documentation
  • Fields follow ECS and naming conventions
  • Dashboards exists (if applicable)
  • Kibana Home Tutorial (if applicable)
    • Open PR against Kibana repo with tutorial. Examples can be found here. 404 on this link
  • Test log files exist for the grok patterns (from Filebeat)
  • Generated output for at least 1 log file exists (from Filebeat)
  • Example data.json exists and an automated way to generate it exists (go test -data) (from Metricbeat)
  • Test environment in Docker exist for integration tests (from Metricbeat)
  • Rewrite Dashboards from TSVB to Lens PR
@gsantoro gsantoro added the Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring] label Nov 18, 2022
@gsantoro gsantoro self-assigned this Nov 18, 2022
@gsantoro gsantoro changed the title Review Istiod metrics integration Make Istiod integration GA Nov 29, 2022
@gsantoro gsantoro changed the title Make Istiod integration GA Make Istio integration GA Nov 29, 2022
@rameshelastic rameshelastic changed the title Make Istio integration GA Review Istio metrics and dashboards and make the integration GA Dec 2, 2022
@gsantoro
Copy link
Contributor Author

gsantoro commented Jan 5, 2023

Comment has been addressed with release 0.2.2 and PR

@gsantoro
Copy link
Contributor Author

gsantoro commented Jan 5, 2023

Notes on checklist for GA:

  • integration and system tests
    • pipelines tests for access_logs datastream
    • system tests for istiod_metrics and proxy_metrics datastream with docker-compose
  • dashboard exists
    • dashboard are valid and can be loaded. Checks done by elastic-package test asset via jenkins
  • Automated checks that all fields are documented
    • all fields in sample_event.json are verified by elastic-package test static. doc
  • documentation in the README
  • instead of data.json, we have sample_event.json (one per data stream)
  • support version is documented in the readme. Istio 1.14.3

The previou checklist should cover all the checks from beats from this doc

@gsantoro
Copy link
Contributor Author

gsantoro commented Jan 5, 2023

More checks specific to integrations:

  • beats module in GA?
    • istio has a beats module but it doesn't use it for either logs (it uses filestream input) or metrics (it uses prometheus/collector module/metricset)
    • integrations in Beta since Sep 12, 2022
  • Have all the data sets been migrated to Integrations as data streams ?
    • Yes. Since with Istio 1.14.3 we don't need as many different datastream. Components like Galley, Citadel, Pilot have been merged into Istiod. The migrated datastream are Proxy and Istiod, which are the only ones necessary. There was no previous module for Istio under filebeat, there is now an access_logs datastream in the integration
  • Data stream specific checks
    • All metrics and logs are visible in Kibana UI
    • User can give proper inputs for each stream from the UI (streams.yml.hbs file)
    • Fields.yml, ecs.yml, base-fields.yml presence
    • Data stream dashboards have been imported properly to integrations. Dashboards should not be with snapshot versions of stack
    • pipeline tests (when applicable)
    • System Tests, generation of sample_event.json
    • documentation: fileds, events
  • versions of service. The integration should be tested with at least 2 versions of the service (one should be the latest release) ??
    • Upgrade to latest version of Istio. Test it and check for errors or new fields
  • kibana version compatiblity (check with all kibana versions that we are supporting)
  • Verify Code Owners

@gsantoro
Copy link
Contributor Author

gsantoro commented Jan 5, 2023

Some optional todo:

  • upgrade the integration to Istio 1.16.1
  • add new datastream for Istiod service logs. Istiod expose logs which are different from the access logs gathered from Proxy sidecar containers.

@gsantoro gsantoro changed the title Review Istio metrics and dashboards and make the integration GA Review Istio metrics and dashboards Feb 1, 2023
@gizas
Copy link
Contributor

gizas commented Feb 9, 2023

We have an open PR that summarises best Practises for Dashboards/ Visualisations here

Long story short, the best practise is to transform our visualisations to Lens.

There are specific cases that TSVB can still have features that are not supported in Lens, but those should be considered per individual case. Lets always consider what the main outcome of each dashboard should be and what is the goal we want to achieve. (Relevant doc - go to section Use Lens if possible)

For cloudnative team we had to change existing gauges and either replace them with Pie charts or with bar charts. We chose bar charts (with ad-hoc dataviews) because we wanted to depict the latest status of cluster (see relevant discussion here), and Pies could not support or case in 8.5

@gsantoro to answer your question regarding replacement of TSVB gauges, I would either try Bar charts or Pie charts (after 8.6) that support latest value filter. We also have another story in our backlog to investigate Pie charts and if is possible to use them in Kubernetes General Overview Dashboards. So it will be a win-win if you can use them here as well

@gizas
Copy link
Contributor

gizas commented Feb 9, 2023

@gsantoro Reminder to include in new version of dashboards the #4978

@gsantoro
Copy link
Contributor Author

gsantoro commented Feb 9, 2023

@gizas are you saying that I should do the dashboards update and #4978 in the same PR?

@gizas
Copy link
Contributor

gizas commented Feb 9, 2023

I would suggest yes. The addition is a small one, why to open 2 prs only for a filter?
But however you feel. It is more like a reminder not to leave it in backlog because we can solve it now

@gsantoro
Copy link
Contributor Author

gsantoro commented Feb 9, 2023

I'm happy to do that but at the same time I have seen some pushback when a PR covers more than a single topic.

@gizas
Copy link
Contributor

gizas commented Feb 9, 2023

I'm happy to do that but at the same time I have seen some pushback when a PR covers more than a single topic.

:) From who ? And I dont think that those two things can be considered as different topics

@mlunadia
Copy link

@gsantoro let's title this accordingly -> review

@gsantoro gsantoro changed the title Review Istio metrics and dashboards Istio metrics and dashboards Feb 15, 2023
@gsantoro gsantoro closed this as completed Mar 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Cloudnative-Monitoring Label for the Cloud Native Monitoring team [elastic/obs-cloudnative-monitoring]
Projects
None yet
Development

No branches or pull requests

3 participants