Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Infra UI] Normalise network and Disk rates for Hosts across time ranges #164152

Closed
1 of 2 tasks
roshan-elastic opened this issue Aug 17, 2023 · 6 comments · Fixed by #165680
Closed
1 of 2 tasks

[Infra UI] Normalise network and Disk rates for Hosts across time ranges #164152

roshan-elastic opened this issue Aug 17, 2023 · 6 comments · Fixed by #165680
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Feature:Metrics UI Metrics UI feature Feature:ObsHosts Hosts feature within Observability Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services
Milestone

Comments

@roshan-elastic
Copy link

roshan-elastic commented Aug 17, 2023

🔗 Key Links

Issues

Note : This will be completed once the epic has been refined and issues created

Related

Preview Give feedback
  1. 0 of 1
    Feature:Metrics UI Feature:ObsHosts Team:obs-ux-infra_services bug needs-refinement
    roshan-elastic

Issues/Tasks

Preview Give feedback

📖 Description

Update Network RX/TX and Disk throughput/IOPS metrics to be normalised across different durations to allow SREs to compare them across different durations in Host List, fly-outs and detail view.

Background

When SREs look across different time periods, they need a consistent metrics to allow them to be able to compare the same metric across different durations.

The following metrics are not normalised in way which makes them comparable across different length durations (across both the Host List, fly-out and detail view):

Disk IOPS (read/write) (docs)

Fly-out
image

Disk Throughput (read/write) (docs)

Fly-out
image

Network (RX/TX) (docs)

Host Fly-out
image

Host List
image

💡 Solution

For normalising all of the metrics, apply the per second normalisation to all of the above in the lens charts:

image

I have confirmed this outcome would be desirable with @elkargig (George the SRE)

✔️ Acceptance criteria

1. Must Have

Must be delivered in this issue in order for the release to be valuable

Name Description Notes
Disk and Network metrics must be normalised across different durations The user should be able to compare the metrics across any duration -

2. Should Have

Name Description Notes
- - -

3. Could Have

Would be nice to have but not critical

Name Description Notes
- - -

4. Will Not Have (for now)

Explicitly will not be looked at within this issue

Name Description Notes
- - -

📈 Telemetry Process

  • Telemetry requirements must be part of the acceptance criteria (above) (defined by the Epic creator, e.g. the Product Manager) during refinement.
  • See Telemetry Process for full details/process/implementation conventions.
@roshan-elastic roshan-elastic added Feature:Metrics UI Metrics UI feature Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services Feature:ObsHosts Hosts feature within Observability labels Aug 17, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

@roshan-elastic roshan-elastic added the needs-refinement A reason and acceptance criteria need to be defined for this issue label Aug 17, 2023
@roshan-elastic roshan-elastic changed the title [Infra UI] Normalise network and throughput metrics for Hosts across time ranges [Infra UI] Normalise network and throughput rates for Hosts across time ranges Aug 17, 2023
@roshan-elastic roshan-elastic changed the title [Infra UI] Normalise network and throughput rates for Hosts across time ranges [Infra UI] Normalise network and Disk rates for Hosts across time ranges Aug 17, 2023
@roshan-elastic
Copy link
Author

Hey @smith, I had some internal feedback on the overview that has made me realise 3 of the charts need updating (minor changes to the config) - otherwise they don't really provide value and undermine the fly-out value.

Do you think there's any scope to making this change and back-porting to 8.10?

If possible, I'd like to add one more chart to the overview as well (it's something which is going into the detail view anyway)?

Note : On this latter one, we've found that the disk usage metric is disguising volumes which are running out of space. I created a separate issue to refine this but the above issue is to add the breakdown by mount point so users can easily see which volume is running out in the fly-out

@roshan-elastic roshan-elastic added bug Fixes for quality problems that affect the customer experience and removed needs-refinement A reason and acceptance criteria need to be defined for this issue labels Aug 17, 2023
@roshan-elastic roshan-elastic added this to the 8.10 milestone Aug 21, 2023
@smith
Copy link
Contributor

smith commented Aug 25, 2023

Do you think there's any scope to making this change and back-porting to 8.10?

Yes probably if we're fixing the normalization

If possible, I'd like to add one more chart to the overview as well (it's something which is going into the detail view anyway)?

This would probably be something we should not backport.

@roshan-elastic
Copy link
Author

Do you think there's any scope to making this change and back-porting to 8.10?

Yes probably if we're fixing the normalization

Great - thanks @smith. Could I leave it with you to try and get it done before 8.10 goes out?

If possible, I'd like to add one more chart to the overview as well (it's something which is going into the detail view anyway)?

This would probably be something we should not backport.

Could you share your thinking here (if only I can understand what is a good/bad candidate for back-porting in the future)?

Would love to get this in here because it's a really important use case:

If it can't/shouldn't be done though, I'll respect your judgement on this...

@smith
Copy link
Contributor

smith commented Aug 29, 2023

@roshan-elastic:

Could you share your thinking here (if only I can understand what is a good/bad candidate for back-porting in the future)?

Following the Kibana guidelines:

  • Features should not be backported to a <major>.<minor> branch.
  • Bug fixes can be backported to a <major>.<minor> branch if the changes are safe and appropriate. Safety is a judgment call you make based on factors like the bug’s severity, test coverage, confidence in the changes, etc. Your reasoning should be included in the pull request description.

Adding one more chart is not a bug fix, it's a feature.

@roshan-elastic
Copy link
Author

Cool - thanks @smith. This is really helpful.

I understand your thinking now so supportive of that. Will leave the new chart for 8.11...

@jennypavlova jennypavlova self-assigned this Sep 5, 2023
jennypavlova added a commit that referenced this issue Sep 6, 2023
…ges (#165680)

Closes #164152 

## Summary

This PR normalizes the disk and network charts per second. The charts
changed:
Disk IOPS (read/write) 
<img width="390" alt="diskiops"
src="https://github.com/elastic/kibana/assets/14139027/d0993032-7309-4fb9-9e4e-0e9bce73e89e">
Network (rx/tx)
<img width="402" alt="network"
src="https://github.com/elastic/kibana/assets/14139027/33b20570-60ca-48ef-839b-0363bcccd24a">
Disk Throughput (read/write)
<img width="590" alt="diskth"
src="https://github.com/elastic/kibana/assets/14139027/115eba5a-7d46-4846-8f88-fe041b4cfeed">

## Testing

1. Go to hosts view and check network and disk charts (they should be
normalized per second)
2. Open the host flyout and check network and disk charts (they should
be normalized per second)



https://github.com/elastic/kibana/assets/14139027/087cd044-b6cc-4612-8fca-391a96848365
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Sep 6, 2023
…ges (elastic#165680)

Closes elastic#164152

## Summary

This PR normalizes the disk and network charts per second. The charts
changed:
Disk IOPS (read/write)
<img width="390" alt="diskiops"
src="https://github.com/elastic/kibana/assets/14139027/d0993032-7309-4fb9-9e4e-0e9bce73e89e">
Network (rx/tx)
<img width="402" alt="network"
src="https://github.com/elastic/kibana/assets/14139027/33b20570-60ca-48ef-839b-0363bcccd24a">
Disk Throughput (read/write)
<img width="590" alt="diskth"
src="https://github.com/elastic/kibana/assets/14139027/115eba5a-7d46-4846-8f88-fe041b4cfeed">

## Testing

1. Go to hosts view and check network and disk charts (they should be
normalized per second)
2. Open the host flyout and check network and disk charts (they should
be normalized per second)

https://github.com/elastic/kibana/assets/14139027/087cd044-b6cc-4612-8fca-391a96848365
(cherry picked from commit 1487672)
kibanamachine added a commit that referenced this issue Sep 6, 2023
…ime ranges (#165680) (#165836)

# Backport

This will backport the following commits from `main` to `8.10`:
- [[Infra UI] Normalise network and Disk rates for Hosts across time
ranges (#165680)](#165680)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT
[{"author":{"name":"jennypavlova","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-09-06T11:30:51Z","message":"[Infra
UI] Normalise network and Disk rates for Hosts across time ranges
(#165680)\n\nCloses #164152 \r\n\r\n## Summary\r\n\r\nThis PR normalizes
the disk and network charts per second. The charts\r\nchanged:\r\nDisk
IOPS (read/write) \r\n<img width=\"390\"
alt=\"diskiops\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/d0993032-7309-4fb9-9e4e-0e9bce73e89e\">\r\nNetwork
(rx/tx)\r\n<img width=\"402\"
alt=\"network\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/33b20570-60ca-48ef-839b-0363bcccd24a\">\r\nDisk
Throughput (read/write)\r\n<img width=\"590\"
alt=\"diskth\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/115eba5a-7d46-4846-8f88-fe041b4cfeed\">\r\n\r\n##
Testing\r\n\r\n1. Go to hosts view and check network and disk charts
(they should be\r\nnormalized per second)\r\n2. Open the host flyout and
check network and disk charts (they should\r\nbe normalized per
second)\r\n\r\n\r\n\r\nhttps://github.com/elastic/kibana/assets/14139027/087cd044-b6cc-4612-8fca-391a96848365","sha":"14876726f4f6edd57672644a322e55cd3d2afd6a","branchLabelMapping":{"^v8.11.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Infra
Monitoring
UI","release_note:skip","backport:prev-minor","v8.11.0"],"number":165680,"url":"https://github.com/elastic/kibana/pull/165680","mergeCommit":{"message":"[Infra
UI] Normalise network and Disk rates for Hosts across time ranges
(#165680)\n\nCloses #164152 \r\n\r\n## Summary\r\n\r\nThis PR normalizes
the disk and network charts per second. The charts\r\nchanged:\r\nDisk
IOPS (read/write) \r\n<img width=\"390\"
alt=\"diskiops\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/d0993032-7309-4fb9-9e4e-0e9bce73e89e\">\r\nNetwork
(rx/tx)\r\n<img width=\"402\"
alt=\"network\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/33b20570-60ca-48ef-839b-0363bcccd24a\">\r\nDisk
Throughput (read/write)\r\n<img width=\"590\"
alt=\"diskth\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/115eba5a-7d46-4846-8f88-fe041b4cfeed\">\r\n\r\n##
Testing\r\n\r\n1. Go to hosts view and check network and disk charts
(they should be\r\nnormalized per second)\r\n2. Open the host flyout and
check network and disk charts (they should\r\nbe normalized per
second)\r\n\r\n\r\n\r\nhttps://github.com/elastic/kibana/assets/14139027/087cd044-b6cc-4612-8fca-391a96848365","sha":"14876726f4f6edd57672644a322e55cd3d2afd6a"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.11.0","labelRegex":"^v8.11.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/165680","number":165680,"mergeCommit":{"message":"[Infra
UI] Normalise network and Disk rates for Hosts across time ranges
(#165680)\n\nCloses #164152 \r\n\r\n## Summary\r\n\r\nThis PR normalizes
the disk and network charts per second. The charts\r\nchanged:\r\nDisk
IOPS (read/write) \r\n<img width=\"390\"
alt=\"diskiops\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/d0993032-7309-4fb9-9e4e-0e9bce73e89e\">\r\nNetwork
(rx/tx)\r\n<img width=\"402\"
alt=\"network\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/33b20570-60ca-48ef-839b-0363bcccd24a\">\r\nDisk
Throughput (read/write)\r\n<img width=\"590\"
alt=\"diskth\"\r\nsrc=\"https://github.com/elastic/kibana/assets/14139027/115eba5a-7d46-4846-8f88-fe041b4cfeed\">\r\n\r\n##
Testing\r\n\r\n1. Go to hosts view and check network and disk charts
(they should be\r\nnormalized per second)\r\n2. Open the host flyout and
check network and disk charts (they should\r\nbe normalized per
second)\r\n\r\n\r\n\r\nhttps://github.com/elastic/kibana/assets/14139027/087cd044-b6cc-4612-8fca-391a96848365","sha":"14876726f4f6edd57672644a322e55cd3d2afd6a"}}]}]
BACKPORT-->

Co-authored-by: jennypavlova <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Metrics UI Metrics UI feature Feature:ObsHosts Hosts feature within Observability Team:Infra Monitoring UI - DEPRECATED DEPRECATED - Label for the Infra Monitoring UI team. Use Team:obs-ux-infra_services
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants