Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.x](backport #41889) Add support for podman metrics in docker module #41967

Merged
merged 4 commits into from
Dec 11, 2024

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Dec 10, 2024

Proposed commit message

  • WHAT: Enhance docker module so that podman metrics are collected and calculated correctly
  • WHY: support for podman metrics in docker module

Details

Podman offers a docker compatible API for metrics collection. So using docker module to collect podman metrics should be working out of the box.
In reality the memory metrics were not published at all and cpu usage percentage calculation was incorrect.
The reason is that the podman api returns zero values for precpu_stats. These stats refer to the latest's read cpu statistics, needed for cpu percentage calculation. Also due to these stats being zero, the memory metrics were not populated because of a sanity check in the code.

if containerStats.Stats.MemoryStats.Limit == 0 || containerStats.Stats.PreCPUStats.CPUUsage.TotalUsage == 0 {

For docker, the precpu_stats are returned every time.
The solution in the Podman's case is to stream the api response. By default we had set the stream option to false as there was no need in case of docker.

As part of this PR we introduce a new configuration parameter named podman (by default false). If set to true, indicating that podman is used, then the stream parameter is set to true, only for the collection of cpu and memory stats.
The reason is that for cpu and memory stats the precpu_stats are needed.
From the streamed response, we get the second response as it was noticed during testing that the precpu_stats of the first response were incorrect and the cpu was miscalculated.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

There is no disruptive user impact.

How to test this PR locally

  1. start podman machine (In macOS use podman desktop)
  2. start an nginx test container podman run -d -p 8080:80 --name mynginx nginx
  3. exec into mynginx pod and start a cpu intensive task:
    a. podman exec -ti mynginx bash
    b. while true; do :; done &
  4. run metricbeat locally with docker module enabled and podman parameter set to true
  5. watch the docker.* metrics being published to ES with correct values. Compare with podman stats command results.

Related issues

Use cases

Screenshots

Podman Stats command (check for mynginx container)
podman stats

docker cpu percentage

docker memory percentage


This is an automatic backport of pull request #41889 done by [Mergify](https://mergify.com).

* Add support for podman metrics

(cherry picked from commit 1fefdbb)
@mergify mergify bot requested a review from a team as a code owner December 10, 2024 09:39
@mergify mergify bot added the backport label Dec 10, 2024
@mergify mergify bot requested review from belimawr and mauri870 and removed request for a team December 10, 2024 09:39
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Dec 10, 2024
@botelastic
Copy link

botelastic bot commented Dec 10, 2024

This pull request doesn't have a Team:<team> label.

@MichaelKatsoulis MichaelKatsoulis enabled auto-merge (squash) December 11, 2024 11:22
@MichaelKatsoulis MichaelKatsoulis merged commit 05fd06c into 8.x Dec 11, 2024
32 checks passed
@MichaelKatsoulis MichaelKatsoulis deleted the mergify/bp/8.x/pr-41889 branch December 11, 2024 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant