Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for CgroupsV2 in beats, migrate away from gosigar #27242

Merged
merged 35 commits into from
Aug 12, 2021
Merged

Add support for CgroupsV2 in beats, migrate away from gosigar #27242

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f57a803
finish first round of cgroupsv2 support
fearful-symmetry Aug 4, 2021
2a4de30
Merge remote-tracking branch 'upstream/master' into cgroups2
fearful-symmetry Aug 4, 2021
e2db3b9
fix tests
fearful-symmetry Aug 4, 2021
2485e87
remove old code, fix PctOpt
fearful-symmetry Aug 4, 2021
bbf1523
fix imports
fearful-symmetry Aug 5, 2021
8315ab8
go mod tidy
fearful-symmetry Aug 5, 2021
6c58941
Merge remote-tracking branch 'upstream/master' into cgroups2
fearful-symmetry Aug 5, 2021
4897385
try to fix make notice again
fearful-symmetry Aug 5, 2021
6e42fff
try again
fearful-symmetry Aug 5, 2021
0cb854c
remove unneeded test files
fearful-symmetry Aug 5, 2021
963a65c
fix test paths
fearful-symmetry Aug 5, 2021
5c2c376
fix crossbuild issues
fearful-symmetry Aug 5, 2021
9c8bc8c
fix tests, remove debug statement
fearful-symmetry Aug 9, 2021
90cd5ec
change metadata, fight with mapping defs
fearful-symmetry Aug 9, 2021
c9c2263
fix v1 test, remove debug line
fearful-symmetry Aug 9, 2021
1afde0e
somewhat hacky fix for fields issues
fearful-symmetry Aug 9, 2021
ea4ed18
fix v1 fetch, update fields again
fearful-symmetry Aug 9, 2021
7cb31a1
Merge remote-tracking branch 'upstream/master' into cgroups2
fearful-symmetry Aug 9, 2021
40ed2f0
fix omitempty issue
fearful-symmetry Aug 9, 2021
783877d
make update
fearful-symmetry Aug 9, 2021
b53184b
remove older test
fearful-symmetry Aug 9, 2021
9e6a9be
fix tests, cgv1 logic
fearful-symmetry Aug 10, 2021
f72ed57
remove old debug statement
fearful-symmetry Aug 10, 2021
5fdfb0a
fix issue with how ubuntu mixes cgroups
fearful-symmetry Aug 10, 2021
10a5395
clean up error handling in libbeat
fearful-symmetry Aug 10, 2021
ba7259a
changes based on feedback, increased error verbosity to try to fix ba…
fearful-symmetry Aug 10, 2021
a092765
fix fields, add more error messages for weird CI bug
fearful-symmetry Aug 10, 2021
a93271c
I give up, add tons of debug statements
fearful-symmetry Aug 10, 2021
e1cf240
fix issue with docker containers running under hybrid cgroups
fearful-symmetry Aug 11, 2021
2e9b94d
fix hostfs state check
fearful-symmetry Aug 11, 2021
0f03ac8
fix more broken tests
fearful-symmetry Aug 11, 2021
038f397
fix names, log levels
fearful-symmetry Aug 11, 2021
dd7a681
more changes, docs, test
fearful-symmetry Aug 11, 2021
b2ccbf1
still making the mapping checks happy
fearful-symmetry Aug 11, 2021
eeda969
fix libbeat code I broke
fearful-symmetry Aug 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions libbeat/metric/system/cgroup/util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,10 @@ func SubsystemMountpoints(rootfsMountpoint string, subsystems map[string]struct{

// V2 option
if mount.filesystemType == "cgroup2" {
if len(mount.mountpoint) < 3 {
// Why is this happening? I don't know.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😃

Where does it happen? How does mountinfo look in these cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think you're looking at an older version, that's while I was debugging an issue with hybrid cgroups.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I was curious about what appeared in mountinfo when this happens 🙂

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! The cgroups line just...isn't there.

return mountInfo, fmt.Errorf("error parsing mountinfo in line '%s', filesystemType is set, but location is invalid", line)
}
mountInfo.V2Loc = mount.mountpoint
}

Expand Down
12 changes: 6 additions & 6 deletions metricbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53942,7 +53942,7 @@ type: long
--
detailed memory IO stats

type: nested
type: object

--

Expand All @@ -53951,7 +53951,7 @@ type: nested
--
detailed memory IO stats

type: nested
type: object

--

Expand Down Expand Up @@ -54214,14 +54214,14 @@ type: keyword
--
per-device IO usage stats

type: nested
type: object

--

*`system.process.cgroup.io.stats.*.*`*::
+
--
type: nested
type: object

--

Expand All @@ -54230,7 +54230,7 @@ type: nested
--
per-device IO usage stats

type: nested
type: object

--

Expand All @@ -54239,7 +54239,7 @@ type: nested
--
per-device IO usage stats

type: nested
type: object

--

Expand Down
2 changes: 1 addition & 1 deletion metricbeat/module/system/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions metricbeat/module/system/process/_meta/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -552,10 +552,10 @@
was reached.

- name: stats.*
type: nested
type: object
description: detailed memory IO stats
- name: stats.*.bytes
type: nested
type: object
description: detailed memory IO stats

- name: stats.active_anon.bytes
Expand Down Expand Up @@ -695,15 +695,15 @@
description: >
Path to the cgroup relative to the cgroup subsystems mountpoint.
- name: stats.*
type: nested
type: object
description: per-device IO usage stats
- name: stats.*.*
type: nested
type: object
- name: stats.*.*.bytes
type: nested
type: object
description: per-device IO usage stats
- name: stats.*.*.ios
type: nested
type: object
description: per-device IO usage stats
- name: pressure
type: group
Expand Down
1 change: 1 addition & 0 deletions metricbeat/module/system/process/process_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,6 @@ func getConfig() map[string]interface{} {
return map[string]interface{}{
"module": "system",
"metricsets": []string{"process"},
//"processes": []string{".*metricbeat.*"}, // in case we want a prettier looking example for data.json
}
}