New component: cgroup aware go runtime extension

[tomershafir]

The purpose and use-cases of the new component

Set go runtime variables based on linux cgroupfs automatically, or let the user set a cgroup relative value. For example, set GOMAXPROCS and GOMEMLIMIT by importing https://github.com/uber-go/automaxprocs and https://github.com/KimMachineGun/automemlimit.

Example configuration for the component

Proxy config for https://github.com/uber-go/automaxprocs and https://github.com/KimMachineGun/automemlimit

Telemetry data types supported

It is data type independent

Is this a vendor-specific component?

• This is a vendor-specific component
• If this is a vendor-specific component, I am proposing to contribute and support it as a representative of the vendor.

Code Owner(s)

No response

Sponsor (optional)

@mx-psi

Additional context

I think about a single extension that should theoretically cover the entire go runtime, starting with the 2 variables mentioned above.

Core issue ref: open-telemetry/opentelemetry-collector#9203

[mx-psi]

cc @open-telemetry/helm-maintainers would love to know what you think (is this something we would want on the Helm chart? If so, anything specific about the design we should take into account?)

[TylerHelmuth]

Seems reasonable to me. In the helm chart an extension like this would allow us to remove some templates like https://github.com/open-telemetry/opentelemetry-helm-charts/blob/main/charts/opentelemetry-collector/templates/_helpers.tpl#L169. A solution like this also helps users running the collector outside of kubernetes.

I'd like to see some example configs of how the extension would be configured.

[tomershafir]

maybe something like:

extensions:
  cgroupgoruntime:
    gomaxprocs: # hardcoded percentage 100
      enable: true
    gomemlimit:
      percentage: 85 # default 90, enabled by value > 0

[TylerHelmuth]

Would gomaxprocs or gomemlimit ever have additional fields in their sections?

[tomershafir]

I assume that the linux and go relevant interfaces should be stable for long, so I guess no. There may be additional sections.

[cforce]

Setting gomaxprocs as env var does not reduce core usage as expected .

[iblancasa]

It would be interesting to have this extension. Recently, Prometheus added automatic memory limit handling based on the same mechanism.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[tomershafir]

I wont be able to work on this issue. @mx-psi do you have someone to assign?

[mx-psi]

@tomershafir no, but I can remove the Stale marker and leave it open for now in case somebody else volunteers

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

[rogercoll]

@tomershafir @mx-psi I can help with the implementation of this extension. I have started a PoC that is working as expected, but it completely relies on the used packages: #35472

• https://github.com/uber-go/automaxprocs
• https://github.com/KimMachineGun/automemlimit

My main concern is regarding the testing, after taking a look at the packages, they seem not to provide an easy way to mock the cgroups file system. A mock of the cgroup file system would be needed for a full coverage unit test. Another option would be an integration test by manually creating a cgroup and launch the test in it, we could leverage the https://github.com/containerd/cgroups package to do so. This last option requires root privileges, do you know if we have any tests in the opentelemetry-collector that uses/require a root user?

[mx-psi]

Just to clarify: I am still willing to sponsor this component :)