Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add automaxprocs #2724

Merged
merged 4 commits into from
Mar 21, 2020
Merged

Add automaxprocs #2724

merged 4 commits into from
Mar 21, 2020

Conversation

anyasabo
Copy link
Contributor

Follow up to #2720

Automatically sets GOMAXPROCS to the container cpu limit if it exists. Otherwise it defaults to the number of CPUs on the host. Our current default cpu limit is 1, so on big hosts golang may use many more processes than we can actually use.

Even if we do end up removing the default cpu limit in our default configuration, I think I would be in favor of keeping this to better support users who do end up setting a cpu limit.

Some further reading:
Benchmarks: uber-go/automaxprocs#12
And a thread with a lot more references: https://threadreaderapp.com/thread/1149654812595646471.html

@anyasabo anyasabo added >enhancement Enhancement of existing functionality v1.1.0 labels Mar 18, 2020
@anyasabo anyasabo mentioned this pull request Mar 18, 2020
@@ -175,6 +176,13 @@ func init() {
}

func execute() {
// update GOMAXPROCS to container cpu limit if necessary
_, err := maxprocs.Set(maxprocs.Logger(log.Info))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this library uses a package init function that runs as soon as you import the package

import _ "go.uber.org/automaxprocs"

Are we not setting it twice now (not sure if it matters much tbh)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Importing go.uber.org/automaxprocs triggers the init. Importing go.uber.org/automaxprocs/maxprocs as Anya has done here should not trigger it (in theory).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have explained this better in the PR. The package docs describe the use case a little bit better:

// Package maxprocs lets Go programs easily configure runtime.GOMAXPROCS to
// match the configured Linux CPU quota. Unlike the top-level automaxprocs
// package, it lets the caller configure logging and handle errors.

So in our case it's just so we can inject our logger.

Copy link
Contributor

@charith-elastic charith-elastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -175,6 +176,13 @@ func init() {
}

func execute() {
// update GOMAXPROCS to container cpu limit if necessary
_, err := maxprocs.Set(maxprocs.Logger(log.Info))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@anyasabo
Copy link
Contributor Author

jenkins test this please

@anyasabo anyasabo merged commit 25e5387 into elastic:master Mar 21, 2020
@anyasabo anyasabo deleted the maxprocs branch March 21, 2020 03:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement Enhancement of existing functionality v1.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants