Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto-tune worker_processes ? #31

Closed
nightlyone opened this issue Aug 5, 2015 · 4 comments
Closed

Auto-tune worker_processes ? #31

nightlyone opened this issue Aug 5, 2015 · 4 comments

Comments

@nightlyone
Copy link

Since nginx can auto-tune now the worker_processes configuration parameter, I would suggest to use
worker_processes auto; instead of worker_processes 1; in /etc/nginx/nginx.conf

Rationale: Docker manages resources using container restrictions and expects them to automatically adapt to the provided amount of CPU and memory.

But for non-docker environments, using only one thread is a safe default. So upstream developers provide it that way.

@thresheek
Copy link
Collaborator

"worker_processes auto;" won't work as you think it should inside cgroups-controlled environment (such as docker or lxc or whatelse).

nginx relies on sysconf(_SC_NPROCESSORS_ONLN) call to determine the number of available CPUs to spawn the workers accordingly. Unfortunately in cgroups this does not work and the number of CPUs is not changing whether you define a cpu subset or not.

Easily checked:

[thresh@centos7-tests]% docker run -d nginx
7a0abb4e7f61c1701a3cef02eaef255fc6272415d718de11d7fe3ecde597b3cd
[thresh@centos7-tests]
% docker exec -t 7a0abb4e7f61c1701a3cef02eaef255fc6272415d718de11d7fe3ecde597b3cd getconf _NPROCESSORS_ONLN
2
[thresh@centos7-tests]% docker run -d --cpuset-cpus=0 nginx
f645a6814270d6c6e5c8e7d491a99ec39aa9c4872f8a1eefe5812744ea2676b8
[thresh@centos7-tests]
% docker exec -t f645a6814270d6c6e5c8e7d491a99ec39aa9c4872f8a1eefe5812744ea2676b8 getconf _NPROCESSORS_ONLN
2
[thresh@centos7-tests]% docker run -d --cpuset-cpus=1 nginx
e8c46ef776269fb0f67b29d4413b957745b1032132a982f18fdeb331d6b02393
[thresh@centos7-tests]
% docker exec -t e8c46ef776269fb0f67b29d4413b957745b1032132a982f18fdeb331d6b02393 getconf _NPROCESSORS_ONLN
2
[thresh@centos7-tests]% docker run -d --cpuset-cpus="0,1" nginx
615edf33aeb9a6c83c6ccff11711d8b3ce73a4818e28b683171450cd03406005
[thresh@centos7-tests]
% docker exec -t 615edf33aeb9a6c83c6ccff11711d8b3ce73a4818e28b683171450cd03406005 getconf _NPROCESSORS_ONLN
2

@nightlyone
Copy link
Author

Thanks for the answer! Any other way to auto-tune it then?

@thresheek
Copy link
Collaborator

For now you can generate nginx configuration based on how many CPUs you want to throw at it.

A better solution would be providing a patch for nginx so "worker_processes auto;" will take cpusets its currently bound to into account.

trajano pushed a commit to trajano/nginx-letsencrypt that referenced this issue May 10, 2018
Addresses the fact that nginx uses 1 worker process by default as `auto` will not work correctly in a docker environment.  nginxinc/docker-nginx#31
pdbrito added a commit to pdbrito/laravel-cloud-run that referenced this issue Aug 30, 2019
Apparently using auto doesn't work well with docker:
nginxinc/docker-nginx#31
@dman777
Copy link

dman777 commented Oct 23, 2019

For this reason is it better not to use nginx in a container? I see in /proc/cpu the docker container only has 1 cpu

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants