Possibly unnecessary polling in gthread

[ankush]

I was tracing system calls for some unrelated reason and observed that an idle gthread worker makes epoll_wait system call every second.

λ sudo strace -p `pgrep -f "gunicorn: worker" | head -n1`
strace: Process 30815 attached
epoll_wait(7, [], 1, 666)               = 0
getppid()                               = 30800
utimensat(6, NULL, [{tv_sec=3157, tv_nsec=198136276} /* 1970-01-01T06:22:37.198136276+0530 */, {tv_sec=3157, tv_nsec=198136276} /* 1970-01-01T06:22:37.198136276+0530 */], 0) = 0
epoll_wait(7, [], 1, 1000)              = 0
getppid()                               = 30800
utimensat(6, NULL, [{tv_sec=3158, tv_nsec=204192934} /* 1970-01-01T06:22:38.204192934+0530 */, {tv_sec=3158, tv_nsec=204192934} /* 1970-01-01T06:22:38.204192934+0530 */], 0) = 0
epoll_wait(7, [], 1, 1000)              = 0
getppid()                               = 30800
utimensat(6, NULL, [{tv_sec=3159, tv_nsec=210145196} /* 1970-01-01T06:22:39.210145196+0530 */, {tv_sec=3159, tv_nsec=210145196} /* 1970-01-01T06:22:39.210145196+0530 */], 0) = 0
epoll_wait(7, [], 1, 1000)              = 0
getppid()                               = 30800
utimensat(6, NULL, [{tv_sec=3160, tv_nsec=215517372} /* 1970-01-01T06:22:40.215517372+0530 */, {tv_sec=3160, tv_nsec=215517372} /* 1970-01-01T06:22:40.215517372+0530 */], 0) = 0
epoll_wait(7, ^Cstrace: Process 30815 detached
 <detached ...>

Here is the relevant code:

gunicorn/gunicorn/workers/gthread.py

Line 210 in bacbf8a

events = self.poller.select(1.0)

This poller is used for waking up process to:

1. Accept incoming connections -

   gunicorn/gunicorn/workers/gthread.py

   Line 201 in bacbf8a

   self.poller.register(sock, selectors.EVENT_READ, acceptor)

2. Read incoming data -

   gunicorn/gunicorn/workers/gthread.py

   Lines 261 to 262 in bacbf8a

   	self.poller.register(conn.sock, selectors.EVENT_READ,
   	partial(self.on_client_socket_readable, conn))

Since same poller is used for both, there's no question of delays induced by blocking epoll call... and if blocking was indeed happening then 1 second blocking is anyways unacceptable.

This doesn't happen for sync worker which has effective timeout same as request timeout. That too is only done to let master process know worker is still alive.

gunicorn/gunicorn/workers/sync.py

Line 36 in bacbf8a

ret = select.select(self.wait_fds, [], [], timeout)

My question is why the timeout needs to be so short? This short timeout causes the process to wake up every second even though nothing has happened.

---

Is this done to clean up futures frequently?

gunicorn/gunicorn/workers/gthread.py

Lines 216 to 217 in bacbf8a

	result = futures.wait(self.futures, timeout=0,
	return_when=futures.FIRST_COMPLETED)

[ankush]

Oof, I meant to post it in https://github.com/benoitc/gunicorn/discussions 😐

Anyway, this is kind of somewhere in the middle.