ModuleNotFoundError: No module named 'pyuwsgi' (when reloading)

[markkuleinio]

For some reason uWSGI packaged by pyuwsgi is apparently not able to find the correct Python: it uses the system-default Python instead of the virtual environment. This is shown as "ModuleNotFoundError: No module named 'pyuwsgi'" error when reloading uWSGI.

Is there something that I'm doing incorrectly?

Steps to reproduce:

mkdir django-test-with-pyuwsgi
cd django-test-with-pyuwsgi
python3 -m venv venv
. venv/bin/activate
pip install -U pip wheel
pip install django pyuwsgi
django-admin startproject testproject
pip list
(output:
Package    Version
---------- ------------
asgiref    3.8.1
Django     5.0.6
pip        24.0
pyuwsgi    2.0.23.post0
setuptools 66.1.1
sqlparse   0.5.0
wheel      0.43.0
)

Create uwsgi.ini:

[uwsgi]
strict = true
master = true
vacuum = true
single-interpreter = true
http-socket = 127.0.0.1:8001
chdir = testproject
module = testproject.wsgi

Create /etc/systemd/system/testproject.service:

[Unit]
Description=Testproject uWSGI Service
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
NotifyAccess=all
Restart=always
RestartSec=30
# Replace the paths and user+group!
WorkingDirectory=/home/markku/devel/django-test-with-pyuwsgi
ExecStart=/home/markku/devel/django-test-with-pyuwsgi/venv/bin/uwsgi --ini uwsgi.ini
User=markku
Group=markku

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl start testproject
sudo journalctl -eu testproject

See the output (abbreviated):

May 09 11:26:30 devel uwsgi[225420]: [uWSGI] getting INI configuration from uwsgi.ini
May 09 11:26:30 devel uwsgi[225420]: *** Starting uWSGI 2.0.23 (64bit) on [Thu May  9 11:26:30 2024] ***
May 09 11:26:30 devel uwsgi[225420]: compiled with version: 10.2.1 20210130 (Red Hat 10.2.1-11) on 10 January 2024 21:25:28
May 09 11:26:30 devel uwsgi[225420]: os: Linux-6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01)
May 09 11:26:30 devel uwsgi[225420]: current working directory: /home/markku/devel/django-test-with-pyuwsgi
May 09 11:26:30 devel uwsgi[225420]: detected binary path: /usr/bin/python3.11
May 09 11:26:30 devel uwsgi[225420]: chdir() to testproject
May 09 11:26:30 devel systemd[1]: Started testproject.service - Testproject uWSGI Service.

Test the Django project:

$ curl -s localhost:8001 | grep success
        <title>The install worked successfully! Congratulations!</title>
        <h1>The install worked successfully! Congratulations!</h1>

Then, send SIGTERM to the service with "sudo kill 225420" (using the correct PID) and see journal log again:

May 09 11:29:08 devel uwsgi[225420]: ...brutally killing workers...
May 09 11:29:08 devel systemd[1]: Stopping testproject.service - Testproject uWSGI Service...
May 09 11:29:09 devel uwsgi[225420]: worker 1 buried after 1 seconds
May 09 11:29:09 devel uwsgi[225420]: binary reloading uWSGI...
May 09 11:29:09 devel uwsgi[225420]: chdir() to /home/markku/devel/django-test-with-pyuwsgi
May 09 11:29:09 devel uwsgi[225420]: closing all non-uwsgi socket fds > 2 (max_fd = 1024)...
May 09 11:29:09 devel uwsgi[225420]: found fd 4 mapped to socket 0 (127.0.0.1:8001)
May 09 11:29:09 devel uwsgi[225420]: running /usr/bin/python3.11
May 09 11:29:09 devel uwsgi[225420]: Traceback (most recent call last):
May 09 11:29:09 devel uwsgi[225420]:   File "/home/markku/devel/django-test-with-pyuwsgi/venv/bin/uwsgi", line 5, in <module>
May 09 11:29:09 devel uwsgi[225420]:     from pyuwsgi import run
May 09 11:29:09 devel uwsgi[225420]: ModuleNotFoundError: No module named 'pyuwsgi'
May 09 11:29:09 devel systemd[1]: testproject.service: Main process exited, code=exited, status=1/FAILURE
May 09 11:29:09 devel systemd[1]: testproject.service: Failed with result 'exit-code'.
May 09 11:29:09 devel systemd[1]: Stopped testproject.service - Testproject uWSGI Service.

(Yes, "reloading" is expected as uWSGI by default considers SIGTERM as a reload signal, not terminate signal.)

= ModuleNotFoundError: No module named 'pyuwsgi', reload is not successful.

Note the logs above:

(when starting)
detected binary path: /usr/bin/python3.11
(and when reloading)
running /usr/bin/python3.11

even though uWSGI was installed and started successfully in the virtual environment.

$ cat venv/bin/uwsgi
#!/home/markku/devel/django-test-with-pyuwsgi/venv/bin/python3
# -*- coding: utf-8 -*-
import re
import sys
from pyuwsgi import run
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(run())

About pyuwsgi vs uwsgi:

If pyuwsgi is replaced by uwsgi (currently uWSGI==2.0.25.1), the path to the uWSGI binary is correctly found and reload is working:

(when starting)
detected binary path: /home/markku/devel/django-test-with-pyuwsgi/venv/bin/uwsgi
(and when reloading)
running /home/markku/devel/django-test-with-pyuwsgi/venv/bin/uwsgi

[markkuleinio]

Oh well, this is duplicate of unbit/uwsgi#2478

[markkuleinio]

Workaround in my example case (where WorkingDirectory was set in systemd):

binary-path = venv/bin/python

Note: This is neither required nor working when using uwsgi instead of pyuwsgi.

[markkuleinio]

I take it that this really is a uwsgi issue and not pyuwsgi issue, so closing this.