Subprocess returncode is not detected when running Gunicorn with aiohttp

[harrylyx]

Describe the bug

When starting Gunicorn with aiohttp worker(s), if the app uses subprocess to start other processes and captures the output, their returncode is in most cases 0, even if the actual exit code was 1.

To Reproduce

import asyncio
import subprocess
from aiohttp import web
from concurrent.futures import ProcessPoolExecutor

EXECUTOR = ProcessPoolExecutor(4)

def create():
    subprocess_run = subprocess.run(
        "pip install numpy==123",
        shell=True,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
    )
    print(subprocess_run.returncode)
    print(subprocess_run.stderr.decode("utf-8"))
    return f"{subprocess_run.returncode}, {subprocess_run.stderr.decode('utf-8')}"


async def index(request):
    loop = asyncio.get_event_loop()
    return web.Response(text=await loop.run_in_executor(EXECUTOR, create))


async def my_web_app():
    app = web.Application()
    app.router.add_get("/", index)
    return app


if __name__ == "__main__":
    web.run_app(my_web_app())

Then run it with:

gunicorn start:my_web_app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker

Expected behavior

The detected returncode should always be 1, as the subprocess always exits with 1.

Logs/tracebacks

❯ gunicorn start:my_web_app --bind localhost:8080 --worker-class aiohttp.GunicornWebWorker
[2021-10-25 16:53:27 +0800] [4190] [INFO] Starting gunicorn 20.1.0
[2021-10-25 16:53:27 +0800] [4190] [INFO] Listening at: http://127.0.0.1:8080 (4190)
[2021-10-25 16:53:27 +0800] [4190] [INFO] Using worker: aiohttp.GunicornWebWorker
[2021-10-25 16:53:27 +0800] [4193] [INFO] Booting worker with pid: 4193
0
ERROR: Could not find a version that satisfies the requirement numpy==123 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0rc1, 1.13.0rc2, 1.13.0, 1.13.1, 1.13.3, 1.14.0rc1, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0rc1, 1.15.0rc2, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0rc1, 1.16.0rc2, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0rc1, 1.17.0rc2, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0rc1, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0rc1, 1.19.0rc2, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0rc1, 1.20.0rc2, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0rc1, 1.21.0rc2, 1.21.0, 1.21.1, 1.21.2, 1.21.3)
ERROR: No matching distribution found for numpy==123

Python Version

$ python --version
Python 3.7.10

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.7.4.post0
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: Nikolay Kim
Author-email: [email protected]
License: Apache 2
Location: /Users/wj/tools/anaconda3/envs/py37/lib/python3.7/site-packages
Requires: async-timeout, attrs, chardet, multidict, typing-extensions, yarl
Required-by: pytest-aiohttp

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 5.2.0
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /Users/wj/tools/anaconda3/envs/py37/lib/python3.7/site-packages
Requires: 
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.7.0
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: [email protected]
License: Apache 2
Location: /Users/wj/tools/anaconda3/envs/py37/lib/python3.7/site-packages
Requires: idna, multidict, typing-extensions
Required-by: aiohttp

OS

macOS Big Sur 11.5.2 (20G95)

Related component

Server, Client

Additional context

You can refer to the same question
encode/uvicorn#894
encode/uvicorn#895

I guess bug in https://github.com/aio-libs/aiohttp/blob/master/aiohttp/worker.py

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[NAnand3200]

i am still seeing this issue. is this fixed ?

[Dreamsorcerer]

The PR to fix the issue is by the same person who opened the issue, so they clearly considered it fixed. Feel free to open a new issue with details of what your seeing (creating a PR with a test reproducing the problem would be best).

Edit: You already did.