Test failures on aiohttp 3.9.0

[polyzen]

Describe the bug

Tests involving HttpRequestParserC fail or are skipped, and test_unsupported_upgrade[pyloop] fails.

To Reproduce

Trying to package the new version for Arch Linux. The build bails after check() due to the test suite failures.

The build script currently:

# Maintainer: Levente Polyak <anthraxx[at]archlinux[dot]org>
# Maintainer: Jonas Witschel <[email protected]>
# Contributor: Philipp A. <[email protected]>

# Check if new updates break python-engineio

_pkgname=aiohttp
pkgname=python-aiohttp
_gitcommit=45b2c2c5773f0ee0d35fce8ff5716c78e91d9135
pkgver=3.9.0
pkgrel=1
pkgdesc='HTTP client/server for asyncio'
arch=('x86_64')
url='https://aiohttp.readthedocs.io'
license=('Apache')
depends=(
  'python'
  'python-aiosignal'
  'python-attrs'
  'python-frozenlist'
  'python-multidict'
  'python-yarl'
)
makedepends=(
  'cython'
  'git'
  'npm'
  'python-build'
  'python-installer'
  'python-setuptools'
  'python-wheel'
)
checkdepends=(
  'gunicorn'
  'python-aiodns'
  'python-brotli'
  'python-proxy.py'
  'python-pytest'
  'python-pytest-mock'
  'python-pytest-xdist'
  'python-re-assert'
  'python-time-machine'
  'python-trustme'
  'python-uvloop'
)
optdepends=(
  'gunicorn: to deploy using Gunicorn'
  'python-aiodns: for fast DNS resolving'
  'python-brotli: for Brotli transfer-encodings support'
)
source=(
  "${pkgname}::git+https://github.com/aio-libs/aiohttp#commit=${_gitcommit}"
  git+https://github.com/nodejs/llhttp.git
)
sha512sums=('SKIP'
            'SKIP')

pkgver() {
  cd ${pkgname}
  git describe --tags | sed 's/^v//;s/\([^-]*-g\)/r\1/;s/-/./g'
}

prepare() {
  cd ${pkgname}
  git submodule init
  git config submodule."vendor/llhttp".url "${srcdir}/llhttp"
  git -c protocol.file.allow=always submodule update --recursive
  sed 's|.install-cython ||' -i Makefile

  # This test calls the Python interpreter, we need to make sure that the path
  # for the C extensions is correct there as well
  sed -i "s/import {import_path!s}/import sys; sys.path.insert(0, '{os.environ['PYTHONPATH']}'); &/" \
      tests/test_circular_imports.py

  # Remove coverage testing
  sed -i '/--cov=/d' setup.cfg
}

build() {
  cd ${pkgname}
  make generate-llhttp cythonize
  python -m build --wheel --no-isolation
}

check() {
  cd ${pkgname}
  local _python_version=$(python -c 'import sys; print("".join(map(str, sys.version_info[:2])))')
  mv tests/autobahn/test_autobahn.py{,.bak} # Docker tests
  PYTHONPATH="$PWD/build/lib.linux-$CARCH-cpython-${_python_version}" pytest
  mv tests/autobahn/test_autobahn.py{.bak,}
}

package() {
  cd ${pkgname}
  python -m installer --destdir="$pkgdir" dist/*.whl
}

# vim: ts=2 sw=2 et:

Expected behavior

Happy test suite

Logs/tracebacks

=================================== FAILURES ===================================
_____________________________ test_c_parser_loaded _____________________________

    @pytest.mark.skipif(NO_EXTENSIONS, reason="Extensions available but not imported")
    def test_c_parser_loaded():
>       assert "HttpRequestParserC" in dir(aiohttp.http_parser)
E       AssertionError: assert 'HttpRequestParserC' in ['ASCIISET', 'Any', 'BadHttpMessage', 'BadStatusLine', 'BaseProtocol', 'BaseTimerContext', ...]
E        +  where ['ASCIISET', 'Any', 'BadHttpMessage', 'BadStatusLine', 'BaseProtocol', 'BaseTimerContext', ...] = dir(<module 'aiohttp.http_parser' from '/build/python-aiohttp/src/python-aiohttp/aiohttp/http_parser.py'>)
E        +    where <module 'aiohttp.http_parser' from '/build/python-aiohttp/src/python-aiohttp/aiohttp/http_parser.py'> = aiohttp.http_parser


tests/test_http_parser.py:102: AssertionError
________________________ test_invalid_character[pyloop] ________________________

loop = <_UnixSelectorEventLoop running=False closed=False debug=False>
protocol = <Mock id='139977382028240'>
request = <FixtureRequest for <Function test_invalid_character[pyloop]>>

    @pytest.mark.skipif(NO_EXTENSIONS, reason="Only tests C parser.")
    def test_invalid_character(loop: Any, protocol: Any, request: Any) -> None:
>       parser = HttpRequestParserC(
            protocol,
            loop,
            2**16,
            max_line_size=8190,
            max_field_size=8190,
        )
E       NameError: name 'HttpRequestParserC' is not defined

loop       = <_UnixSelectorEventLoop running=False closed=False debug=False>
protocol   = <Mock id='139977382028240'>
request    = <FixtureRequest for <Function test_invalid_character[pyloop]>>

tests/test_http_parser.py:128: NameError
________________________ test_invalid_linebreak[pyloop] ________________________

loop = <_UnixSelectorEventLoop running=False closed=False debug=False>
protocol = <Mock id='139977382709008'>
request = <FixtureRequest for <Function test_invalid_linebreak[pyloop]>>

    @pytest.mark.skipif(NO_EXTENSIONS, reason="Only tests C parser.")
    def test_invalid_linebreak(loop: Any, protocol: Any, request: Any) -> None:
>       parser = HttpRequestParserC(
            protocol,
            loop,
            2**16,
            max_line_size=8190,
            max_field_size=8190,
        )
E       NameError: name 'HttpRequestParserC' is not defined

loop       = <_UnixSelectorEventLoop running=False closed=False debug=False>
protocol   = <Mock id='139977382709008'>
request    = <FixtureRequest for <Function test_invalid_linebreak[pyloop]>>

tests/test_http_parser.py:148: NameError
_______________________ test_unsupported_upgrade[pyloop] _______________________
[XPASS(strict)] The behavior of C-extensions differs from pure-Python: https://github.com/aio-libs/aiohttp/issues/6446

Python Version

$ python --version
Python 3.11.5

aiohttp Version

3.9.0

multidict Version

$ pacman -Q python-multidict
python-multidict 6.0.4-2

yarl Version

$ pacman -Q python-yarl
python-yarl 1.9.2-1

OS

Arch Linux

Related component

Server, Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[Dreamsorcerer]

Well, it either failed to build or install the C parser.

It appears you have the steps to build it. But, when I try running the tests locally, I usually need to install the package for it to be able to import the C parser, it doesn't seem to work when run from the source directory (unless maybe, pip install -e . has been run, which is what our CI does).

[dotlambda]

I'm observing the exact same failures on NixOS.

[Dreamsorcerer]

I don't think there should be a difference from 3.8 to 3.9. I don't think there's any changes related to building or testing that should be relevant.

[dotlambda]

I ran git bisect which blames it on 9258d34. Indeed, reverting that commit fixes the tests for me.