Uppercase header is incompatible with ab (Apache Bench)

[iceboy233]

Long story short

ab hangs when keep-alive enabled and the response is long.

Expected behaviour

from aiohttp import web

async def handle(request):
    return web.Response(body=b'x' * 16384)

app = web.Application()
app.router.add_route('GET', '/', handle)
web.run_app(app)

Expect ab -k -n 2 http://127.0.0.1:8080/ to work.

Actual behaviour

ab hangs after sending the second request.

Cause and suggestion

ab fails to parse CONTENT-LENGTH. From the source code of ab we can see that there is some hand-craft HTTP parser (httpd/support/ab.c:read_connection):

if (keepalive &&
(strstr(c->cbuff, "Keep-Alive")
 || strstr(c->cbuff, "keep-alive"))) {  /* for benefit of MSIIS */
    char *cl;
    cl = strstr(c->cbuff, "Content-Length:");
    /* handle NCSA, which sends Content-length: */
    if (!cl)
        cl = strstr(c->cbuff, "Content-length:");
    if (cl) {
        c->keepalive = 1;
        /* response to HEAD doesn't have entity body */
        c->length = method != HEAD ? atoi(cl + 16) : 0;
    }
    /* The response may not have a Content-Length header */
    if (!cl) {
        c->keepalive = 1;
        c->length = 0;
    }
}

Although it's ab that violates the HTTP standard, considering that ab is a common tool for HTTP benchmarking, the behavior that ab works on other HTTP library and only hangs on aiohttp will frustrate users. Making HTTP headers camel case also have other benefits discussed in #223, #813.

Steps to reproduce

Install aiohttp and ab, and use the above example program and command line.

Your environment

Linux iceboy-m7h 4.6.2-1-ARCH #1 SMP PREEMPT Wed Jun 8 08:40:59 CEST 2016 x86_64 GNU/Linux
Python 3.5.1
aiohttp==0.21.6
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>

[ludovic-gasc]

A patch for ab should be easier to implement than change all aiohttp behaviour.
BTW, you should use wrk and/or wrk2: There are more interesting than ab to saturate an aiohttp application.
Or use an efficient reverse proxy like Nginx is also a workaround: You'll saturate aiohttp before Nginx and Nginx override theses headers.

[asvetlov]

Fixed by #1015

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.