Switch all services to use UWSGI

[mnaser]

At the moment, the services are running the (very) inefficient version of their eventlet API endpoint.

We should flip over to uWSGI (or something using mod_wsgi) in order to be able to have a proper HTTP server that can scale to the demand.

[mnaser]

This has become more of an issue, with more recent and larger clouds hitting issues.

• Nova has deprecated services running under eventlet
• DevStack has long moved towards uWSGI

In terms of comparing other deployments tools:

• OpenStack-Ansible: uWSGI
• Kolla: mod_wsgi
• TripleO: mod_wsgi

So the favour seems to be towards mod_wsgi, another neat thing about mod_wsgi is that Horizon and Keystone technically use it (and Keystone requires it because of the SSO things)... so I think we have to move towards mod_wsgi I believe...

[mnaser]

nova:

[uwsgi]
chmod-socket = 666
socket = /var/run/uwsgi/nova-api-wsgi.socket
lazy-apps = true
add-header = Connection: close
buffer-size = 65535
hook-master-start = unix_signal:15 gracefully_kill_them_all
thunder-lock = true
plugins = http,python3
enable-threads = true
worker-reload-mercy = 80
exit-on-reload = false
die-on-term = true
master = true
processes = 8
wsgi-file = /opt/stack/data/venv/bin/nova-api-wsgi

[mnaser]

I started a very simple layout here:

#596

If this works, I'll merge it for now since we actually need to land support for Nova due to some issues, but we should do this for all other services:

• Build latest images so uWSGI with pcre support is included
• Pin latest images with PCRE support
• Follow same format as other PR to add support for other services
• Tune the health check timeouts since uWSGI is much faster

It should be pretty straight forward and just a process of going over all of them one by one.

[okozachenko1203]

neutron has no wsgi file for neutron server. Just neutron-api file but it is using eventlet.

#PBR Generated from 'wsgi_scripts'

import threading

from neutron.cmd.eventlet.server import main_api_eventlet

if __name__ == "__main__":
    import argparse
    import socket
    import sys
    import wsgiref.simple_server as wss

    parser = argparse.ArgumentParser(
        description=main_api_eventlet.__doc__,
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        usage='%(prog)s [-h] [--port PORT] [--host IP] -- [passed options]')
    parser.add_argument('--port', '-p', type=int, default=8000,
                        help='TCP port to listen on')
    parser.add_argument('--host', '-b', default='',
                        help='IP to bind the server to')
    parser.add_argument('args',
                        nargs=argparse.REMAINDER,
                        metavar='-- [passed options]',
                        help="'--' is the separator of the arguments used "
                        "to start the WSGI server and the arguments passed "
                        "to the WSGI application.")
    args = parser.parse_args()
    if args.args:
        if args.args[0] == '--':
            args.args.pop(0)
        else:
            parser.error("unrecognized arguments: %s" % ' '.join(args.args))
    sys.argv[1:] = args.args
    server = wss.make_server(args.host, args.port, main_api_eventlet())

    print("*" * 80)
    print("STARTING test server neutron.cmd.eventlet.server.main_api_eventlet")
    url = "http://%s:%d/" % (server.server_name, server.server_port)
    print("Available at %s" % url)
    print("DANGER! For testing only, do not use in production")
    print("*" * 80)
    sys.stdout.flush()

    server.serve_forever()
else:
    application = None
    app_lock = threading.Lock()

    with app_lock:
        if application is None:
            application = main_api_eventlet()

[mnaser]

I believe Neutron does ship with a WSGI. Also, I think we missed Barbican.

[ricolin]

tracking community patch: https://review.opendev.org/q/topic:%22use-uwsgi%22

[ricolin]

a relate issue regarding neutron-server+uwsgi https://bugs.launchpad.net/neutron/+bug/2062009

[ricolin]

Note: ovn+wsgi in neutron seems still not there https://bugs.launchpad.net/neutron/+bug/1912359

[ricolin]

All services should now moved to uWSGI in atmosphere, I think we can close this issue now

[mnaser]

Yeah. I think the Horizon bits would be nice to move.

For Keystone, we can maybe try to find some sidecar process that takes care of OIDC

[ricolin]

For neutron+ovn we now can add neutron-ovn-maintenance-worker to support WSGI