Feature: Add M3 interface

[davidjwbbc]

This feature adds an M3 API interface to the 5GMS Application Server.

The 5GMS Application Server no longer accepts a Content Hosting Configuration (and optional Certificates) JSON file as command line parameter. Instead this information is provided by an M3 client, i.e. the 5GMS Application Function. This requires the 5GMS Application Function to use the M3 API to configure the Application Server (see PR 5G-MAG/rt-5gms-application-function#23).

Testing with the Application Function

1. Install both the 5GMS Application Server and Application Function.
2. Configure the Application Function with the M3 port number of the Application Server (i.e. 7777) in the msaf.applicationServers section of the msaf.yaml file.

msaf:
    ...
    applicationServers:
      - canonicalHostname: localhost
        urlPathPrefixFormat: /m4d/provisioning-session-{provisioningSessionId}/
        m3Port: 7777
    ...

3. Start the Application Server.
4. Start the Application Function.

The Application Function should then configure the Application Server using the Server Certificates and Content Hosting Configuration it has been configured with. This should be evident from log messages from the Application Server and Application Function and from the /tmp/rt_5gms_as.conf NGINX configuration.

See the 5GMS Application Function documentation for more details on configuring it with Certificates and a Content Hosting Configuration.

Testing without the Application Function

To enable debugging and testing, a simple M3 client command can be found the the tests subdirectory:

Usage:
    m3_client_cli.py -h | --help
    m3_client_cli.py -c | --certificate <connect>
    m3_client_cli.py -c | --certificate <connect> (add|update) <certificate-id> <pem-file>
    m3_client_cli.py -c | --certificate <connect> delete <certificate-id>
    m3_client_cli.py -H | --content-hosting-configuration <connect>
    m3_client_cli.py -H | --content-hosting-configuration <connect> (add|update) <provisioning-session-id> <content-hosting-configuration-json-file>
    m3_client_cli.py -H | --content-hosting-configuration <connect> delete <provisioning-session-id>
    m3_client_cli.py -H | --content-hosting-configuration <connect> purge <provisioning-session-id> [<pattern>]

Parameters:
    connect                  Hostname:Port of the server providing M3.
    provisioning-session-id  Provisioning Session Identifier.
    certificate-id           Certificate Identifier.
    pem-file                 Server PEM format X.509 public certificate, private key and intermediate CA certificates.
    content-hosting-configuration-json-file
                             Filename of a ContentHostingConfiguration in JSON format.
    pattern                  Regular expression to match the cache entry URL paths to delete.

Options:
    -h --help                Display the command help
    -v --version             Display command version
    -c --certificate         List known certificates or perform a certificate operation.
    -H --content-hosting-configuration
                             List known ContentHostingConfigurations or perform an operation on ContentHostingConfigurations.

Application Server setup and running for testing

Follow the instructions from the README for the "Install dependencies"

Then setup a test environment with an install of the application server:

cd ~
git clone --recurse-submodules -b feature-add-m3-interface https://github.com/davidjwbbc/rt-5gms-application-server.git
cd ~/rt-5gms-application-server
python3 -m venv venv
venv/bin/python3 -m pip install --upgrade pip setuptools
venv/bin/python3 -m pip install .
cat > local-dev.conf <<EOF
### Defaults for the 5G-MAG Reference Tools: 5GMS applications
[DEFAULT]
log_dir = /tmp/rt-5gms-as/logs
run_dir = /tmp/rt-5gms-as

### 5GMS Application Server specific configurations
[5gms_as]
log_level = debug
cache_dir = /tmp/rt-5gms-as/cache
certificates_cache = /tmp/rt-5gms-as/certificates
http_port = 8080
https_port = 8443
m3_listen = localhost
m3_port = 7777
#access_log = %(log_dir)s/application-server-access.log
#error_log = %(log_dir)s/application-server-error.log
#pid_path = %(run_dir)s/application-server.pid

### 5GMS Application Server nginx specific configuration
[5gms_as.nginx]
#root_temp = /var/cache/rt-5gms/as
root_temp = /tmp/rt-5gms-as
#client_body_temp = %(root_temp)s/client-body-tmp
#proxy_temp = %(root_temp)s/proxy-tmp
#fastcgi_temp = %(root_temp)s/fastcgi-tmp
#uwsgi_temp = %(root_temp)s/uwsgi-tmp
#scgi_temp = %(root_temp)s/scgi-tmp
#pid_path = %(root_temp)s/5gms-as-nginx.pid
EOF
venv/bin/5gms-application-server -c local-dev.conf

To configure a simple HTTP Application Server

cd ~/rt-5gms-application-server
tests/m3_client_cli.py -H localhost:7777 add ps1 external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest.json

This should respond with a "Success!" message, and NGINX should now be running on port 8080 using the example Big Buck Bunny configuration. You can check the NGINX configuration in /tmp/rt_5gms_as.conf.

To configure an HTTPS Application Server

This requires that the server certificate is pushed to the Application Server before the content hosting configuration is.

To generate server certificates, ensure that openssl is installed (e.g. apt -y install openssl), and then:

cd ~/rt-5gms-application-server
external/rt-common-shared/5gms/scripts/make_self_signed_certs.py external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest_https.json external/rt-common-shared/5gms/examples/Certificates.json

The 5GMS Application Server stores the certificates it has been configured with in a certificates cache. This cache is reloaded when the Application Server starts up, so it will remember certificates from previous runs.

The 5GMS Application Server can be checked for what certificates it already has by using the command:

cd ~/rt-5gms-application-server
tests/m3_client_cli.py -c localhost:7777

To push a new certificate (with id "testcert1" using the generated certificate file):

cd ~/rt-5gms-application-server
tests/m3_client_cli.py -c localhost:7777 add testcert1 external/rt-common-shared/5gms/examples/certificate-1.pem

...or to update an existing certificate:

cd ~/rt-5gms-application-server
tests/m3_client_cli.py -c localhost:7777 update testcert1 external/rt-common-shared/5gms/examples/certificate-1.pem

Now the Content Hosting Configuration can be pushed:

cd ~/rt-5gms-application-server
tests/m3_client_cli.py -H localhost:7777 add ps1 external/rt-common-shared/5gms/examples/ContentHostingConfiguration_Big-Buck-Bunny_pull-ingest_https.json

This should result in "Success!" and NGINX will now be listening on "https://localhost:8443/...".

[rjb1000]

The testing procedure in the description of this PR could usefully be added to the Application Server documentation.

[rjb1000]

Thanks for the explanations, @davidjwbbc.

And thanks for the updates.