HLS stats

[caf2022]

Hi everyone, someone have any solution to get stats from HLS listeners?

[JonathanThorpe]

The challenge with this is that there aren't necessarily persistent connections from a client with HLS.

This might be bit of a convoluted way to do this as I don't really want listeners connecting directly to LiquidSoap:

1. Having Liquidsoap generate my HLS stream and save it on a local tmpfs filesystem.
2. On the same machine in a different container, nginx then serves out of LiquidSoap's HLS tmpfs filesystem
3. A second upstream nginx server runs as the actual endpoint to which listeners connect and also caches the fragments and playlist.

On the second nginx server, I have it configured to send syslogs over a TCP connection to a Python script which then generates Prometheus metrics.

The way I ensure uniqueness per listener is to to handle connections to the m3u8 and use nginx's sub_filter to change the content of the file to include a unique ID. For example (on the caching nginx server that listeners connect to):

location ~ ^/hls/(station_1_name|station_2_name)/live/(.*\.m3u8)$ {
      proxy_cache hls;
      proxy_cache_key $request_uri;
      proxy_cache_valid 200 4s;
      proxy_cache_lock on;
      proxy_cache_lock_timeout 4s;
      proxy_cache_lock_age 4s;
      proxy_pass http://hls_backend/hls/$1/live/$2;
      include conf.d/includes/hls_proxy_params.conf;
      proxy_set_header Accept-Encoding "";
      auth_basic off;

      types {
          application/vnd.apple.mpegurl m3u8;
      }

      sub_filter_once off;
      sub_filter_types 'application/vnd.apple.mpegurl';
      sub_filter 'aac_lofi.m3u8' '$request_id/aac_lofi.m3u8';
      sub_filter 'aac_midfi.m3u8' '$request_id/aac_midfi.m3u8';
      sub_filter 'aac_hifi.m3u8' '$request_id/aac_hifi.m3u8';  
    }

This gives me something unique to regex against in the Python syslog processor which allows capturing of concurrent listeners etc.

[caf2022]

This is a great idea, add a unique ID for each user then is more easy to control this with the database to add or update listen time, thanks a lot for sharing this!