PLCTestbench

PLCTestbench is a companion tool for researchers and developers working on Packet Loss Concealment (PLC). It greatly simplifies the process of measuring the reconstruction quality of PLC algorithms and allows to easily test the effects of different packet loss models and distributions.

It features the implementation of some of the most common packet loss models, PLC algorithms and metrics:

Packet Loss Simulation

• Binomial: uniform distribution of packet losses, governed by the Packet Error Ratio (PER) parameter.
• Metronome: periodic packet losses governed by the burst period, the burst length, and an offset.
• Gilbert-Elliot: bursty distribution of packet losses, governed by the four probabilities associated to its two states (For each state, the probability of packet loss and the probability of transitioning to the other state) [1].

PLC Algorithms

• Zeros: the lost samples are replaced by zeros.
• Last Packet: the lost samples are replaced by the last received packet.
• Low-Cost: implementation of the algorithm proposed in [2].
• Burg: Python bindings for the C++ implementation of the Burg method.
• Deep Learning: implementation of the algorithm proposed in [3].
• External: Python bindings for C++ to simplify the integration of existing algorithms.
• Advanced: allows to apply different PLC algorithms to different frequency bands and audio channels (M/S processing included).

Metrics

• Mean Square Error: the mean square error between the original and reconstructed signal.
• PEAQ: the Perceptual Evaluation of Audio Quality (PEAQ) metric, as defined in [4].
• Human: this metric produces the config file for a MUSHRA test using as stimuli excerpts of the reconstructed audio tracks. It also gathers the results of the test to be displayed alongside the other metrics.

Installation

You will need a mongoDB database to store the results. You can install it locally or use a cloud service like MongoDB Atlas. It is recomended however to use the Docker image provided by MongoDB.

Pull the image

    docker pull mongo:6.0.8

Then run the container setting the port to 27017 and the name to mongodb. Also set the username and password for the database.

    docker run -d -p 27017:27017 --name mongodb \
    -e MONGO_INITDB_ROOT_USERNAME=myUserAdmin \
    -e MONGO_INITDB_ROOT_PASSWORD=admin \
    mongo:6.0.8

Clone this repository, install the requirements and the plctestbench package:

    git clone https://github.com/LucaVignati/plc-testbench.git
    cd plc-testbench
    pip install -r requirements.txt
    pip install .
    cd ..

If you want to use it inside Jupyter Notebook you also need to install the ipywidgets package:

    pip install ipywidgets

Clone and install the burg-python-bindings:

    git clone https://github.com/LucaVignati/burg-python-bindings.git
    cd burg-python-bindings
    python setup.py install
    cd ..

Clone and install the cpp_plc_template:

    git clone https://github.com/LucaVignati/cpp_plc_template.git
    cd cpp_plc_template
    python setup.py install
    cd ..

If you want to use the PEAQ metric, you also need to install the GSTREAMER library and the PEAQ plugin:

    sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio git gtk-doc-tools git2cl automake libtool
    mkdir gstpeaq && git clone https://github.com/HSU-ANT/gstpeaq.git gstpeaq
    cd gstpeaq
    aclocal && autoheader && ./autogen.sh && sed -i 's/SUBDIRS = src doc/SUBDIRS = src/' Makefile.am && ./configure --libdir=/usr/lib && automake && make && make install
    cd ..

If you want to use the HumanCalculator metric, you also need to install webMUSHRA and pyMUSHRA:

    mkdir listening_tests && cd listening_tests
    mkdir db
    git clone https://github.com/audiolabs/webMUSHRA.git webmushra
    git clone https://github.com/nils-werner/pymushra.git pymushra
    pip install -e pymushra
    cd ..

Basic Usage

The file plctestbench.ipynb contains a Jupyter Notebook with a basic example of how to use the tool.

Input the settings of the testbench as follows:

    testbench_settings = {
        'root_folder': 'path/to/root/folder',
        'db_ip': 'ip.of.the.database',
        'db_port': 27017,
        'db_username': 'myUserAdmin',
        'db_password': 'admin',
}

List the audio files you want to input as follows (path relative to root_folder):

original_audio_tracks = [(OriginalAudio, OriginalAudioSettings('Blues_Drums.wav')),
                         (OriginalAudio, OriginalAudioSettings('Blues_Piano.wav'))]

List the packet loss models you want to test as follows:

packet_loss_simulators = [(GilbertElliotPLS, GilbertElliotPLSSettings()),
                          (MetronomePLS, MetronomePLSSettings()),
                          (BinomialPLS, BinomialPLSSettings())]

List the PLC algorithms you want to test as follows:

plc_algorithms = [(ZerosPLC, ZerosPLCSettings()),
                  (LastPacketPLC, LastPacketPLCSettings()),
                  (LowCostPLC, LowCostPLCSettings()),
                  (BurgPLC, BurgPLCSettings()),
                  (DeepLearningPLC, DeepLearningPLCSettings()),
                  (ExternalPLC, ExternalPLCSettings())]

❗The DeepLearningPLC algorithm requires the bufer_size to be set to 128 in the Settings of the PacketLossSimulator of choice.

List the metrics you want to use as follows:

metrics = [(MSECalculator, MSECalculatorSettings()),
           (PEAQCalculator, PEAQCalculatorSettings())]

Finally, run the testbench:

testbench = Testbench(testbench_settings, user, original_audio_tracks, packet_loss_simulators, plc_algorithms, metrics)
testbench.run()

If you want to change the parameters of any of the modules, you can do so by passing the settings as a parameter to the constructor of the module. For example, to change the PER of the Binomial packet loss model:

packet_loss_simulators = [(BinomialPLS, BinomialPLSSettings(per=0.1))]

For the full list of settings, check the settings.py file.

To plot the results:

testbench.plot(to_file=True, original_tracks=True, lost_samples_masks=True, output_analyses=True)

You can also plot the waveform of the reconstructed audio tracks, however since we're plotting the entire duration of the audio file, the differences with the original tracks are not going to be visible. This is why we developed a user interface for this application.

You will find both the audio files and the results in the folder specified in the root_folder setting.

Advanced Usage

More advanced features are hidden by default to simplify the basic usage of the tool. In this paragraph we will show how to use them.

Crossfade

When a packet is lost, the PLC algorithm has to reconstruct the lost samples. However, the reconstructed samples are not going to be identical to the original ones. This is why we implemented a crossfade feature, which allows to gradually transition from the original samples to the reconstructed ones, and vice versa.

The crossfade can be customized in the following aspects:

• Length: the duration of the crossfade in milliseconds.
• Function: the function to use:
  • Power: the crossfade is $x^n$.
  • Sinusoidal: the crossfade is sinusoidal ($\sin\left(x\right)$ for $0 < x < \frac{\pi}{2}$).
• Exponent: the exponent of the power crossfade function.
• Type:
  • Equal Power: the sum of the squares of the two crossfades is equal to 1.
  • Equal Amplitude: the sum of the two crossfades is equal to 1.

The crossfade can be applied to the left or the right of the lost samples, or both. The following example illustrates how to configure the settings to use the crossfade:

    crossfade_settings = ManualCrossfadeSettings(length=1,\
                        function='power', exponent=2.0, type='power')

    (ZerosPLC, ZerosPLCSettings(fade_in=QuadraticCrossfadeSettings(length=1),\
        crossfade=crossfade_settings))

As shown in the previous example, the fade_in parameter of any PLCAlgorithm determines the crossfade to apply to the left of the lost samples, while the crossfade parameter determines the crossfade to apply to the right of the lost samples. It is important to note that the length of the fade_in crossfade cannot be greater than the length of the lost packet.

The ManualCrossfadeSettings class allows to manually set the parameters of the crossfade. However, there are other convenience classes that are pre-set to some common configurations:

• NoCrossfadeSettings: disables the crossfade.
• LinearCrossfadeSettings: applies a linear crossfade (function = 'power' and exponent = 1.0).
• QuadraticCrossfadeSettings: applies a quadratic crossfade (function = 'power' and exponent = 2.0).
• CubicCrossfadeSettings: applies a cubic crossfade (function = 'power' and exponent = 3.0).
• SinusoidalCrossfadeSettings: applies a sinusoidal crossfade (function = 'sinusoidal').

Both the fade_in and crossfade parameters always default to NoCrossfadeSettings so that the crossfade is disabled by default.

Multiband Crossfade

Different crossfade settings can be applied to different frequency bands. This can be useful mitigate some of the artifacts introduced by the crossfade.

This is an example configuration for three bands crossfade:

    multiband_crossfade_settings = \
        [MultibandSettings(frequencies = [200, 2000]),
         QuadraticCrossfadeSettings(length=50),
         QuadraticCrossfadeSettings(length=5),
         QuadraticCrossfadeSettings(length=1)]

    (ZerosPLC, ZerosPLCSettings(crossfade=multiband_crossfade_settings))

The MultibandSettings class allows to specify the frequencies of the bands. The first frequency is the upper bound of the first band, while the last frequency is the lower bound of the last band. The number of frequencies determines the number of bands.

The list beginning with the MultibandSettings class contains the crossfade settings for each band. The first element of the list is the crossfade settings for the first band, the second element is the crossfade settings for the second band, and so on. The length of the list must be equal to the number of bands.

Each band can have its own crossfade settings, totally unrelated to the other bands.

Advanced PLC

The AdvancedPLC object allow for two complex beheviours simultaneously:

• Multiband PLC: different PLC algorithms can be applied to different frequency bands.
• Spatial PLC: different PLC algorithms can be applied to different audio channels (featuring M/S processing).

An example configuration of the settings to achieve such beheaviours is the following:

    frequencies = {'mid': [200, 2000], 'side': [1000]}
    band_settings = {\
        'mid':
            [(ZerosPLC, ZerosPLCSettings()),
             (BurgPLC, BurgPLCSettings(order=512)),
             (BurgPLC, BurgPLCSettings(order=256))],
        'side':
            [(LastPacketPLC, LastPacketPLCSettings()),
             (BurgPLC, BurgPLCSettings(order=256))]}

    (AdvancedPLC, AdvancedPLCSettings(band_settings, frequencies = frequencies, channel_link=False, stereo_image_processing = StereoImageType.MID_SIDE))

The frequencies parameter is a dictionary that maps the name of the channel to a list of frequencies.

The band_settings parameter is a dictionary that maps the name of the channel to a list of tuples. Each tuple contains the PLC algorithm to apply to the band and its settings. The length of each list must be equal to the number of bands of that channel.

If L/R or M/S channels require different PLC algorithms, the channel_link parameter needs to be set to False and the related settings need to be specified in the band_settings parameter using the left and right or mid and side keys.

Alternatively, if the same PLC algorithms need to be applied to both channels (regardless of the L/R or M/S coding), the channel_link parameter needs to be set to True and the related settings need to be specified in the band_settings parameter using the linked key.

User Interface

This user interface is an ongoing thesis project carried out by Stefano Dallona under the supervision of Luca Vignati. It is a web application developed using the React framework. The code is available in the following two repositories:

• plc-testbench-ui
• react-test

The easiest way to try it out is to use the Docker image provided by Stefano Dallona:

    docker pull cimil/plc-testbench-ui:latest

This Docker image already contains the code of PLCTestbench so it only requires a running MongoDB instance (see previous section).

Run the following command to start the container:

    docker run -e DB_USERNAME=$DB_USERNAME \
               -e DB_PASSWORD=$DB_PASSWORD \
               -e DB_HOST=$DB_HOST \
               -e DB_CONN_STRING=$DB_CONN_STRING \
               -e GEVENT_SUPPORT=$GEVENT_SUPPORT \
               -e FLASK_APP=$FLASK_APP \
               -e FLASK_DEBUG=$FLASK_DEBUG \
               -e FRONTEND_DATA_FOLDER=$FRONTEND_DATA_FOLDER \
               -e SECURITY_ENABLED=$SECURITY_ENABLED \
               -p 5000:5000 \
               -v /path/to/root/folder:/original_tracks \
               --name plc-testbench-ui \
               cimil/plc-testbench-ui:latest

Where the environment variables are:

Variable	Value	Description
DB_USERNAME	myUserAdmin	Username of the database
DB_PASSWORD	admin	Password of the database
DB_HOST	ip.of.the.database	IP address of the database
DB_CONN_STRING	mongodb://ip:27017	Connection string of the database
GEVENT_SUPPORT	True	Enable gevent support
FLASK_APP	app.py	Flask application
FLASK_DEBUG	True	Enable Flask debug mode
FRONTEND_DATA_FOLDER	/original_tracks	Path to the folder containing the audio files
SECURITY_ENABLED	False	Enable security

Then open your browser and go to localhost:5000.

❗Please consider the pre-release status of this user interface when using it.

References

[1] Elliott, Edwin O. "Estimates of error rates for codes on burst-noise channels." The Bell System Technical Journal 42.5 (1963): 1977-1997.

[2] Fink, Marco, and Udo Zölzer. "Low-Delay Error Concealment with Low Computational Overhead for Audio over IP Applications." DAFx. 2014.

[3] Verma, Prateek, et al. "A deep learning approach for low-latency packet loss concealment of audio signals in networked music performance applications." 2020 27th Conference of open innovations association (FRUCT). IEEE, 2020.

[4] Thiede, Thilo, et al. "PEAQ-The ITU standard for objective measurement of perceived audio quality." Journal of the Audio Engineering Society 48.1/2 (2000): 3-29.