VRSYS Unity-to-Pybinsim Spatializer Interface

This repo contains code to build a Unity Native Audio Spatializer Plugin that allows audio from Audio Sources in Unity to be processed by pyBinSim, a python-based binaural simulation tool.

The plugin is intended to work with a pyBinSim server, running on the same machine as the Unity application. The pyBinSim server repository can be found here: https://github.com/gary444/pyBinSim

The plugin can be built with cmake, following the instructions below. Note that the built plugin (.dll) must be copied to the Unity project manually. In this project, the built plugin should be copied to the folder unity-to-pybinsim-spatializer-example/Assets/Plugins. See below for more details.

This repository is based on unity-nativeaudioplugins, a version of Unity's Native Audio Plugins repository that was adapted for building with CMake.

The instructions below were provided with the unity-nativeaudioplugins repo, and still apply to this one.

---

unity-nativeaudioplugins

The unity native audio SDK integrated with cmake. Use the most recent version of cmake when possible, at time of writing it was 3.15.3.

Building: CMake GUI

In CMake GUI:

• set the source code and build directories
• 'Configure' and 'Generate'
• Open the Visual Studio project

In Visual Studio

• Select the 'Release' and 'x64' configuration
• Build the 'ALL BUILD' target

Building: command line

Clone the repo, then run the following shell commands:

cd unity-nativeaudioplugins
mkdir build
cd build
cmake .. -G"*generator*"

Where the *generator* is :

• nothing or Unix Makefiles for linux or osx makefiles
• Xcode for Apple's Xcode
• Ninja for Microsoft command line build
• Visual Studio 15 2017 Win64 for Visual Studio 2017 solution generation

The buildtool can be invoked (make or ninja) or the ide project/solution file can be opened.

More generators can be found on this page

'Installing'

The built plugin (.dll) must be copied to the Plugins folder of the Unity project manually.

---

Spatializer Plugins

A .dll can contain multiple audio plugins, defined in the PluginList.h header.

The plugins contained in the DLL built by this repository are:

• pyBinSim Spatializer: spatializer plugin that sends audio buffer via ZMQ connection to a pyBinSim server instance. The audio buffer for each source is sent (and received) individually and sequentially.
• pyBinSim Batched Spatializer: spatializer plugin that sends audio buffers from each spatialized source to a pyBinSim server instance. Once all sources are received by the server, they are processed together ('batch processing') and returned. The last audio source to be processed receives the processed audio for all sources and writes to the output buffer.

Both the 'pyBinSim Spatializer' and 'pyBinSim Batched Spatializer' plugins allow a fixed delay to be applied to each audio source individually, for aligning audio and video streams. The fixed delay can be set from a C# script using the following function:

float fixedDelaySec = 0f;
int fixedDelayParameterIndex = 4;
audioSource.SetSpatializerFloat(fixedDelayParameterIndex, fixedDelaySec);

Example code also exists for the following plugins:

• Passthrough Spatializer: miminal spatializer plugin example that does not change the audio buffer at all. The input data is copied directly to the output buffer.
• Fixed Delay Spatializer: minimal spatializer plugin that applies a fixed delay to each source.

Spatializer Plugin Usage

This project contains a minimal Unity project in the folder unity-to-pybinsim-spatializer-example. The Unity scene in the project demonstrates the required settings to spatialize an audio source. The requirements are:

• The .dll built from the spatializer plugin code should be in the Assets/Plugins folder. In the inspector, while the Plugin is selected, Load On Startup should be checked (and applied when first set)
• On the AudioSource component,
  • toggle Spatialize on
  • set the Spatial Blend value > 0
• In Project Settings > Audio
  • Make sure the Spatializer Plugin you want to use is selected (e.g. PyBinSim Spatializer)
  • The DSP Buffer Size should align with what is expected by pyBinSim (see note below on DSP buffer size)

Important: the PyBinSim server must be running before the Unity app is started, otherwise the Unity app will crash! For details on how to run the PyBinSim server, see the repository.

DSP buffer size

PyBinSim expects buffers to arrive with a fixed number of samples per channel. This value is set in the configuration file read by pyBinSim server on startup. The Unity bufferlength must be set accordingly. Unfortunately, Unity does not allow direct setting of the buffer length, instead allowing a choice of "Best Latency", "Good Latency" and "Best Performance".

It seems that "Good Latency" corresponds to a DSP buffer size of 512 samples per channel. This is therefore the default, recommended setting.