-
Notifications
You must be signed in to change notification settings - Fork 317
HotPlug
HotPlug refers to the capability to detect newly connected devices once PortAudio is running, and also to gracefully deal with disconnected devices. We would like to add support for this to PortAudio. Some progress has already been made by David Stuart, Robert Bielik and Ross Bencina.
The feature was originally discussed in ticket #11.
The feature has a work-in-progress implementation on the hotplug branch, now available in the git repo here: https://github.com/PortAudio/portaudio/tree/hotplug
Key requirement: the new hotplug feature should be transparent to existing clients that don’t directly use the hotplug API.
Prior to calling Pa_RefreshDeviceList():
- There are no observable changes to the PA device list.
-
Pa_OpenStreamon a disconnected device results in a "device unavailable" error - Opening the specified device actually opens that device (not some other device). i.e. not only is the PA device list stable, but the PA device indices reliably map to unchanging native devices, even if native devices are connected and disconnected. [note: this may not be the case in some host APIs right now.]
During/after calling Pa_RefreshDeviceList():
- Unrelated open streams are unaffected by device connection/disconnection. Audio is not interrupted (unless a device associated device with the stream is disconnected). In all cases, open stream handles remain valid and must be closed by the client.
- The device list is updated with the currently connected devices (similar to calling
Pa_Initialize()) - No device notifications are dropped during refresh. If devices are inserted/removed during refresh an additional notification may be received just before
Pa_RefreshDeviceList()returns. - (?)
PaDeviceInfopointers for disconnected devices become invalid - (Sept. 19) The following values relating to default devices may change after a successful call to
Pa_RefreshDeviceList():PaHostApiInfo::defaultInputDevice,PaHostApiInfo::defaultOutputDevice,Pa_GetDefaultHostApi(),Pa_GetDefaultInputDevice()andPa_GetDefaultOutputDevice(). On failure they are unchanged.
Unclear:
- There may be scope for a notification race around
Pa_Initialize. Possibly allow for setting the connection notification callback prior to callingPa_Initialize
(Updated September 19, 2016)
This section describes the current proposed API that is available in the hotplug branch in GIT here: https://github.com/PortAudio/portaudio/tree/hotplug
The public hotplug API is listed below in full (this was derived from a diff of portaudio.h in the hotplug branch.
Pa_Initialize() is extended with the following information:
@note The device list returned by Pa_GetDeviceCount() et al, default devices,
and the default host API are all frozen when Pa_Initialize() is called. The
device list is not automatically updated when hardware devices are connected or
disconnected. To refresh the list of devices, and default devices, either call
Pa_RefreshDeviceList() or uninitialize PortAudio using Pa_Terminate(),
and then reinitialize it by calling Pa_Initialize(). To register a callback
when native devices or defaults change, call Pa_SetDevicesChangedCallback().
A new Pa_RefreshDeviceList() function is added:
/** Refresh PortAudio's list of devices to reflect currently available
native audio devices. Also refreshes default devices and default host API.
PortAudio's list of devices is usually frozen when Pa_Initialize() is
called. Call Pa_RefreshDeviceList() to refresh PortAudio's device list
at any time while PortAudio is initialized.
@return an error code indicating whether the refresh was successful.
If the function succeeds, future calls to Pa_GetDeviceCount() may return
a different count than before, all device indexes may refer to
different devices, and any previously returned PaDeviceInfo pointers are
invalidated. Default devices reflected by PaHostApiInfo.defaultInputDevice,
PaHostApiInfo.defaultOutputDevice and global defaults returned by
Pa_GetDefaultHostApi(), Pa_GetDefaultInputDevice() and
Pa_GetDefaultOutputDevice() may changed.
If the function fails, the device list and defaults are unchanged.
@note Open streams will not be affected by calls to this function.
*/
PaError Pa_RefreshDeviceList( void );
(Sept. 19) In addition to refreshing the device list, Pa_RefreshDeviceList() updates default devices. This was already being done in the prototype implementations and is now explicit in the API documentation. Updating default devices has the consequence that the default host API may change. (The default host API is specified as the first host API with an available device.) The implementation of Pa_RefreshDeviceList() has been updated to correctly update the default host API in hotplug branch.
It is possible to register a callback that is called when devices are inserted and/or removed:
/** Functions of type PaDevicesChangedCallback are implemented by PortAudio
clients. They can be registered with PortAudio using the Pa_SetDevicesChangedCallback
function. Once registered, a callback is invoked whenever available native audio
devices may have changed. For example, when a USB audio device is connected or
disconnected. The callback is also invoked whenever the native default device
selection changes.
When the callback is invoked, there is no change to the devices listed by
PortAudio. To update PortAudio's view of the device list, the client
must call Pa_RefreshDeviceList(). This must not be done from within
the callback.
The callback may occur on any thread. The client should not call other PortAudio
functions from within a PaDevicesChangedCallback. In particular, the client
should not directly call Pa_RefreshDeviceList() from within the notification
callback. Instead, the client should set an atomic flag, or queue a message to
notify a controller thread that would then call Pa_RefreshDeviceList().
@param userData The userData parameter supplied to Pa_SetDevicesChangedCallback()
@see Pa_SetDevicesChangedCallback
*/
typedef void PaDevicesChangedCallback( void *userData );
/** Register a callback function that will be called when native audio devices
become available or unavailable, and when native default devices change.
See the description of PaDevicesChangedCallback for further details about
when the callback will be called.
@param userData A client supplied pointer that is passed to the devices changed
callback.
@param devicesChangedCallback a pointer to a callback function with the same
signature as PaDevicesChangedCallback. Passing NULL for this parameter will
un-register a previously registered callback function.
@return on success returns paNoError, otherwise an error code indicating the
cause of the error.
@note Only one function pointer may be registered at a time.
@see PaDevicesChangedCallback
*/
PaError Pa_SetDevicesChangedCallback( void *userData, PaDevicesChangedCallback* devicesChangedCallback );
Connection ids uniquely identifies a device, so long as the device is plugged in and PortAudio is initialized. There is now a stub implementation in all host APIs. The stub implementation works fine for APIs that don't support refreshing the device list.
/** An integer id that uniquely identifies a device, so long as the device is
plugged in and PortAudio is initialized. Its purpose is to identify (correlate)
devices across calls to Pa_RefreshDeviceList().
Each device reported by Pa_GetDeviceInfo() has a distinct connectionId.
PortAudio makes a best-effort to ensure that a persistent underlying device
has the same connectionId before and after a single call to Pa_RefreshDeviceList().
If a device is disconnected and later reconnected it may be assigned a new
connectionId.
@see Pa_RefreshDeviceList()
*/
typedef unsigned long PaDeviceConnectionId;
typedef struct {
...
PaDeviceConnectionId connectionId; /**< @see PaDeviceConnectionId */
} PaDeviceInfo;
connectionIds are unique and not reused (until PA is terminated). Disconnecting and reconnecting a device results in a new connectionId value. If a device is not disconnected, it should retain the same connectionId across calls to Pa_RefreshDeviceList(). There may be corner-cases where there is no programmatic way to implement this behavior -- we will have to finish the implementations to learn this.
(Sept. 19) IMPLEMENTATION GOAL: We are currently trying to implement the invariant: "The connectionId is guaranteed to change whenever the device is disconnected, even if it is immediately reconnected." If experience shows that this is not feasible we will fall back to the relaxed specification "may be assigned a new connectionId".
Use case: 5 identical USB microphones (all with the same name, indistinguishable except for their PA device index and connection id), use connectionId to correlate individual devices across device list refresh. Disconnect one device, others retain their connectionIds. (Host API implementations will need to use low-level knowledge such as USB connection paths to implement this).
N.B.: connectionId is considered orthogonal to the “persistent device identifiers” concept discussed below.
A PaUtil_MakeDeviceConnectionId() internal method is available to implementers to generate new Ids.
Possible alternative names: monotonic device id, hotplug seed, device locator, insert id, session id, hotplug id
This is the interface that each host API implementation must implement in order to support hot-plug. Three methods are added to PaUtilHostApiRepresentation defined in pa_hostapi.h. The interface involves 3 methods: ScanDeviceInfos, CommitDeviceInfos, DisposeDeviceInfos. pa_front.c calls these methods to update the device list using a two-phase approach. It first rescans all of the host APIs by calling ScanDeviceInfos on each host API, if this succeeds then pa_front calls CommitDeviceInfos to "install" the new device infos, on failure it calls DisposeDeviceInfos.
/** ScanDeviceInfos, CommitDeviceInfos, DisposeDeviceInfos are used to refresh the device list.
You can think of the caller doing something like:
void *scanResults = 0;
int newDeviceCount = 0;
if( hostApi->ScanDeviceInfos(hostApi, hostApiIndex, &scanResults, &newDeviceCount) == paNoError )
{
... do other stuff ...
if( ok to commit )
{
hostApi->CommitDeviceInfos(hostApi, hostApiIndex, scanResults, newDeviceCount);
}
else
{
hostApi->DisposeDeviceInfos(hostApi, hostApiIndex, scanResults, newDeviceCount);
}
}
NB: ScanDeviceInfos may be called on all host APIs first, prior to pa_front
deciding whether to commit or cleanup.
*/
/**
Scan for current native device information and compute a new device count.
Preparation step of a two-stage transaction that updates the active device list.
@param hostApi The target host API.
@param index The host API index of the target host API.
@param scanResults (OUT) Result parameter. Upon success, set to point to an
opaque structure containing collected device information. Ignored on failure.
@param newDeviceCount (Out) Result parameter. Upon success, set to the
number of discovered devices. Ignored on failure.
@return on success returns paNoError, otherwise an error code indicating
the cause of the error.
This function should not make any visible changes to the host API's internal
state. In particular, it should not update the active device list, device
information or device count.
If the function returns successfully, the caller will either (1) pass
scanResults and newDeviceCount to CommitDeviceInfos() to complete the
transaction, or (2) pass scanResults and newDeviceCount to
DisposeDeviceInfos() to abort the transaction.
@note CommitDeviceInfos and DisposeDeviceInfos should not fail, therefore
ScanDeviceInfos should perform all necessary preparations,
such as memory allocation, so that CommitDeviceInfos can always execute
without failure.
@see CommitDeviceInfos, DisposeDeviceInfos
*/
PaError (*ScanDeviceInfos)( struct PaUtilHostApiRepresentation *hostApi,
PaHostApiIndex index,
void **scanResults,
int *newDeviceCount );
/**
Update the active device list with new device information that was
returned by ScanDeviceInfos(). This is the final commit step of a two-stage
transaction that updates the active device list.
As this function is not permitted to fail, it should be very simple:
swap in new device information and free the old information.
@param hostApi The target host API.
@param index The host API index of the target host API.
@param scanResults An opaque structure containing collected device
information previously returned by ScanDeviceInfos. This is the information
that will be installed into the active device list.
@param newDeviceCount The number of discovered devices previously returned
by ScanDeviceInfos. This is the new number of devices in the active device
list.
@note This function is not permitted to fail. It should always return paNoError.
@see ScanDeviceInfos, DisposeDeviceInfos
*/
PaError (*CommitDeviceInfos)( struct PaUtilHostApiRepresentation *hostApi,
PaHostApiIndex index,
void *scanResults,
int deviceCount );
/**
Free device information that was returned by ScanDeviceInfos. This is
used in the cleanup process for a failed two-stage transaction.
@param hostApi The target host API.
@param index The host API index of the target host API.
@param scanResults An opaque structure containing collected device
information previously returned by ScanDeviceInfos. scanResults will be
freed by this function.
@param newDeviceCount The number of discovered devices previously returned
by ScanDeviceInfos.
@note This function is not permitted to fail. It should always return paNoError.
@see CommitDeviceInfos, ScanDeviceInfos
*/
PaError (*DisposeDeviceInfos)( struct PaUtilHostApiRepresentation *hostApi, void *scanResults, int deviceCount );
-
portaudio.hnew API -
pa_hostapi.hhost-api internal rescan/refresh interface -
pa_front.cimplements the 2-phase device list refresh -
pa_hotplug.hinterface between platform notification engine andpa_front.c -
pa_win_ds.chas a working device refresh implementation for DirectSound -
pa_win_wdmks.chas a working device refresh implementation for WDM/KS -
pa_win_hotplug.cimplements device insertion/removal notifications for Windows usingRegisterDeviceNotificationWDEVICE_NOTIFY_ALL_INTERFACE_CLASSESAPI. Includes a device cache to track which devices are currently connected -
patest_refresh_device_list.cis a simple test of the hotplug API that displays device insertion/removal events, rescans devices, and lists them.
Other files with minor changes
-
portaudio.defexport new API symbols -
pa_win_wmme.czero function ptrs for no-op device refresh
(Last Updated September 19)
At the time of writing, testing is very ad-hoc. We need to write more tests, and to test each implementation in detail. Some tests are on feature branches, not yet merged to hotplug (links below).
-
Test that the
PaDevicesChangedcallback is invoked, and thatPa_RefreshDeviceList()adds/removes devices. hotplug:patest_refresh_device_list.c -
Test that streams do not hang when the stream's device(s) are unplugged (callback,
Pa_ReadStream(),Pa_WriteStream(). hotplug:patest_unplug.c rb-hotplug-wmme:patest_unplug.c rb-hotplug-wmme:patest_unplug_readwrite.c -
Test that streams can be aborted, stopped and/or closed without hanging after a streaming device is disconnected. rb-hotplug-wmme:patest_unplug.c rb-hotplug-wmme:patest_unplug_readwrite.c (Test is very preliminary, requires manual manipulation of
#ifto test the stop and abort cases.)
In all cases below, tests involving streams need to test input-only, output-only, full-duplex, callback and blocking read/write streams.
-
Test that unreleated streams are not affected by
PaDevicesChangedcallback and calls toPa_RefreshDeviceList() -
Test that streams becomes inactive and enters the correct error state when their device is unpluged (requires updated specification of the stream state machine for asynchronous errors).
-
Test that the correct device is opened when calling
Pa_OpenStream()after connecting a new device, but prior to callingPa_RefreshDevices(). (There is a possible failure mode where a different device is opened, depending on how PA devices are matched to host API devices. This is likely buggy with WMME at the moment, a partial test would be to open each device with max channels to ensure we at least maybe have the correct devices by expecting paNoError or paDeviceUnavailable.) -
Test that opening a disconnected device with
Pa_OpenStream(), prior to callingPa_RefreshDeviceList()returnspaDeviceUnavailable. -
Test that default device changes are notified, and that defaults are updated (maybe extend
patest_refresh_device_list.cto indicate default devices? Erik Bunce's patch might give some ideas.) -
Could have a test that plays audio out the default device and "follows" the current default device(s) when they change.
(Updated September 2, 2016)
This section describes changes that we plan to make on the hotplug branch, but that haven't been actioned yet.
(Sept. 19) Windows (and Linux?) notification engines don't call PaDevicesChangedCallback when default devices change.
Device info connectionId has been added, but it is not yet correctly persistent for the host APIs that support refresh (DirectSound and WDM/KS)
The CommitDeviceInfos and DisposeDeviceInfosinternal functions should be coded in such a way that the never fail. Therefore returning an error code probably doesn’t make sense (it would be like throwing an exception from a destructor).
Work prior to August 2016 was stored in the hotplug branch in SVN (now merged to git, see link above). Old SVN link: https://app.assembla.com/spaces/portaudio/subversion/source/HEAD/portaudio/branches/hotplug For reference, there is an annotated version of the SVN log for the hotplug branch here: HotPlug_SVN_Log. It includes links to the individual commits.
There are also two other hotplug contributions that may be useful:
David Stuart’s 2010 patch for DirectSound and CoreAudio, last known CoreAudio version, written prior to 2-phase commit. (click “expand all diffs”) 7fcdfbf5dfbd43ee042bfbc72de7136ed5895c64
Erik Bunce’s 2007 patch, CoreAudio, includes notifications (download the patch) https://app.assembla.com/spaces/portaudio/tickets/11/details?comment=65652633
After device list refresh, what happens to PaDeviceInfo struct pointers that reference disconnected devices? Proposal: they are freed and no longer valid. Discuss, decide.
Review names for functions and fields.
Question: does Pa_RefreshDevices() guarantee to not update existing devices, or can it update existing devices (e.g. if some system parameter has changed such as default sample rate)?
When should the notification engine be started/stopped? At the moment, the device changed notification engine is started during Pa_Initialize() and stopped during Pa_Terminate(). Alternatively it could be started when the notification callback is registered. There might be reasons why Robert configured it the way he did. [note also: if we're going to allow registering the callback before Pa_Initialize, we may want to defer starting the notification engine until Pa_Initialize().]
Device disconnection may require async error reporting, or at least an “error” stream state, which is not currently implemented. A Pa_GetStreamError() function could query why a stream failed (e.g. due to disconnection). This is bound up with adding an error state to the stream state machine. (Ross to move documentation for stream state machine into the main documentation.)
How functions that call multiple-APIs should fail. During Pa_Initialize() and Pa_RefreshDeviceList() each host API is called in turn. When a host API returns an error, there are many error handling policies. So far, two policies are of interest. Policy 1 is what is currently implemented. (1) As soon as an error is encountered, roll back to the previous state and return an error. In the case of Pa_Initialize(), this means that Pa_Initialize() fails and cleans up when the first host API fails. In the case of Pa_RefreshDeviceList(), this means that if any host API fails, an error is returned and all device lists are rolled back to their pre-call state. (2) When an error is encountered, skip or roll-back that host API but continue to initialize/refresh other host APIs.
Policy 1 does not guarantee forward progress. Policy 2 does not return full error information. Both policies suffer from making a policy decision that only the client can decide on for sure. Note that something close to policy (2) is implemented for some host APIs (errors during device enumeration cause devices to be skipped), possibly this is not consistent.
Options discussed so far:
- Stick with current “consistent” design for Pa_Initialize() and Pa_RefreshDevices()
- Could introduce a policy parameter to Pa_RefreshDevices()
enum { paFailFast, paSkipOnHostApiError } - Add a new API function:
Pa_RefreshHostApiDevices(hostApiIndex)that allows for initializing individual host APIs. This would allow the client to implement the “fail fast” policy. It would not allow the client to implement the current “global rollback on first error” policy, as the rollback mechanism would not be exposed in the API.
(August 2021) Ticket #615 discusses sensing default device changes. The system default device can change either in response to device connection/disconnection, or due to user manipulation of a system control panel. So far we acknowledge that Pa_RefreshDevices() may change the PA default devices, but we do not necessarily notify when the default device changes if there are no other device changes -- we may want to issue such a notification. We could consider passing a flag to the notification callback to indicate whether devices and/or defaults have changed.
Evan proposed a Pa_GetDeviceError function to probe for whether a device is still connected (Returns PaNoError if the device appears healthy, or an error code if some kind of problem, such as disconnected state, is identified.). It’s unclear whether this is implementable. To be explored. (Note that you can already track device insertion/removal notifications and call refresh if you want to know whether a device is still connected).
Robert pointed out that WASAPI has its own notification mechanism for insertion/removal of WASAPI endpoints (this might be needed for e.g. virtual endpoints which would no show up as devices). Therefore we may need to support per-API notification engines. Right now we assume there is only one per platform. Not sure which other APIs have their own notification mechanisms.
Sqweek raised the issue of “Default output device” transparently switching between headphone and speakers when headphones are plugged in. It’s not clear whether this conflicts with our goal of hotplug not messing with open streams. Ross thinks this is a detail for individual host APIs to resolve, but noted that "not messing with open streams" refers to not messing with streams that involve streams that are not related to the connected/disconnected devices.
It is possible that PaDevicesChangedCallback could return additional information, for example: it could specify which APIs were affected, and whether it is a connection, disconnection, or unknown event. Adding this functionality only makes sense if we can reliably source this information -- until we have more implementation experience that's not clear. For now we have the lowest common denominator: "something changed".
(Sept. 19) On the mailing list we discussed the idea that the PaDevicesChangedCallback could have a parameter to indicate whether the notification involved a device list change, or a default device change. This could be done by adding a flags parameter to the callback, with values such as { paDeviceAdded, paDeviceRemoved, paDefaultDeviceChanged }. This is only practical if such granularity can be implemented on all platforms. See the mailing list thread for further discussion:
https://lists.columbia.edu/pipermail/portaudio/2016-September/000761.html
There are two ways to generate an updated device list: (1) do a full rescan and then cross-correlate to the old device list to synchronise connectionIds, (2) only scan new devices, reuse device info records from previous scan (requires reference counting device records). Different host APIs may use different schemes, but scheme 2 (partial/minimal rescan) is preferred.
There is the option to add a Pa_RefreshHostApiDeviceList(hostApiIndex) to allow for refreshing the device list of a specific host api. We would like to avoid this as it adds complexity to the API with no obvious benefit.
(Sept. 19) Most platforms will eventually have multiple notification sources (e.g. udev and JACK on Linux, CoreAudio and JACK on OSX, system devices and WASAPI on Windows). The infrastructure code will need to be extended to support device notifications from multiple sources. The most likely structure is the following:
- A single per-platform hotplug file will manage the callback mutex and any other global initialization
- Multiple notification engine back-ends can be implemented in separate modules.
- Notification engines can be initialized by the host APIs that need them during host API
Initialize()and Terminated during host APITerminate(). Use reference counting to support notification engines initialized by multiple APIs.
We will refactor to this structure (or something else) when we first need to support multiple notification engines on a single platform. This will most likely happen when we add a WASAPI notification engine on Windows.
Open question: does the PaDevicesChangedCallback notification engine get started at Pa_Initialize()/Pa_Terminate? Or only when a callback is registered/unregistered? This will affect how hooks for initializing and terminating the notification engine work. Having it always initialized seems simpler and more robust, but also adds overhead even for clients who don't use it.(End Sept. 19)
Do we want a way to poll for device changes in addition to the callback? (This wouldn’t be hard to implement with a couple of atomic counters. Doesn't critically affect developing the hotplug implementation so leave it out for now.)
Evan proposed having an API to map between device indexes and connection ids. Although this is possible, it could be implemented by the client as a simple search over device info structures (PA would probably implement it that way internally). This could be added later, and doesn’t seem worth adding until connection id API has been implemented/proven.
Before merge, need to update pa_hostapi_skeleton.c to include any generic functionality for refresh etc.
Need to test that running streams behave correctly when device is connected/disconnected.
Streams should call stream finished callback if they fail due to device disconnection.
Maybe Pa_SetDevicesChangedCallback should be named differently, as it doesn't actually indicate that devices changed. It indicates that devices might need to be refreshed. It's also called when default devices change.
- Monotonic device list (devices are added, but never removed)
- Just use device name to correlate devices (not unique enough for “5 duplicate headsets” use-case)
- Use PaDeviceInfo pointer to uniquely identify connection (probably not good if we allow for freeing device ptrs, i.e. suffers from ABA problem)
This section provides context for what we are including in HotPlug.
(“Persistent” here means across calls to Pa_Initailize()/Pa_Terminate(), across application invocations, and/or across reboots).
Possibly add two opaque string fields to PaDeviceInfo:
-
persistentDescriptor(e.g. Maybe a GUID or combination of USB vendorId, productId and serial number. Fallback would be the device name string.) -
connectionPath(e.g. a string representing the path of the device from the root USB hub. Fallback would be an empty string)
When combined with the host API ID, the persistentDescriptor and connectionPath could be used to persistently identify audio devices, whether they were connected to the same hardware port, or a different port. This covers both the use-case of storing a reference to a device irrespective of how it is connected (just use persistentDescriptor to identify the device) or storing references to multiple identical devices that are connected to different ports (use a combination of persistentDescriptor, connectionPath). Although the fields may reference host-api neutral hardware identifiers, they should be treated as blobs and couldn’t necessarily be used to correlate hardware devices across multiple host APIs.
TODO: document uniqueness constraints (or not) on these values.
Although it is not proposed to implement this feature in the public API, similar mechanisms may be needed internally to ensure that session ids for existing devices are stable -- especially when multiple devices with the same name are present.
Select Which Host APIs to Initialize, Allow Initializing/Terminating Specific host APIs While PA is Running.
The following ticket covers selecting which host API to initialize: #10
There is now also a draft implementation in the ticket-10-select-host-apis branch.
Initializing/terminating host APIs while PA is active overlaps with hotplug in so far as it causes the global device list to change. It’s unclear whether there would be a big impact here. There might be some tweaks required for host APIs that share common underlying code (e.g. COM initialization.)