Importing sounddevice interrupts other audio streams on Windows when exclusive mode is enabled

[arkrow]

As the title mentions, importing sounddevice interrupts any currently playing audio streams momentarily on Windows when exclusive mode is enabled on the sound device. This issue seems to originate from PortAudio's initialization logic, however, I can't seem to pinpoint the problematic code there, and there isn't a similar issue opened there either.

To replicate the issue:

1. On Windows 10/11, enable exclusive mode on the default sound device.
2. Run any audio stream in the background (it doesn't seem necessary for the stream to be using exclusive mode)
3. import sounddevice (can be run in the python interactive interpreter; the current audio stream will be interrupted as a result)

This issue does not occur if exclusive mode is disabled in the sound device. I'd assume this issue can be reproduced on Windows 7 as well.

[mgeier]

This issue seems to originate from PortAudio's initialization logic, however, I can't seem to pinpoint the problematic code there, and there isn't a similar issue opened there either.

Then you should open one!

This doesn't seem like anything that can be solved from the Python side.

To be sure, you could replace your step 3 with some other software that uses PortAudio, for example Audacity, and listen if it behaves the same.

[arkrow]

To be sure, you could replace your step 3 with some other software that uses PortAudio, for example Audacity, and listen if it behaves the same.

Great call. I've tested the same scenario with Audacity and it doesn't exhibit this issue. In fact, using Audacity's PortAudio binary (instead of the ones linked in _sounddevice_data) solves the issue. From what I can gather, Audacity's PortAudio binary is based on the 19.7 release, same as here, confirmed by comparing .Pa_GetVersion() for both libraries (though the revision is unknown for Audacity's binary). Regardless, the discrepancy in behavior is unexpected.

[mgeier]

Hmm, that's interesting.

I don't know exactly how the DLL from Audacity is built, probably with Conan. I found https://github.com/audacity/audacity/blob/52708c770ffcd98cb86527cd7756413c4179f7fe/conan/conanfile.py#L119-L121, which suggests that WDM-KS is disabled. Maybe that causes the problem? It could be something completely different, though.

I don't know if that helps, but you could also try some older versions of the DLL, e.g. https://github.com/spatialaudio/portaudio-binaries/blob/d9765fc440e4181576b5a801afd2e3624ac2edd9/libportaudio64bit.dll and https://github.com/spatialaudio/portaudio-binaries/blob/bab7ef6a62415c2c83bdb632585fa439d3edc48b/libportaudio64bit.dll.

[mgeier]

I made a little experiment building the DLL with MSVC, can you please try the DLL from https://github.com/spatialaudio/portaudio-binaries/actions/runs/6523747765?

[arkrow]

All three linked binaries unfortunately exhibit the same issue.

I don't know exactly how the DLL from Audacity is built, probably with Conan. I found https://github.com/audacity/audacity/blob/52708c770ffcd98cb86527cd7756413c4179f7fe/conan/conanfile.py#L119-L121, which suggests that WDM-KS is disabled. Maybe that causes the problem? It could be something completely different, though.

Audacity's PortAudio binary does indeed support fewer APIs, namely just: MME, DirectSound and WASAPI; while the PortAudio bundled with sounddevice is built with ASIO and WDM-KS support enabled.

To help triangulate the root cause, I've tested another PortAudio binary and found that FlexASIO's bundled PortAudio binary also does not exhibit the issue like Audacity's. The most distinct differences are that it's built without the ASIO SDK (so API support is the same as sounddevice, minus ASIO), and it's built using a more recent commit from PortAudio's master branch: 80ef9ac.

[mgeier]

Thanks for testing this. It's good to know that switching from MXE cross-compiling (on my local computer) to directly compiling with MSVC (via Github Actions) doesn't change the behavior.

Feel free to experiment with my github-action branch, where you can select any PortAudio commit (see https://github.com/spatialaudio/portaudio-binaries/blob/7e139d8202945feff9150b550684d28ab7f6f43a/.github/workflows/build-libs.yml#L13) and any combination of host APIs (see https://github.com/spatialaudio/portaudio-binaries/blob/7e139d8202945feff9150b550684d28ab7f6f43a/.github/workflows/build-libs.yml#L28).

[arkrow]

After some experimentation compiling different builds of PortAudio, it's clear that building PortAudio with ASIO causes the problem.

Since the problem is upstream and its root cause unclear (could be PortAudio or the ASIO SDK), would it be possible to modify sounddevice to conditionally load a non-ASIO PortAudio binary instead, using an os.environ flag at import time? I think that would be the simplest workaround, especially for those that do not require ASIO for their use case. I can provide the PR for this change, if a non-ASIO binary is added to spatialaudio/portaudio-binaries.

[mgeier]

Thanks for testing this, that's very interesting.

would it be possible to modify sounddevice to conditionally load a non-ASIO PortAudio binary instead,

Well, this should already be possible, but it is platform-dependent.

On Linux, you can simply put a custom libportaudio.so into the directory /usr/local/lib/, and the sounddevice module should use that one automatically.
In some cases you might have to call sudo ldconfig for the new library to be found.

On macOS it should work similarly.

On Windows, however, I don't know if/how it works.
I guess the custom DLL has to be placed somewhere where ctypes.util.find_library() can find it.

We might also have to add another library name to the code:

python-sounddevice/sounddevice.py

Lines 63 to 65 in 4aa98dc

	'portaudio', # Default name on POSIX systems
	'bin\\libportaudio-2.dll', # DLL from conda-forge
	'lib/libportaudio.dylib', # dylib from anaconda

It would be great if you could check this out and suggest any changes that are necessary to make it work properly.

using an os.environ flag at import time?

Yes, this might also be possible, it has already been discussed some time ago: #130.

[...] if a non-ASIO binary is added to spatialaudio/portaudio-binaries.

Providing a different DLL is a follow-up concern, but if we create the DLLs with CI (as in my experimental github-action branch), it should be straightforward to provide multiple variants of the DLL.

[Prosperelucel]

Hello,

I wanted to confirm that when using the .dll from the FLEXASIO .exe, other audio streams are not interrupted when I import the sounddevice module in Python 3.10 on Windows 10. I simply replaced the Lib\site-packages_sounddevice_data\portaudio-binaries\libportaudio64bit.dll with the portaudio.dll from the FlexASIO\x64 folder, and then renamed it.

Thank you for providing this workaround!

[arkrow]

I guess the custom DLL has to be placed somewhere where ctypes.util.find_library() can find it.

I think the best approach would be bundling it with sounddevice rather than installing the alternative binary separately. I've had several users report the same problem, so it would be great if this is a possible option to consider. I've opened PR #498 to demonstrate my suggested change for this issue while being fully backwards-compatible, but I'm very much open to feedback and changes in approach.

Thank you for providing this workaround!

Glad to help, thanks for confirming @Prosperelucel

[mgeier]

Before jumping into work-arounds, I would like clarification on two questions:

• does this problem still occur on the master branch of PortAudio?
• is the PortAudio project aware of this problem?

It would be good to know if this has been discussed already in the PortAudio project and if there are any ideas to solve this within PortAudio.

There should at least be an issue at https://github.com/PortAudio/portaudio/issues before we start a work-around here.

[arkrow]

does this problem still occur on the master branch of PortAudio?

Yes, unfortunately.

is the PortAudio project aware of this problem?

Latest ongoing discussion: PortAudio/portaudio#858

Relevant issues in the PortAudio project:

• PortAudio ASIO driver scan PortAudio/portaudio#696
• Windows: just calling Pa_Initialize kills audio in other apps PortAudio/portaudio#564

It would be good to know if this has been discussed already in the PortAudio project and if there are any ideas to solve this within PortAudio.

There should at least be an issue at https://github.com/PortAudio/portaudio/issues before we start a work-around here.

Alright, sounds great. Ideally, the issue is resolved upstream and a new version of sounddevice can then be released with the fixed PortAudio binaries. However, it seems that the issue is much more fundamental with PortAudio's ASIO support. That said, I'm open to seeing where the ongoing issue discussion at PortAudio leads and alternative ideas with regards to this issue's fix.

[mgeier]

Thanks for further looking into that!

I found this interesting:

PortAudio/portaudio#696 (comment):

I suspect many people building PortAudio are not aware that PortAudio's ASIO support is a giant footgun in that regard.

I'm one of those people who were not aware of that.

Do you also experience the startup delay mentioned at PortAudio/portaudio#696 (comment)?

I'm wondering if we should provide a DLL without ASIO by default and provide a work-around to enable ASIO for those who need it?

I don't know how many people actually want to use the sounddevice module with ASIO, but according to the discussions in the issues you mentioned, there are people who want to use ASIO (but maybe not in a Python context?).

[arkrow]

Do you also experience the startup delay mentioned at PortAudio/portaudio#696 (comment)?

Yes. With the Realtek ASIO driver I had, which I assume many would also have through Realtek's audio drivers, importing sounddevice adds a noticeable >=1 second delay, to the point where I decided to lazy load the library instead before I knew the actual root cause.

I'm wondering if we should provide a DLL without ASIO by default and provide a work-around to enable ASIO for those who need it?

Some users likely depend on ASIO for the low-latency it offers, if it is necessary for their use case (e.g. minimal latency for live recording->processing->playback), but for the majority of cases I'd imagine most users do not specify a specific host API, ASIO or otherwise. Having ASIO be optional and enabled explicitly would be the better default in my opinion. Though, existing users would need to be alerted to the change, and they should have an environ flag or otherwise simple method to enable ASIO. Perhaps a simpler change would be to move ASIO support to package extras, so the necessary binary is packaged if the user installs e.g. sounddevice[asio].

[mgeier]

importing sounddevice adds a noticeable >=1 second delay

That's very good to know, I wasn't aware of that.
I think that's a good reason to deactivate ASIO by default.

existing users would need to be alerted to the change

I'd just make a 0.5.0 release and mention the breaking change in the release notes.
I don't really know any other feasible channel to communicate the breakage.

Perhaps a simpler change would be to move ASIO support to package extras, so the necessary binary is packaged if the user installs e.g. sounddevice[asio].

I didn't think about that. This sounds good, but is it even possible with wheel packages?

[mgeier]

I just remembered one more thing to try: https://www.lfd.uci.edu/~gohlke/pythonlibs/#sounddevice
This contains its own DLL which is supposed to be compiled with ASIO.
Just for completeness, it would be interesting to know if this also has the same problem.

[DarrenHaba]

I just remembered one more thing to try: https://www.lfd.uci.edu/~gohlke/pythonlibs/#sounddevice This contains its own DLL which is supposed to be compiled with ASIO. Just for completeness, it would be interesting to know if this also has the same problem.

I installed the python 3.9 version sounddevice‑0.4.4‑cp39‑cp39‑win_amd64.whl and can confirm it still interrupts other audio streams on Windows.

[mgeier]

Thanks for confirming that!

What is currently blocking me is that I don't know how to name a custom DLL and where to put it (see #496 (comment)).

Can someone please help?

Once this works and is documented, we can hopefully make progress here.

[mgeier]

A little update:

I have finally installed Windows on a VirtualBox and tried the DLL lookup. I have documented how to do it in #518.

I have also worked on compiling different DLLs with Github Actions at spatialaudio/portaudio-binaries#14.

I'm planning to use the DLLs without ASIO by default for the next release.
People who need ASIO can download the appropriate DLL from GitHub, rename it to portaudio.dll and move it to %SystemRoot%\system32, done!

[dechamps]

Apologies for the drive-by comment, but I would certainly like to understand why you are suggesting people put an application DLL in %SystemRoot%\system32. This is inappropriate and leads to DLL hell. Neither people nor apps should mess around with system directories. Instead, DLLs should be placed next to the executable that depends on them, so that multiple apps depending on different versions/builds of that DLL do not conflict with each other.

If you are suggesting this because some installer is already putting the PortAudio DLL there, then that installer has a bug and should be fixed.

[mgeier]

Apologies for the drive-by comment,

No problem at all, I'm glad for any help I can get! I'm not a Windows user, so I know nothing about the rituals.

I would certainly like to understand why you are suggesting people put an application DLL in %SystemRoot%\system32.

It's really simple: I did a web search for "DLL search path" and found that path on a random page. I tried it in VirtualBox, and it worked, done!

This is inappropriate and leads to DLL hell. Neither people nor apps should mess around with system directories.

I'm open for other suggestions!

Instead, DLLs should be placed next to the executable that depends on them

OK, sounds reasonable, but what is "the executable"?

python.exe?

I'm fine with putting it there, but I don't know if there is an unambiguous way to locate that path on a user's system, especially given about a million different ways to set up virtual environments and Python versions etc.

If you are suggesting this because some installer is already putting the PortAudio DLL there, then that installer has a bug and should be fixed.

No, that's not it, there was no installer involved.

[dechamps]

I am not familiar with how Python manages DLL dependencies of Python packages. I think the cleanest way would be to search for the location (using any kind of search tool) of the existing portaudio.dll DLL and replace that.

The main thing I take issue with regarding your original suggestion is you are telling people to make a system-wide change that can potentially affects all apps on the machine, just to fix an issue with python-sounddevice specifically. This will not end well.

[mgeier]

I am not familiar with how Python manages DLL dependencies of Python packages.

Well, Python doesn't really manage any DLLs, except that I'm using ctypes.util.find_library() (https://docs.python.org/3/library/ctypes.html#finding-shared-libraries) from the standard lib.

I just need users to put portaudio.dll anywhere that's found by find_library().
I have checked that %SystemRoot%\system32 works, and I think the current working directory also works. Other than that, I'm open for new ideas!

If you know any alternatives to find_library(), I'm of course also open for suggestions.

I think the cleanest way would be to search for the location (using any kind of search tool) of the existing portaudio.dll DLL and replace that.

I'm not sure if that's what you mean, but the "existing DLL" is in _sounddevice_data/portaudio-binaries, which acts as a fallback if no DLL is found by find_library().

Here is the whole mechanism:

python-sounddevice/sounddevice.py

Lines 61 to 83 in a7c3259

	try:
	for _libname in (
	'portaudio', # Default name on POSIX systems
	'bin\\libportaudio-2.dll', # DLL from conda-forge
	'lib/libportaudio.dylib', # dylib from anaconda
	):
	_libname = _find_library(_libname)
	if _libname is not None:
	break
	else:
	raise OSError('PortAudio library not found')
	_lib = _ffi.dlopen(_libname)
	except OSError:
	if _platform.system() == 'Darwin':
	_libname = 'libportaudio.dylib'
	elif _platform.system() == 'Windows':
	_libname = 'libportaudio' + _platform.architecture()[0] + '.dll'
	else:
	raise
	import _sounddevice_data
	_libname = _os.path.join(
	next(iter(_sounddevice_data.__path__)), 'portaudio-binaries', _libname)
	_lib = _ffi.dlopen(_libname)

Users can of course overwrite the DLL in _sounddevice_data/portaudio-binaries, but on the one hand that's just a fallback, and on the other hand it's quite hard to describe how to find that directory on a user's system.
Also, it will depend on virtual environments and it will likely be overwritten on updates of my package.

The main thing I take issue with regarding your original suggestion is you are telling people to make a system-wide change that can potentially affects all apps on the machine, just to fix an issue with python-sounddevice specifically. This will not end well.

I think I understand your concern. I simply don't know a better place where to put it (which I can explain to users and which works on a majority of systems out there).

[arkrow]

Sorry for the long absence @mgeier. Unfortunately, I haven't had much success with other alternative solutions, and so was stuck for a bit regarding this issue.

With regards to finding the DLL/library, ctypes.utils.find_library is flexible and has a built-in search, given the name or relative path provided, and tries to find it in each of the directories in PATH (cpython implementation can be found here for reference: https://github.com/python/cpython/blob/main/Lib/ctypes/util.py). If an absolute file path is provided to find_library, then that overrides the PATH search due to the way os.path.join works and is returned as long as the file exists.

From my experimentation on both Windows and Linux, the environment variable implementation suggested in PR #499 works well for both library names / relative paths (if relying on system libraries or searching in PATH directories), and absolute paths (even if the portaudio path provided is not in PATH).

The main drawback of such an approach, would be that developers that rely on sounddevice with ASIO (or alternative binaries), will have to provide their own ASIO-built/alternative portaudio binaries with their application, and set the SD_PORTAUDIO os.environ variable to the correct portaudio binary with respect to the user's platform/arch.

Since the issue with ASIO is exclusively on Windows as far as I can tell, an amendment to PR #498 (i.e., shipping both ASIO/non-ASIO binaries with sounddevice_data and loading the non-ASIO binary by default) would be a simpler implementation with better distribution of binaries with this package.

Or maybe providing both options. Though I'd like to hear your thoughts on this so far.

[mgeier]

@arkrow Thanks for looking up the source for find_library()!

The only reasonably common non-system-wide directory I could find in %PATH% is %LOCALAPPDATA%\Microsoft\WindowsApps.
I have updated #518 to mention that directory and to mention PATH as well.

@dechamps Would you think this is acceptable? Or do you have a better idea?

an amendment to PR #498 (i.e., shipping both ASIO/non-ASIO binaries with sounddevice_data and loading the non-ASIO binary by default) would be a simpler implementation with better distribution of binaries with this package.

Or maybe providing both options. Though I'd like to hear your thoughts on this so far.

I'm still very much skeptical if introducing and checking a brand-new environment variable is really necessary, since the same thing can be achieved by copying portaudio.dll to any directory within the %PATH%.
I certainly don't think it is simpler or easier.

As for providing ASIO and non-ASIO DLLs, that's already prepared in spatialaudio/portaudio-binaries#14.

[arkrow]

The only reasonably common non-system-wide directory I could find in %PATH% is %LOCALAPPDATA%\Microsoft\WindowsApps.
This directory should not manipulated by most users as it is used by the Microsoft Store on Windows 10/11 (for exposing certain Windows 10/11 App binaries to PATH).

The only safe PATH directory for manually adding custom binaries to, would have to be manually created and added for such purposes (e.g. C:\CustomLibraries\). Not to mention, the order of the directory in PATH affects its evaluation. That said, I can't speak for the use cases where a custom portaudio binary is actually necessary, since it's not within my scope and I can't imagine the need to do so.

For the scope of this issue, using DLLs without ASIO is a great step forward for default behaviour and would solve this issue. Thanks for the work on spatialaudio/portaudio-binaries#14.

Now, to provide ASIO-support for those who need it without: (1) messing with the PATH directories; (2) changing the initialization behaviour to be manually called instead of triggering on import, and (3) monkey-patching. The only way left would be through setting an os.environ key before import, which is temporary and only exists internally for the lifespan of the executing python interpreter and its subprocesses only. Something like:

import os

# other imports

os.environ["SD_ENABLE_ASIO"] = 1
import sounddevice as sd

Full disclaimer, though, I have no need for ASIO (in fact, it's the one causing issues due to poor drivers on many Windows PCs), so as far this GitHub issue is concerned, it'll be closed once the new default binaries without ASIO are released.

However, I'd like to encourage anyone who relies on sounddevice with ASIO to add their insights/feedback to this discussion. Perhaps here, or if it's closed, in a new PR/issue.

[mgeier]

The only safe PATH directory for manually adding custom binaries to, would have to be manually created and added for such purposes (e.g. C:\CustomLibraries\).

That's good to know, then that's what I'll recommend in #518.

[mgeier]

I have prepared the removal of ASIO support here: #539.

Please check if that version indeed solves the problem described in this issue.

If everything works fine, I'll release that in version 0.5.0.

[arkrow]

I have prepared the removal of ASIO support here: #539.

Please check if that version indeed solves the problem described in this issue.

If everything works fine, I'll release that in version 0.5.0.

Yes, I've tested both the current live version again, and the one linked, and I can confirm that the wheels linked in #539 fixes this behavior. Many thanks for your work on this issue @mgeier.

[mgeier]

Thanks for testing, @arkrow!

[mgeier]

If everything works fine, I'll release that in version 0.5.0.

Done: https://pypi.org/project/sounddevice/0.5.0/

I think this issue can be closed now. Let me know if something is not working as intended!