Refactor PortAudio backend

[sakertooth]

Applies various changes in an attempt to resolve certain issues with the PortAudio backend and improve the quality of the code.

This PR fixes DirectSound and MME when using PortAudio, but for some reason WDM-KS and WASAPI still lay down broken. This might have to be looked into later as I couldn't find a solution.

[bratpeki]

How does one test this? Should I play two demo tracks and look at the latency?

[bratpeki]

Here's my config, @sakertooth, with recent files and directory paths obscured:

<?xml version="1.0"?>
<!DOCTYPE lmms-config-file>
<lmms version="1.3.0-alpha.1.740+g303215f8b" configversion="3">
  <MidiJack device="lmms"/>
  <app loopmarkermode="dual" nanhandler="1" openlastproject="0" nommpz="0" sololegacybehavior="0" configured="1" displaydbfs="1" disablebackup="0" language="sr"/>
  <audioengine mididev="WinMM MIDI" framesperaudiobuffer="256" audiodev="SDL (Simple DirectMedia Layer)"/>
  <audiojack clientname="lmms" channels="2"/>
  <audioportaudio backend="" device=""/>
  <audiosdl inputdevice="" device=""/>
  <midi midiautoassign="none" autoquantize="0"/>
  <paths ladspadir="..." workingdir="..." stkdir="data:/stk/rawwaves/" defaultsf2="..." sf2dir="..." vstdir="..." theme="data:/themes/default/" backgroundtheme="" gigdir="..."/>
  <tooltips disabled="0"/>
  <ui saveinterval="2" enablerunningautosave="0" displaywaveform="1" vstembedmethod="none" animateafp="1" smoothscroll="0" sidebaronright="0" trackdeletionwarning="0" printnotelabels="1" mixerchanneldeletionwarning="1" compacttrackbuttons="0" disableautoquit="0" vstalwaysontop="0" oneinstrumenttrackwindow="0" enableautosave="0" letpreviewsfinish="0"/>
  <recentfiles>
  ...
  </recentfiles>
</lmms>

[bratpeki]

Saker claims this is merge-ready, as there is little to be done left.

MSYS compilation yields four backend, with the following results, after some crude testing:

1. For WDM-KS, the output is "Could not open PortAudio: Invalid device"
2. For WASAPI, the output is "Could not open PortAudio: Invalid sample rate"
3. DirectSound works
4. MME works

Additionally, with WDM-KS, with certain outputs selected, LMMS won't report the above message, but will instead hang and result in the following output:

Lv2 plugin SUMMARY: 0 of 0  loaded in 0 msecs.
(This is where LMMS hangs after being disabled. Ctrl-C shows the below text)
QObject::killTimer: Timers cannot be stopped from another thread
QObject::~QObject: Timers cannot be stopped from another thread

[bratpeki]

That should be it, really. Let me know if anything more is needed!

[bratpeki]

Checked again!

For WDM-KS, the output is "Could not open PortAudio: Invalid device"
For WASAPI, the output is "Could not open PortAudio: Invalid sample rate"
DirectSound works
MME works

Still true, all of it, including the hanging.

However, 2/4 backends now work well, which I think is great!

Please merge ASAP. 🚀

[sakertooth]

Please merge ASAP.

GitHub demands your blessing with an explicit approval before I can merge (in general though it needs one approver before merge, at least for this repository in particular currently). I mentioned one last test if you want to take that on though, which is just checking if MSVC builds work for WASAPI and WDM-KS. Here's a link to them for this PR.

[bratpeki]

Tested with MSVC. Pretty much the same as MSYS2:

• MME works.
• DirectSound works.
• WASAPI doesn't create sound.
• WDM-KS doesn't create sound. It also hangs the program, so it remains active in the Task Manager after the program is closed from the GUI.

Gonna test using MinGW again as well.

[sakertooth]

Tested with MSVC. Pretty much the same as MinGW:

• MME works.

• DirectSound works.

• WASAPI doesn't create sound.

• WDM-KS doesn't create sound. It also hangs the program, so it remains active in the Task Manager after the program is closed from the GUI.

Gonna test using MinGW again as well.

This is truly incredible

[bratpeki]

Corrected #7444 (comment).

MinGW features only two backends:

• MME works.
• WDM-KS doesn't create sound. Also, unlike MSVC, it doesn't hang the program? Odd!

[bratpeki]

After trying both, it seems neither of them generate the hanging behaviour, LOL!

In any case, I'd say that PortAudio is fixed, for the most part. It doesn't chop like it used to, and the ability to specify the channel count has been added. I would investigate this further outside of this PR and we can make future improvements to PortAudio.

[bratpeki]

Approved from a tester standpoint; I haven't looked at the code.

[bratpeki]

Yup, I can't reproduce the hanging now, somehow! Must be something to do with running MinGW before/after MSVC, that's my best guess.

[tresf]

@bratpeki asked me on Discord to test macOS support.

• ✅ PASS: Sound output: Testing demos/unfa-Spoken.mmpz, built-in speakers: Playback is fine
• 🚫 FAIL: Testing with Apple Airpods, each airpod shows as a single 1-channel device (sometimes they report as 0-channel and 2-channel respectively), playback is broken, closing and reopening the software will always re-prompt for the device.
  • Master shows two devices as well, but playback is possible and it does not re-prompt for the device when closing and reopening

[sakertooth]

Pushed the last few fixes I had in mind. Will merge soon if there are no objections. Any other potential issues might have to be looked at later.

[bratpeki]

There is definitely the issue stated by @tresf.

[tresf]

Pushed the last few fixes I had in mind. Will merge soon if there are no objections. Any other potential issues might have to be looked at later.

They aren't potential issues, they're actually issues because the behavior regresses from master.

Here's what the terminal says:

Could not open PortAudio: Invalid number of channels
No audio-driver working - falling back to dummy-audio-driver
You can render your songs and listen to the output files...

I wasn't able to get this message to reproduce reliably, but it's the closest thing to a usable error I was able to obtain. No matter of configuration will allow the AirPods to work on this PR.

[sakertooth]

I set the default to 2 in the code, so I'm confused.

[tresf]

I set the default to 2 in the code, so I'm confused.

Do you mind linking to the code?

[sakertooth]

Do you mind linking to the code?

I made some more sweeping changes because I didn't like the scoping issues (i.e., having to handle both input and output logic at the same time). Instead, I encapsulated this logic into classes.

More importantly though, I stopped updating the config after the stream has opened. I'm not sure if that will fix the problem, but I couldn't find any other issue.

[tresf]

Do you mind linking to the code?

I made some more sweeping changes because I didn't like the scoping issues (i.e., having to handle both input and output logic at the same time). Instead, I encapsulated this logic into classes.

More importantly though, I stopped updating the config after the stream has opened. I'm not sure if that will fix the problem, but I couldn't find any other issue.

That fixed it for the output device, thanks! Note that for the built-in Mic, I confirmed that it should be 1 in the screenshot below, however please keep reading...

As it turns out, both the built-in mic on the laptop and the AirPods are stereo output (2-channel) but mono input (1-channel). So this is behaving properly.

Next, I tested with a stereo microphone... Although it properly detects two channels, it defaults to only one, possibly due to the default device only having one. I would expect this to default to 2 for a stereo microphone.

[sakertooth]

Next, I tested with a stereo microphone... Although it properly detects two channels, it defaults to only one, possibly due to the default device only having one. I would expect this to default to 2 for a stereo microphone.

const auto defaultNumChannels = QString::number(DEFAULT_CHANNELS);
const auto selectedNumChannels = ConfigManager::inst()->value(tag(), channelsAttribute(m_direction), defaultNumChannels);
m_channelModel.setValue(selectedNumChannels.toInt());

This is the code that gets ran when we need to populate the combo boxes for both the input and output devices. DEFAULT_CHANNELS equals 2, so it should be defaulting to stereo. Even when you deleted .lmmsrc.sml it was still defaulting to 1? If the channel attribute is specified in the config, that will always be used. If it isn't, then by default it is set to 2, and then when the user presses "OK", it saves whatever value was in the channel to the config.

[tresf]

I tested again and it selected the Yeti by default this time and it showed 2, but switching to a 1-channel mic and back shows the bug. (if the video is blurry it may still be processing)

https://youtu.be/itXMo5HQDTk

[sakertooth]

I tested again and it selected the Yeti by default this time and it showed 2, but switching to a 1-channel mic and back shows the bug. (if the video is blurry it may still be processing)

https://youtu.be/itXMo5HQDTk

Thanks, the video helped. This happens because when you switch devices, only the channel range (i.e., the allowed minimum and maximum channel range) are updated. The channel value itself isn't updated because PortAudio only provides the maximum number of channels, but not a value that says "this is the number of channels this device is expected to use".

Should we be always trying a default value of 2 channels when switching devices?

[tresf]

The channel value itself isn't updated because PortAudio only provides the maximum number of channels, but not a value that says "this is the number of channels this device is expected to use".

Hmm... well at some point the code path assumed that 2 was OK, then a code path assumed that 1 was OK. I think preferring 2 if supported is a good default for now. We can't really know what the future will hold in regards to this (the future recording) feature, but I can say that I've always recorded in stereo when available, but this may not represent the masses. Regardless, having a default code path that's consistent is always preferred, easy enough to change later.

[bratpeki]

I can see this is being worked on very actively. I'll reassign myself and test Linux and Windows when it's time. Please request a review then!

[sakertooth]

Hmm... well at some point the code path assumed that 2 was OK, then a code path assumed that 1 was OK. I think preferring 2 if supported is a good default for now. We can't really know what the future will hold in regards to this (the future recording) feature, but I can say that I've always recorded in stereo when available, but this may not represent the masses. Regardless, having a default code path that's consistent is always preferred, easy enough to change later.

That happened because the channel count wasn't specified in the config, so it defaulted to 2. The second time around, it was in there, so the code just used that value instead. However, I do agree that its okay to default to 2 when switching devices (which gets clamped down to 1 if the maximum is 1, so we're fine in that regard).

[sakertooth]

Wait nevermind, scratch off what I just said... it wasn't saved to the config since you never pressed "OK", so it should've used 2... I'll have to take a closer look.

[sakertooth]

Hopefully its fixed now.

[tresf]

Tested on macOS Sonoma, passes with mono and stereo inputs. Playback works as expected.