Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ableton Live 11/12 Support / Notes-Docs #4

Open
nine7nine opened this issue Dec 19, 2022 · 33 comments
Open

Ableton Live 11/12 Support / Notes-Docs #4

nine7nine opened this issue Dec 19, 2022 · 33 comments
Assignees
Labels
bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request

Comments

@nine7nine
Copy link
Owner

nine7nine commented Dec 19, 2022

In playing around with Live 11 with Wine-NSPA-7.5 onward and I've been able to identify some issues (and fixed many). Moving onto Wine-8.x/9.x I've been able to fix most issues.

  • Slow load of ableton instruments fixed by using -DontCombineAPCs option
  • High CPU overhead on idle may be reduced by using -DontCombineAPCs option (read below)
  • dll / VST loading/unloading crashing was solved by some upstream fixes: 5aaf7ae
  • Slow load on (most) VSTs was fixed by several commits from wine-7.6+: 5aaf7ae
  • Closing of some plugin windows crashing wine was solved by an MR: 1614c65 (deadlock issue)
  • Blank/black popup menus bug in Ableton Live 11 is fixed now, as of commit: a987f3f
  • Any stalling is fixed my RT patchwork && multi-threaded RT Wineserver. (Wine-NSPA only)
  • Switching to SCHED_RR improves performance, reducing lock contention.
  • setup mscv runtimes, winetricks && add dll overrides: Ableton Live 11/12 Support / Notes-Docs #4 (comment)
  • workaround Live 11/12 failing to launch via custom .desktop launcher: Ableton Live 11/12 Support / Notes-Docs #4 (comment) .. but should be fixed with: 4d0644e
  • the odd wined3d crash with specific plugins can be remedied with DXVK: https://github.com/doitsujin/dxvk
  • high CPU usage on idle resolved with 289aa99
  • Fix potential deadlocks in win32u code: 3e10c92
  • MVCRT: File Locking Thread Safety: 7524681
  • Gnome's Tracker-fs service can cause problems. Stop it from searching in ~/Documents/Ableton to avoid high CPU and general problems

That covers the basics and beyond. Ableton Live 11/12 are actually usable now.

What does "usable" mean?

  • I have not encountered any crashes, since resolving initial issues. (it's been very stable)
  • Performance is reasonably good with some caveats: eg: heavy plugins/presets won't magically use less CPU.

TODOs:

  • Investigate other places in Wine where CS aren't initialized, in particular for File Locking. Commit: 7524681 Fixes a long-standing issue in Wine-NSPA that would cause xruns when loading stuff, or doing MIDI program changes with some plugins (for example).

There are bound to be other areas that lack thread safety.


  • DontCombineAPCs

Ableton Live's two main threads use an excessive amount of CPU. As it turns out, Ableton Live has a configuration file that can be modified and has a number of hidden settings. ref: https://github.com/ibekso/medium/blob/main/article_ressources/options_txt/lists/options_available_live_11.txt ... One stuck out to me as being a possible cause of the high CPU usage, so I did some more digging, we have a option relating to APCs;

DontCombineAPCs

Line to add to Options.txt file: -DontCombineAPCs

Deactivates the APC combination mode: won't align and sync the session rings of multiple APCs 
so they can be moved independently.

ref: https://help.ableton.com/hc/en-us/articles/6003224107292-Options-txt-file

Turning on this option eliminates one of the high CPU threads (that causes 30-40% CPU usage on all CPUs).

  • The "APC combination mode" isn't very good for Wine... APCs are still handled by Wineserver's main thread, AFAIK (regardless of fsync or the shm multi-threaded bits), and I believe this mode expects to synchronize APCs across multiple threads (natively).
@nine7nine
Copy link
Owner Author

nine7nine commented Dec 19, 2022

my current options.txt

-DontCombineAPCs
-_ForceGdiBackend
-DisableAutoBugReporting
-AsioDisableMultiClient
-AsioSupportProcessNow
-UseFileSystemCacheForReading
-UseFileSystemCacheForWriting
-SharedMemStoragePath
-UseOwnGeneratedMidiTimeStamps
-MidiEventThinning
-DriveEngineTiming
  • Interestingly, while the -_ForceGdiBackend option does pass correctly, I'm not sure it's actually supported (it used to be, but isn't advertised for Live 11). That said, usually when you pass an unknown or unsupported option; Ableton Live lets you know on startup, but -_ForceGdiBackend seems to be recognized still / allowed... this option was said to help fix performance problems on some machines in Ableton's forums...

@nine7nine nine7nine added the documentation Improvements or additions to documentation label Dec 19, 2022
@nine7nine nine7nine self-assigned this Dec 19, 2022
@nine7nine
Copy link
Owner Author

nine7nine commented Dec 20, 2022

One bit needed to avoid xruns (in Jackd) with Ableton Live is to use a script like this:

https://github.com/nine7nine/Wine-NSPA/blob/main/examples/system/cgprio

  • Setup cgroups audio_rt group and put relevant processes/threads into it
  • handle setting niceness and ionice for RT and Desktop groups
  • make sure relevant threaded irqs are handled correctly (if not using rtirq, which i do not).

some of this really matters as Wine-NSPA supports niceness && Linux-NSPA supports latency.nice (in cgroups)... Strangely, linuxaudio related docs don't cover using cgroups and other handy linux technologies... I try my best to utilize these things.

a few bit in here are system-specific, but can easily be adapted for other setups

NOTE: I probably need to convert this type of script into a systemd service file (that also handles suspend/wake up)

@nine7nine
Copy link
Owner Author

nine7nine commented Dec 20, 2022

blank/black popup menu bug is fixed now, as of commit: a987f3f

@nine7nine nine7nine added bug Something isn't working enhancement New feature or request labels Dec 20, 2022
@nine7nine nine7nine changed the title Ableton Live 11 (Poking the bear) / Notes Ableton Live 11 Support / Notes-Docs Dec 21, 2022
@nine7nine
Copy link
Owner Author

Interesting note about -DontCombineAPCs

  • On my Surface 7 (Intel Core i7-1065G7) the second thread is never running at 99% by default.

I'm not sure if this means that use of this feature is CPU dependent (and may be disabled on my i7... OR if this means that my AMD 3500U cpu just can't use this feature correctly (due to a bug, the cpu, something in wine. idk).

I'll need to test Ableton Live on my Surface 7 more (with and without disabling the APC Combination mode) to make any kind of sense of it. -- Regardless, I will be using my Surface 7 now for Live 11 anyway -> the Core i7-1065G7 destroys the AMD 3500U and the MOBO on the Surface is much better designed than my Lenovo Flex 14.

@nine7nine
Copy link
Owner Author

nine7nine commented Jan 11, 2023

Another important set of details... aside from installing msvc runtimes, gdiplus needs to be installed and also a number of dll overrides:

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Wine\DllOverrides]
"*api-ms-win-crt-conio-l1-1-0"="native,builtin"
"*api-ms-win-crt-convert-l1-1-0"="native,builtin"
"*api-ms-win-crt-environment-l1-1-0"="native,builtin"
"*api-ms-win-crt-filesystem-l1-1-0"="native,builtin"
"*api-ms-win-crt-heap-l1-1-0"="native,builtin"
"*api-ms-win-crt-locale-l1-1-0"="native,builtin"
"*api-ms-win-crt-math-l1-1-0"="native,builtin"
"*api-ms-win-crt-multibyte-l1-1-0"="native,builtin"
"*api-ms-win-crt-private-l1-1-0"="native,builtin"
"*api-ms-win-crt-process-l1-1-0"="native,builtin"
"*api-ms-win-crt-runtime-l1-1-0"="native,builtin"
"*api-ms-win-crt-stdio-l1-1-0"="native,builtin"
"*api-ms-win-crt-string-l1-1-0"="native,builtin"
"*api-ms-win-crt-time-l1-1-0"="native,builtin"
"*api-ms-win-crt-utility-l1-1-0"="native,builtin"
"*atl100"="native,builtin"
"*atl110"="native,builtin"
"*atl120"="native,builtin"
"*atl140"="native,builtin"
"*atl80"="native,builtin"
"*atl90"="native,builtin"
"*concrt140"="native,builtin"
"*gdiplus"="native"
"*msvcm80"="native,builtin"
"*msvcm90"="native,builtin"
"*msvcp100"="native,builtin"
"*msvcp110"="native,builtin"
"*msvcp120"="native,builtin"
"*msvcp140"="native,builtin"
"*msvcp80"="native,builtin"
"*msvcp90"="native,builtin"
"*msvcr100"="native,builtin"
"*msvcr110"="native,builtin"
"*msvcr120"="native,builtin"
"*msvcr140"="native,builtin"
"*msvcr80"="native,builtin"
"*msvcr90"="native,builtin"
"*msvcrt40"="native,builtin"
"*ucrtbase"="native,builtin"
"*vcomp"="native,builtin"
"*vcomp100"="native,builtin"
"*vcomp110"="native,builtin"
"*vcomp120"="native,builtin"
"*vcomp140"="native,builtin"
"*vcomp90"="native,builtin"
"*vcruntime140"="native,builtin"
"atl"="native,builtin"
"concrt140"="native,builtin"
"msvcp140_atomic_wait"="native,builtin"
"msvcrt"="builtin"
"msvcrt40"="native,builtin"
"vcruntime140"="native,builtin"
"vcruntime140_1"="native,builtin"

the above is a registry key that I exported from my lenovo laptop and imported onto my Surface 7 Laptop... After running winetricks, plus manually installed the x64 mscv runtimes && importing the above .reg file -- Ableton Live doesn't crash anymore on my Surface (which is a clean wine prefix / install).

special note:

"msvcp140_atomic_wait"="native,builtin"

the above seems to be needed to avoid some VST3 crashing in Ableton Live, when closing the window (Massive X for example)

@nine7nine
Copy link
Owner Author

nine7nine commented Feb 15, 2023

odd...

  • after setting up a fresh Arch install / fresh wine prefix on my one laptop; Ableton Live 11 doesn't launch
  • Even more odd: it launches just fine using the normal launcher on my other laptop.

after some digging, I noticed it launches fine form the command line. Easy fix: just create a new .desktop file

[Desktop Entry]
Version=1.1
Type=Application
Name=Live 11 Lite
Comment=Ableton Live 11 Lite
Icon=1FE6_Ableton Live 11 Lite.0
Exec=wine start /unix '/home/ninez/Winebox/winebox-master/drive_c/ProgramData/Ableton/Live 11 Lite/Program/Ableton Live 11 Lite.exe'
Actions=
Categories=AudioVideo;
StartupWMClass=ableton live 11 lite.exe
  • simply use the full unix path (from wineloader) and it starts fine.
  • this creates a "Live 11 lite" app entry in Sound & Video menu (wine-programs entry can be deleted or hidden)
  • needs to be placed in/as /usr/share/applications/Live11.desktop

@ntippie
Copy link

ntippie commented Jun 26, 2023

Great documentation, thank you.

Perhaps there is more to learn from the Push 3 standalone which runs on Linux.

Have you considered WINE compatibility with Ableton Push? I tried using a Push 2 with Ableton via Wine but Push2DisplayProcess.exe would fail. I now have a Push 3 which I have not tried with WINE, but it seems to use a different executable.

@nine7nine
Copy link
Owner Author

Well, at the moment - Wine-8.x is kind of broken, in terms of getting good performance + due to limited time, atm... I'm not actively working on Ableton Live support... that'll likely change in the future, but for now I'm a bit busy (with work / life). Kind of waiting for Wine Upstream to iron out some regressions...

I know nothing about Ableton Push (other than what it is / does)... but if it requires some windows driver to function - that DOA in Wine. Highly unlikely to work properly... IIRC, some of the push devices are supported by Linux though. (eg: with Ardour)

@nine7nine
Copy link
Owner Author

Latest Wine-NSPA-devel (8.14+) are starting to work reasonably well... all of the nasty bugs that i was facing in 8.x builds have largely disappeared.

I've been playing around with Ableton Live - no crashes or anything, still slightly higher CPU usage than Wine-7.5.x builds, but overall much better than a couple of months ago...

So i can work on getting a stable 8.x release, along with improving Ableton Live use.

@revast
Copy link

revast commented Sep 2, 2023

Nice to see that 8x finally starts to work! I will try to build 8x soon, will report back if something is bogus. Going with a fresh arch install for now, but for maximum compatibility with my setup I will also try to get it into Debian/Ubuntu. I struggled with converting the arch custom kernel package to a debian package, so I probably will have to manually add the patches to my preferred kernel, if possible...

@nine7nine
Copy link
Owner Author

@revast

Ya, it is nice to see that 8x builds are starting to work. Most of the issues were upstream problems, all of the PE conversion stuff, the heap stuff and CRT changes were introducing a lot of bugs (crashes, instability), but it seems that that has all been worked through now... eg: Kushview Element used to not even start/crash on Wine-8.x for a while, but now it works fine... I'd get other issues with apps freezing or 2x CPU usage with some VSTs -- but again, that all seems to not be a problem anymore...

Unfortunately, this kind of thing just happens sometimes with Wine. ~ I've been using Wine since pre-1.0, and I've had custom builds of wine for almost as long. ~ the goal is always the same, but sometimes the patchwork changes (eg: old wine i used to use an entirely diffferent set of patchwork for synchronization primitves. pre-esync or fsync... RT patch used to be the one in Wine-Staging, etc)... Wine is a moving target. sometimes things work well, sometimes not... sometimes new ways of doing things are needed.

Sorry, that i can't resolve the debian package issues for you -- I strictly use Arch, unfortunately... but assuming you use the same version kernel that my patches apply over (and don't use other patchwork that cause conflicts and/or can resolve conflicts) -- it should be that hard to make a custom kernel package for debian/ubuntu.

@revast
Copy link

revast commented Sep 3, 2023

Yeah thanks 4 insights...
I also played around with Liquorix and Xanmod Kernels, both in AUR btw. - Do you think those patches could help performance-wise? Or have you tried these already?
How about pipewire native driver? Would this bring in some advantages? If not now, maybe when IRQ based scheduling like what jack does lands? From comments, it looks like its already committed in wine gitlab...

package issues: I will try makedeb -> someone built this on top of the makepkg code, to generate debian packages from PKBUILD files

@nine7nine
Copy link
Owner Author

nine7nine commented Sep 3, 2023

bah, i wrote a detailed reply, but it seems to have disappeared.

  • Liquorix/Xanmod: not a lot of interest there, given their goals are different. Xanmod also uses some code that i simply don't see any value in / or is irrelevant.

  • Pipewire: Not good enough / I still get better performance/latency and reliability out of JACK... Pipewire would also require an ASIO or WDM Exclusive Mode driver for me to consider using it... The driver you linked to it not that, at all... Windows proaduio applications behave differently depending on the backend used, and i suspect the Pipewire backend is a bit closer to the WinePulse backend (from taking a quick look at the code). not what i want, at all.

One day I likely will swtich to Pipewire, but I'm not really in a rush. It does offer some advantages over my Pulse/JACK setup, but also some regressions (and those are regressions that directly affect my use with Wine-NSPA / and proaudio on linux, in general -- so it's not the best time for me to try and switch over).

  • regarding makedeb - no idea... the 'non-makepkg-build.sh' script does mention some pain with .deb based systems... but given that i use Arch -- I don't even look at that stuff, really.

@revast
Copy link

revast commented Sep 10, 2023

I like details ;-)

Pipewire: Not good enough / One day I likely will swtich to Pipewire, but I'm not really in a rush.

me neither, I am also waiting for IRQ based scheduling.

Pipewire would also require an ASIO or WDM Exclusive Mode driver

Do you mean the "pipewire-implementation" in Wine must be capable of doing the the same as WineAsio or must it work well with WineAsio? because there are some bug reports about the latter which I can find...

Liquorix/Xanmod: not a lot of interest there

good to know that you already have evaluated these...

regarding makedeb

I will probably stick with a Arch setup anyways for now - so I can also follow your development tighter.. There is also distrobox to chime in some other distro's stuff if needed...

So I guess standard procedures will needed to be done, all realtime stuff (á la realtimeconfigquickscan) define IRQ Priorities, disable unneeded stuff, run latencymon in windows / osjitter / hiccups / numbers on the machine to find hardware which gets in the way....

Probably also coping with tuning guides, turn mitigations off, rtla-tool ...CPU pinning ulatencyd, linux-music-utils, numactl...bcache, zram/zcache,, run audiogridder linux server locally to pin vst's to specific CPU cores... just throwing in some things here I encountered..

@nine7nine
Copy link
Owner Author

  • re: Pipewire - I mean that any 'pipewire' dirver would have to work like WDM Exclusive mode for it being something that could be used directly, instead of WineASIO. (and obviously, wineASIO would need to run flawlessly with Pipewire too).

  • Yeah, Arch is easier: I just don't have time to bother with other distros (unless it's work-related). I have time-constraints on when i can work on this stuff: Career, Family & life... and then I also (ATM) only have one PC that I'm running this stuff on. That said, I do always manage to find some time to work on side-projects like this.

  • re: tuning guides. I can't comment too much. I have a pretty simple script/service I run: Set all of the relevant IRQ rtprios, setup Cgroups, all of the nice/ionice bits, etc... Don't need numactl or any of that jazz... anything wine-related it set in /etc/environment. it's all very straight-forward really.

that aside, I'm just selective about what i run in my userspace, as well. avoiding stuff that wants to periodically run in the background, or that may introduce unwanted effects...

@revast
Copy link

revast commented Sep 21, 2023

there is also someone trying to patch wine's pulseaudio-driver for getting low audio latency, maybe from interest
https://blog.thepoon.fr/osuLinuxAudioLatency/
https://gitlab.com/tasgon/wine-osu

@nine7nine
Copy link
Owner Author

the osu audio patches are outdated and unnecessary, given it's jack, WineASIO, etc that are being used here / with proaudio stuff... totally aware of the osu audio hacks.

I have patchwork for hacking on both the alsa and PA wine drivers ~ just haven't rebased them, since wine-devs recently reworked / moved around some of that driver code.

@deusnovus
Copy link

deusnovus commented Nov 1, 2023

I don't know if this is of any help, but in regards to Pipewire in Ableton Live, there's this: https://copr.fedorainfracloud.org/coprs/patrickl/pipewire-wineasio/

I'm currently trying to get this working, in order to get low latency on Ableton Live, but I had to downgrade wine from 8.18 to 8.4 due to a seemingly known bug that fails to open any input/output, whereas it works okay on 8.4, but the WineASIO script cannot be built on older wine-staging.

Also, CPU temperatures are insanely high on vanilla Wine, even on idle. I really want to follow this Ableton Live 11 guide, but I'm really new to Linux audio, so I don't understand most things here. Any pointers on how to achieve low-latency audio and decent CPU temps would be greatly appreciated.

@nine7nine
Copy link
Owner Author

nine7nine commented Nov 1, 2023

Unfortunately, I don't use Fedora, but from the look of it: that WineAsio package simply links against pipewire's jack lib...

ATM, I also don't use Pipewire for anything Jack-related... I have recently replaced PulseAudio with Pipewire, but Pipewire is running on top of Jack (via pipewire-jack-client). ~ which means I am using Jack in the end (not pipewire's jack implementation)... I will probably test out Pipewire as a jack replacement when Pipewire-1.0+ is available in Archlinux repos. Until then, i won't touch it.

It's a bit unfortunate that you are somewhat of a newbie: as I wouldn't say setting up a linux machine for proaudio / really low-latency and messing around with custom wine-builds + running tricky software like Ableton Live is very newbie friendly, and I'd still say it's a bit experimental - although, I do run Ableton Live fine on my machine, atm.

The only real advice I can give is making sure your system is actually setup nicely for prouadio, before trying anything like this. That means following whatever Wiki for your distribution that covers this stuff. For running something like Ableton Live or very new VSTs - you also need a new / powerful PC. it's not going to work well with older hardware... and honestly, I would suggest not even dealing with Live 11 - if you have a better option: such as running a native DAW + yabridge (if you need windows-specific VSTs).

there is still the odd pain-point with running Ableton Live in Wine, and as you can see from above ~ it's not-trivial to do in the first place (custom builds of wine, needing XYZ ms runtimes + dll overrides, messing with ableton's hidden options -- on top of configuring your machine correctly, in the first place).

in any case; rolling back my builds to Wine-8.4 isn't happening. Wine-8.4 was busted for my use-case, things weren't really ironed out until 8.10+ ~ so i definitely will NOT be going back or rebasing patches for a version of Wine that had stability issues for me.

@deusnovus
Copy link

Thanks for responding! I mean, I'm experienced enough to understand technical terms and following guidelines, but obviously not enough to create and optimize a custom wine build.

I built a fairly strong computer last month and Ableton Live 11 and VSTs run fairly well, with the exception of slow boot times, M4L glitches and the I/O bug on Wine-8.18. While I can deal with these, the issue of latency and high CPU temperatures worry me. I did try the Bitwig + yabridge combo and it's perfectly serviceable, but I've been using Ableton Live since 2015 and I'm really deep into its workflow, so it would have been awesome if I could use it on Linux as well.

Also to clarify, I also want to use the latest Wine version just like everybody else, if it wasn't for that I/O bug in Ableton Live.

@nine7nine
Copy link
Owner Author

nine7nine commented Nov 1, 2023

  • Ableton Live will always start slow. There is no avoiding that, I believe it has to do with it's 'checks' on startup (eg: for VSTs) and for whatever reason -- that just takes longer with Wine.

  • The Latency issue (assuming the rest of your system is configured well), is almost certainly caused by a couple of things: No Fsync support. No RT scheduling support && no multi-threaded Wineserver... Trying to set lower latencies just doesn't work well without all 3 of the things I've mentioned... you just can't get the performance without Fsync, threads aren't prioritized correcty without RT support -- and wineserver gets hammered without being multi-threaded.

I typically have Ableton Live 11 set between 5.33ms and 10.7ms on my system - and that works fine (even with my internal soundcard, to an extent).

  • re: Idle + CPU usage. That won't fully go away, even with my builds. but depending on the CPU + some of the options.txt stuff (eg: APC settings) it can be reduced significantly. But one or two threads will always be consuming a chunk of CPU (one Winserver thread, one Ableton thread)... but it doesn't actually mean that that percentage of CPU is fully utilized, really. ~ at least not in terms of Ableton Live processing. eg: a couple of threads may consume 15% CPU, but when i start playback, or playing instruments in Ableton Live - it may sit at that relative 15% CPU consumption, until i exceed a certain DSP Load. (if that makes sense?)

  • I mean you can try to build Wine-NSPA on a non-Arch distro. It IS possible, but I've never tested it. Wine-NSPA is based on Wine-TKG: i just use a whole bunch of patchwork they do not...

https://github.com/Frogging-Family/wine-tkg-git/tree/master/wine-tkg-git
https://github.com/Tk-Glitch/PKGBUILDS/wiki/wine-tkg-git

it does mention building on Fedora.

That all said: if you are really tied to Ableton Live - why not just use it on Windows? you'd avoid a lot of headaches.

@deusnovus
Copy link

Thank you for the detailed answer, I really appreciate it.

Unfortunately, as you would imagine, Wine in Fedora isn't all that well documented, as opposed to the Archwiki for instance and I wouldn't trust myself with tasks I don't fully understand, so I think I'll use Bitwig + yabridge until any further developments with Ableton Live, Wine etc and hopefully, I'll also become more experienced in programming to help out someday.

That all said: if you are really tied to Ableton Live - why not just use it on Windows? you'd avoid a lot of headaches.

I was a macOS user since 2006 and last year, I fell in love with Linux and GNOME in specific, so I'm pretty dead set on being on Linux for the foreseeable future.

@nine7nine
Copy link
Owner Author

No problem. I do think Bitwig + Yabridge is probably the better option. It's just less hassles / less problems.

For me, I started using Ableton Live in Wine to help analyse and improve performance in Wine. - Heavily multi-threaded apps are good for that.... it's been helpful, and i do think there is still a lot that can be improved -- but most of that will continue to come from non-vanilla wine patchwork (which is why i patch + build myself).

It may not be that hard to build Wine in fedora. You really would just need all of Wine's dependencies and related development packages... after that, building wine-nspa would be pretty simple (modifying one config file, executing the build script and waiting)... unfortunately, i haven't used Fedora since 2008 or something - so i'm just not up on the details, even with how things are packaged in the repos.

@revast
Copy link

revast commented Jan 24, 2024

Looks like there is some move with the NT Sync Part: https://www.phoronix.com/news/Windows-NT-Sync-RFC-Linux

re: tuning guides. I can't comment too much. I have a pretty simple script/service I run...

is it possible that you share (some) of that into a github gist? I am trying to replicate your setup as thoroughly as possible..

@nine7nine
Copy link
Owner Author

@revast - yeah, the NT sync / winesync code is now beginning it's initial review. ~ I belong to wine-devel, LKML, etc mailing lists - so I did see this in my inbox / I'm following it.

IMHO, this will be a lengthy process - I don't expect this kernel driver to be upstreamed anytime soon (nor into Wine). It'll take time, there will be a lot of feedback + revisions to the kernel code && Wine before it ever is accepted.

I've played around with NT Sync (not in a couple of months though). It caused some stalls/freezes wiht Kontakt and some other VSTs I use ~ you'll also note the Phoronix article doesn't show benchmarks of NT Sync vs. Futex2, which may paint a different story vs. upstream wine.

I'll likely give NT sync another spin, after I rebase on Wine-9.0+ (which I will do soon, likely within a month or something). ~ I'll see where it's at then, and provide builds with it (but will likely disable it by default, preferring Fsync/Futex2).

@nine7nine
Copy link
Owner Author

nine7nine commented May 7, 2024

Playing around with Ableton Live 12, along with my LibRTPI patchwork -- it's become pretty obvious where/what causes Ableton's one TC thread && Wineserver's one shmem thread to consume so much CPU -- it has to do with epoll_wait + FUTEX_WAKE/FUTEX_WAIT in Winserver. (In Wine-NSPA-pi build it's easy to spot: those are the only 2 non _PRIVATE / _PI futexes left, after mutex conversions).

One idea I have is building a 'Monitor' synchronization mechanism - which can be built using pthread mutexes and condition variables, this could eliminate WAKE/WAIT in the server requests and potentially fix the high CPU usage of the threads. They would no longer be busy-waiting... I have no control over Ableton, but it would get rid of the spinning in wineserver, either way.

It's going to take a bit of thought, research and refactoring. but it's plausible. Furthermore; it would be compatible with LibRTPI, as I can just convert the mutexes/locks- reaping the PI benefits, as well...

AFAICT: Fastsync/NTsync will do nothing to fix this problem.

@deusnovus
Copy link

Playing around with Ableton Live 12

Hello again, it's been a while. I wanted to ask if you also encountered that "Starting Max..." boot hang on Ableton Live 12. It seems a lot of people have had the same issue which is "solvable" by renaming the Max folder into something else, but then you won't be able to use M4L at all. Thank you for your research!

@nine7nine
Copy link
Owner Author

nine7nine commented May 9, 2024

ya, i saw that. thanks!

Personally though: I won't consider seriously using Ableton Live in Wine until I resolve the above issue (the busy-waiting high-CPU problem in Wineserver)... Frankly, I just don't think it's worth using until then. At least on my box: it's burning a lot of cpu cycles / mucho overhead. (and obviously without using the multi-threaded wineserver: i wouldn't try using Ableton at all. IMHO, it's a complete waste of time -- because it's easy to stall wine/Ableton without it - to many server requests to a single thread).

I am working on a patch though; I mostly have the Monitor thingy worked out, but currently it's getting stuck. It's not so trivial to implement. However; if/when i kill the busy-waiting: Ableton Live will become more viable (basically no overhead)... That said; I expect this is going to take a few more failed attempts / bit of frustration along the way, before I get a working implementation.

@deusnovus
Copy link

No no, I've been on Bitwig for the past year and it's perfectly serviceable! It's just that, you know... the grass always being greener on the other side and all that jazz hahaha -

For what it's worth though, Ableton Live 11 Suite is running absolutely spectacular via Patrick's wine-tkg corp on Fedora. Something about Wine v8.18 that solved many relevant issues and made Live 11 run almost at native level back when it was released. But regardless, working full-time on a DAW via Wine started to feel wrong, so I eventually switched to Bitwig.

@nine7nine
Copy link
Owner Author

Yeah, upstream development during Wine-8.x fixed most of the issues with Ableton Live (black menus and crap like that). That said; Wine-TKG is actually missing some Fsync related patchwork (in non-Proton builds). I have plugins (like NI Kontakt, certain U-He plugins) that run substantially worse without them. Patrick's build is affected by that.

Likewise, his builds are running a single-threaded Wineserver; -- and Ableton Live still will have high CPU on one of it's threads. So usable? maybe... But still very problematic / won't scale well. Not if running a ton of tracks with heavy plugins DSP + low latency. There's really no solution to that in any build of Wine, ATM. In mine, if I can replace the spinning -- that'll probably fix it. But it's proving very difficult to do. There's really only a couple of options; building a Monitor synchronization mechanism, using semaphores or maybe spinlocks... Each has there own specific-issues/difficulties to get working properly.

Yeah, Bitwig is very good, and probably the best option for a Native Ableton Live-like DAW on Linux. For my own use-case: While I have been experimenting with Ableton (and it's been helpful in that way; testing, finding issues), I primarily use my VSTs in Kushview Element (with carla-single + yabridge). Basically, running them in containers... More live-play with some midi/ recording (controlled from DAW, routed in). I haven't had any issues with my setup in a long-time... It's really just ableton that exposes the spinning/scalability problem in an obvious way.

@kz-n kz-n mentioned this issue Jul 2, 2024
@nine7nine
Copy link
Owner Author

nine7nine commented Jul 13, 2024

Update: Latest builds of Wine-NSPA no longer has excessive CPU usage in Ableton Live. On idle, Ableton Live / wineserver should consume very little CPU. It came down to DWM behavior where a thread couldn't sleep, which then caused deeper issues within wineserver to do with KERNEL_APCs and async io.

This was the last show-stopping bug for running Ableton Live in Wine-NSPA.

However, due to some other recent Shell32 changes in Wine-NSPA: Ableton's stupid VST Scanner (that can no longer be disabled on launch via options.txt in Live 12) - actually works now (ugh!). I'm looking into a way to prevent it running on start (maybe some kind of hack in wineloader to detect and stop it on launch). not sure exactly how to handle that yet -- trying to kill it from linux-side just results it it respawning, so this must be done within wine / another way.

That said: once I resolve the vst scanner issue -- this issue will be closed!

@nine7nine nine7nine changed the title Ableton Live 11 Support / Notes-Docs Ableton Live 11/12 Support / Notes-Docs Jul 13, 2024
@nine7nine
Copy link
Owner Author

nine7nine commented Jul 18, 2024

Update:

I'm working on thread safety in Wine-NSPA. One immediate result with Ableton Live (and various VST/VSTi), there are no more xruns with certain file operations. This includes loading/switching plugins, or switching presets while a project is playing in Ableton Live with DSP load... There may be exceptions, but it's a significant improvement.

xruns still happen when loading projects, but that (hopefully) will change soon. I'm planning to go through and audit Wine-NSPA's codebase -- make as much of it thread-safe as possible... Starting with all of the file ops related stuff, as it's the biggest pain-point for that. I have a pretty good idea on where to look / what patterns to look for -- So i am hoping I can get these cases all handled.

big news! (latest code has the fixes)

@revast
Copy link

revast commented Aug 1, 2024

Very Nice! Recompiling now...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants