Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proper surround sound playback with video #1581

Open
ThreeDeeJay opened this issue Aug 13, 2021 · 19 comments
Open

Proper surround sound playback with video #1581

ThreeDeeJay opened this issue Aug 13, 2021 · 19 comments
Labels
enhancement New feature or request player-related Player-related changes are currently on hold due to the ongoing player migration.

Comments

@ThreeDeeJay
Copy link

Is your feature request related to a usage problem (not a bug)? Please describe.
To my surprise, FreeTube supports 5.1 surround playback, but requires Settings > Default Video Format > Audio Formats so higher audio bitrates are selectable. However, this will freeze video playback since it's an audio-only mode.

Describe the solution you'd like to see implemented
Allow selecting highest/surround bitrates during regular video playback

Describe alternatives you've considered
Not really solutions but here are other methods of YouTube surround playback I've tried:

Screenshots
https://i.imgur.com/r20Uqfb.mp4
Recorded using virtual surround so anyone can listen to the surround effect on regular stereo headphones.

Additional context
On a side note, YouTube also supports spatial audio, which requires less channels (4) yet is capable of 3 dimensions (with height) thanks to the Ambisonics format, which is supported by OpenAL Soft, an API that can render it on headphones in case VR/360 support ever gets implemented and plugins like SoundField, which allow decoding it into virtual surround.

@ThreeDeeJay ThreeDeeJay added the enhancement New feature or request label Aug 13, 2021
@ThreeDeeJay
Copy link
Author

Here's the original 5.1 video I used to record the virtual surround clip, just in case: https://youtu.be/PqVCPE8_ntE

@GilgusMaximus
Copy link
Contributor

It surely looks like a good idea, but with whatever the player is doing with qualities and stuff, my intuition will be that this will go very bad. So if someone attempts to add more complex quality selection to support this, please contact us first in the dev channel to clear things up or write a detailed pr

@mbirth
Copy link

mbirth commented Oct 30, 2021

This sounds a lot like it depends on #1460 for video-vr , adding OmniTone and videojs/videojs-vr#69

@ThreeDeeJay
Copy link
Author

While I'm not particularly experienced in the code, I propose OpenAL Soft for spatial audio rendering. It already supports First-Order Ambisonics, which is the format used by YouTube.
OpenAL Soft is also capable of rendering virtual surround, which allows hearing the surround effect on headphones like this, without needing a speaker setup or proprietary hardware/software.

@ThreeDeeJay
Copy link
Author

Something I noticed the other day.
Opus audio codec track is the default (and arguably highest quality) BUT it doesn't seem to be used for surround tracks, only some AAC, AC-3 and EC-3 have 6 channels (5.1 surround)
So perhaps all we need is to be able to manually switch/select specific audio tracks (specifically one with 6 channels).
Here are some confirmed 5.1 surround videos for testing: https://youtu.be/Yd7pZRhAcW8?list=PLVDbhbTh1d8fFGCQuz0hsIdV2HvnvgQR3

@github-actions
Copy link
Contributor

github-actions bot commented Nov 2, 2022

This issue is stale because it has been open 28 days with no activity. Remove stale label or comment or this will be closed in 7 days.

@ThreeDeeJay
Copy link
Author

Forgot to mention, there are Opus tracks with 6 channels, but I've only seen them in VR/360 videos with spatial audio (4 channels for First Order Ambisonics + 2 channels for head-locked stereo) which could be used for surround but would require decoding ambisonics first.

@absidue
Copy link
Member

absidue commented Nov 13, 2023

Just leaving some comments here.

Using OpenAL Soft is not possible, as it's native code and FreeTube is written in JavaScript.

Also from my testing and the chromium docs, AC-3 and EC-3 aren't supported by Electron, so we can't play them in FreeTube (FreeTube uses Electron, which is a set of patches on top of Chromium).

@FreeTubeApp FreeTubeApp deleted a comment from github-actions bot Nov 13, 2023
@ThreeDeeJay
Copy link
Author

@absidue Then in that case, it seems the best way to get surround is adding an option to select a specific audio track that does support surround sound like AAC.
Because like I mentioned, FreeTube can already output surround sound, but only when setting Default Video Format to Audio Formats (just tested this video which I assume uses AAC in 0.19.1 beta), but then that plays audio only with a still thumbnail instead of video.

@kommunarr kommunarr added the player-related Player-related changes are currently on hold due to the ongoing player migration. label Apr 17, 2024
@JacobDrivers
Copy link

I am new to FreeTube (And I love it!). — Although, I am in the same boat where I have 5.1 surround sound, and would love for it to work by default. Is there anyway this can be done? Has anything new came out?

@ThreeDeeJay
Copy link
Author

@JacobDrivers Now it's in an even worse state than it was when I initially reported this issue
Like, it used to work when setting Settings > Default Video Format > Audio Formats and selecting one of these options, but now they don't even show up anymore
IMAGE

So it's better to just use PotPlayer: https://www.youtube.com/watch?v=AMv8yrd3VUM

@absidue
Copy link
Member

absidue commented Aug 21, 2024

@ThreeDeeJay Yes, losing the Opus streams was a side-effect of switching from pretending to be the YouTube website to pretending to be the YouTube iOS app, although YouTube seems to be experimenting with adding them to the iOS app, so they will randomly show up. But if we hadn't switched then you would have been confronted with constant errors and nothing playing. We might be able to switch back to pretending to being the website again soon, as people have figured out a way around YouTube's current hurdle.

@ThreeDeeJay
Copy link
Author

@absidue Opus is only stereo (or 4 channels for ambisonics in VR videos), so it's not what we really need.
And given iOS has spatial audio, I wouldn't be surprised if they enabled surround tracks to virtualize them.

@absidue
Copy link
Member

absidue commented Aug 21, 2024

The streams you get in FreeTube are the same as what the current top of the line iPhone gets in the official app.

@ThreeDeeJay
Copy link
Author

ThreeDeeJay commented Aug 21, 2024

what a missed opportunity for a practical use of Apple spatial audio smh
Kinda like how even if we buy Dolby Atmos on the MS store, we still can't decode TrueHD Atmos included in Blu-ray movies 🤦
But alas, just audio getting neglected so business as usual 😔

@JacobDrivers
Copy link

JacobDrivers commented Aug 21, 2024

@JacobDrivers Now it's in an even worse state than it was when I initially reported this issue Like, it used to work when setting Settings > Default Video Format > Audio Formats and selecting one of these options, but now they don't even show up anymore IMAGE

So it's better to just use PotPlayer: https://www.youtube.com/watch?v=AMv8yrd3VUM

I assume that even if this worked, it wouldn’t help make the video or what you’re watching support 5.1 as well, right? It seems like what you were talking about focuses on audio only (and possibly integrating that audio into video, which is the ultimate goal). That’s fine if it’s music or something similar, but I’m not asking for full 5.1 support if the track doesn’t have it. I’m more interested in the upmix (like Speaker Fill) where 2-channel audio can be played through all speakers IF 5.1+ isn't naturally detected on the video/audio.

Firefox handles this natively it seems, but with Chrome and Edge, I (and others) have had to use the --try-supported-channel-layouts flag. I’m not sure if that’s still required, but I continue to use it, and it works when I occasionally use those browsers. Could this be implemented/used in FreeTube somehow? (I see it is built somewhat on Chromium)

I’m not sure how difficult it would be to program this into FreeTube—to upmix if 5.1 (or more) is detected, handle channel mapping, or use an appropriate audio library (which FreeTube might already be using), or implement an upmixing algorithm. If it’s possible, it would be amazing. I’m sure many people use multichannel systems on their computers, and this would greatly enhance the experience.

EDIT: I just tried the --try-supported-channel-layouts on FreeTube, and it works! At-least, it works for up mixing. All my speakers are used when playing a video. When testing with the Dolby Surround Test, it still sees that video as 2 channel I guess and up mixes it a bit (But it works to some degree). Obviously, it isn't “true” 5.1+ surround, but I will take it — My speakers are being used.
image

Here is what my sound configuration looks like, with “Speaker fill” enabled.
image

@JacobDrivers
Copy link

JacobDrivers commented Aug 21, 2024

I wanted to add a quick follow-up to simplify my previous comment. I hope this helps others.

How to Fix Surround Sound Not Utilizing All Channels When Upmix/Speaker Fill is Enabled (Windows)

To address this, you can add the --try-supported-channel-layouts flag to FreeTube:

  1. Locate the FreeTube Shortcut: Right-click on the shortcut you use to launch FreeTube and select “Properties.”
  2. Add the Flag: In the “Target” field, add --try-supported-channel-layouts at the end of the existing path (see the screenshot from my previous comment).
  3. Apply and Save: Click “Apply” and then “OK.”

What This Flag Does: The --try-supported-channel-layouts flag forces the application to try different audio channel layouts supported by your system. This helps ensure that audio is routed through all your speakers, particularly useful for upmixing 2-channel audio in a 5.1/7.1/etc. setup. It is off by default in Chromium applications for performance, compatibility, stability, reasons (5.1+ takes more resources to output) and it is a niche thing I guess… It has worked fine for me for the years I have used it.

Note: This flag won't interfere with natural 5.1/7.1/etc. audio. It simply helps upmix 2-channel audio to use all your speakers when needed, while 5.1/7.1/etc. content will still play correctly through the appropriate channels.

Extra

If you want this fix to apply to all shortcuts (Such as it being pinned on the Windows taskbar) or need more detailed instructions, just Google/AI “adding a command-line argument”, “add command-line argument to Windows shortcut”, and “--try-supported-channel-layouts” for more information.

Alternatively, you can create a simple batch file (wrapper) in the FreeTube directory that includes the flag, and then point all shortcuts to this wrapper instead of the original FreeTube.exe. This ensures the flag is always used, regardless of how you launch the program. (Google or use AI for more information)

If you're comfortable with code and want to make this change permanent in FreeTube, you can add app.commandLine.appendSwitch('try-supported-channel-layouts'); to the main.js or index.js file in the source code, then compile it yourself.

Note: If you’re using Linux or another OS, the process is slightly different.

@ThreeDeeJay
Copy link
Author

I’m more interested in the upmix (like Speaker Fill) where 2-channel audio can be played through all speakers IF 5.1+ isn't naturally detected on the video/audio.

@JacobDrivers You might wanna try installing EqualizerAPO and HeSuvi, which has a decent stereo upmixer instead of a repeater.
Though if you're using speakers, you'll have to open Configuration Editor and add a new include and locate "C:\Program Files\EqualizerAPO\config\HeSuVi\matrix.txt". Note that using outside of the option in HeSuVi (headphones only) will attempt upmixing content that's already surround so you'd have to manually toggle it.

@JacobDrivers
Copy link

JacobDrivers commented Aug 22, 2024

I’m more interested in the upmix (like Speaker Fill) where 2-channel audio can be played through all speakers IF 5.1+ isn't naturally detected on the video/audio.

@JacobDrivers You might wanna try installing EqualizerAPO and HeSuvi, which has a decent stereo upmixer instead of a repeater. Though if you're using speakers, you'll have to open Configuration Editor and add a new include and locate "C:\Program Files\EqualizerAPO\config\HeSuVi\matrix.txt". Note that using outside of the option in HeSuVi (headphones only) will attempt upmixing content that's already surround so you'd have to manually toggle it.

Thank you for the suggestion. I might check them out (Install and try). I did get into the modded Realtek/any chip audio drivers and here (The person who made these just retired the project. It still works. The GitHub page is down. I'm sure there will be forks) everything there for a bit (I have full 5.1 surround sound speakers). With so many programs to manage—Dolby Digital Encoder, DTS Interactive Encoder, Creative Sound Blaster Connect, Nahimic, Dolby Atmos, DTS, Sonic Studio 3, and support for Sonic Radar 3—I found myself tinkering more than actually enjoying the setup, as most tinkerers tend to do. It did sound better, though.

The method I suggested above upmixes OK enough by default. Now it sounds great coming out of my 2 front, 1 center — and the 2 back channels (Subwoofer sounds good/better too even when upmixed)… The back speakers may not sound the best (It tries to mimic 5.1 and can be grainy), but, it is upmixed… At least they are being utilized. I don't have to hit the “matrix button” on my sound system now for it to mimic 5.1, it just naturally works on Windows even if 5.1 isn't detected. Spotify works fine, Firefox, etc. I have just had issues with Chromium apps. My front left, right, and center + bass all sound better now. More clarity. I see no point in having anything above 2 channels and not using them. I am still using the old analog 5.1 (Support for up to 7.1-Channel High Definition Audio).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request player-related Player-related changes are currently on hold due to the ongoing player migration.
Projects
Status: To assign
Development

No branches or pull requests

7 participants