[STOP OPENING DUPLICATES] Support multiple audio tracks

[selurvedu]

Checklist

• I checked, but didn't find any duplicates (open OR closed) of this issue in the repo.
• I have read the contribution guidelines given at https://github.com/TeamNewPipe/NewPipe/blob/HEAD/.github/CONTRIBUTING.md.
• This issue contains only one feature request. I will open one issue for every feature I want to request.

Describe the feature you want

I noticed that some videos on this channel are played with non-English audio track: https://www.youtube.com/channel/UCBSs9x2KzSLhyyA9IKyt4YA (look for the "Multi-Audio Clip" ones)

The same video is played in English on mobile YouTube website. Changing the UI language (Settings -> Account -> Language) also changes the audio track language.

Is your feature request related to a problem? Please describe it

Yes. Some videos get played in a language I don't understand, while mobile YouTube web plays them in English by default (at least on my phone, tested on Firefox Klar/Focus with cookies cleared).

Additional context

See the channel link above.

How will you/everyone benefit from this feature?

This feature will let our users select which language they prefer and watch a video with corresponding audio track. It will also fix the problem I described above.

Mod edit for search keywords: different other

[AudricV]

Related: #1059

[ChunkyProgrammer]

I think this would require an extractor change as well.
Here are some example videos that I was able to find:
Would You Sit In Snakes For $10,000? by Mr Beast
Assasin's Creed Valhalla: Story Trailer by Ubisoft North America

The setting is found here on the youtube desktop site:

[vinc3nz]

Any updates to this? Watching MrBeast in Spanish is really annoying and I have to use YouTube app for such videos.

[AudricV]

Yes, an initial extractor support has been added. Support of getting whether an audio is descriptive needs to be added in the extractor, and the corresponding app part needs to be done.

[QL568A6]

Does this code from the LibreTube repo help?

private fun getAudioStreamGroups(audioStreams: List<PipedStream>?): Map<String?, List<PipedStream>> {
        return audioStreams.orEmpty()
            .groupBy { it.audioTrackName }
    }

    override fun onAudioStreamClicked() {
        val audioGroups = getAudioStreamGroups(streams.audioStreams)
        val audioLanguages = audioGroups.map { it.key ?: getString(R.string.default_audio_track) }

        BaseBottomSheet()
            .setSimpleItems(audioLanguages) { index ->
                val audioStreams = audioGroups.values.elementAt(index)
                val lang = audioStreams.firstOrNull()?.audioTrackId?.substring(0, 2)
                trackSelector.updateParameters {
                    setPreferredAudioLanguage(lang)
                }
            }
            .show(childFragmentManager)
    }

https://github.com/libre-tube/LibreTube/blob/master/app/src/main/java/com/github/libretube/ui/fragments/PlayerFragment.kt

[MoralCode]

by my crude understanding, it seems like there may be some features missing in the underlying extractor that provides the raw data (such as streams.audioStreams in your example code) that would be needed to support this. I took a brief look at some of the linked issues and it seems partly there tho, the bulk of the remaining work seems to be UI/UX changes.

Your code does raise an interesting question though, it looks like that code is using string manipulation to detect the language. Where do the names of those tracks come from? are they defined by the uploader? or does youtube set them to something thats (mostly) consistent across the majority of youtube videos on the platform?

[AudricV]

Does this code from the LibreTube repo help?

No, not at all, as LibreTube uses a different way to get audio language tags that we will be able to use, thanks to my changes and data is different between LibreTube which uses Piped's API which uses the extractor one and NewPipe which uses directly the extractor API.

Your code does raise an interesting question though, it looks like that code is using string manipulation to detect the language. Where do the names of those tracks come from? are they defined by the uploader? or does youtube set them to something thats (mostly) consistent across the majority of youtube videos on the platform?

That's what I said in my previous comment, we have to parse an audio track ID to get an audio language code (we could also use the stream URL). All of this is defined by YouTube.

To keep you updated, TeamNewPipe/NewPipeExtractor#1026, which contains the changes I talked above, has been opened and merged. There is only one mistake that I made on the DASH manifest role attribute for descriptive audios I need to fix and then the app implementation could start.

A similar implementation to the one of the subtitles could be the way to implement the audio switch UI, but I am not sure, as services such as MediaCCC use progressive media and ExoPlayer would probably have no information about what language an audio track would have.

For audio selection, the audio selection needs to be changed to return probably a list of audio streams with different languages and a specific quality (in ListHelper).

An external audio player stream dialog needs also to be implemented, which would allow language selection and probably quality selection like the video dialog one.

Note that @Theta-Dev showed on the NewPipe subreddit a preview of this feature, so they may share their implementation insights and/or their ideas.

[MoralCode]

since this has been in demand for almost two years, would it be acceptable to implement the NewPipe frontend changes in 2 stages? i.e. stage 1 being a basic-but-imperfect implementation that just picks an audio track that matches the language setting device to reduce the number of disrupted viewing experiences, and stage 2 being the full UI implementation that lets the user pick their own language track if they want.

on one hand i see some benefit in doing this in two passes to deal with the problem for the majority of users, but also doing it this way seems like it might take a lot of the pressure off of this issue and could cause phase 2 to be de-prioritized in comparison to more important issues (i.e. "theres nothing more permanent than a temporary fix").

[Theta-Dev]

I have the audio language selector implemented in #9937. You can download the test build and try it out yourself.