No audio in 720p60 and 1080p (or higher) streams for external players (VLC & MX)

[doodhout]

• I carefully read the contribution guidelines and agree to them.
• I checked if the issue/feature exists in the latest version.

Preface:

I suppose this PR is included in the 0.9.3 release from a few days ago, because I only noticed the new streams being available in that version.

Context:

I use VLC for Android and MX Player as an external player when watching videos through NewPipe.

The issue:

When watching 720p60, 1080p or 1080p60 (and higher of course) (in one of the mentioned external players), then there is no sound to be heard. Inspecting the properties of the mux/container in MX Player indicates the absence of an audio stream in the mux.

Can it be that the sound is only successfully muxed when using the (new) integrated player?

[theScrabi]

Right, we should disable the option for 720p60 and 1080p, etc. To be watched in an external player sice the external player woulf not be able to mix video only and audio only streams together.

[KKKloss]

this is not a problem that occurs only watching videos with external players.
when i try to download a video the 480p and 1080p versions don't have audio, on the contrary 360p and 720p (not 60fs) are working well.
so 480p and 1080p are only working if i use internal player.
disable external player isn't the right choice to make, since even the download feature is broken.

thanks for this app

[theScrabi]

Confirmed.
@mauriciocolli we need to fix this ASAP.

Ok now the details (for anyone else wanting to fix it): Here and here is the code we talk about.

NewPipe nows 3 Types of streams:

• Video Stream with audio.
• Video Stream without audio.
• Audio only Streams

Since 0.9.3 we mix together non audio and audio only streams. The combined once somehow get ignored. But they should not when it comes in terms of downloading, external player, old internal player. So we need code that can select the steams based on the action.

[mauriciocolli]

@theScrabi I think you misunderstood this part, we never really mux the streams, the ExoPlayer just play both the Video Only and the Audio streams together, without muxing them, just giving the "illusion" that they are just one.

I found a library that allows us to mux at least the MP4+M4A, and it maybe get implemented, it just needs more testing (it's the mp4parser to anyone interested).

Android has it's own media library/muxer, although I don't have the necessary knowledge to implement it, I did tried some examples that I found and it went well, bad thing that's for API 18+.

I even considered adding ffmpeg to this project, but its size is over 20MB just for one cpu arch...

---

For now I'm thinking in adding some warning to the user like this (a rough idea):

External player enabled	Download Dialog

• If External Player is enabled and stream is video only, show the w/o sound icon
• When the user press download, let him choose the streams and show the w/o sound icon for video only streams too

If you have another idea of what to do for now, tell me here

[utack]

Sorry it ask here: What kind of audio can we select currently
Does audio choice webm now mean opus with the new player, or still ogg?

[theScrabi]

@mauriciocolli I know, thats what I meant. I just didn't know the correct words and explanation. Also I think your solution is quiet good :)

About ffmpeg, in another thread I said that we should not put it into NewPipe. If we ever have to add it we should put ffmpeg into another app, and than make NewPipe work with that app.

[doodhout]

@utack: it's either Ogg Vorbis or Ogg Opus, but Ogg by itself is like MP4 (e.g. containing AVC/H264 and HE-AAC): not the actual codec, but a kind of container.

[theScrabi]

Yea I thought webm audio was vorbis in a webm container.

[mauriciocolli]

Here is a quick comparison for 2 audios' codecs downloaded with NewPipe (bitrate of 128kpbs):

	M4A	WebM
File size	2.75 MiB	2.46 MiB
Duration	3 min 1 s	3 min 1 s
Format	AAC	Vorbis
Codec ID	dash (iso6/mp41)	A_VORBIS
Bit rate	126 kb/s	128 kb/s
Sampling rate	44.1 kHz	44.1 kHz
Stream size	2.72 MiB (99%)	2.77 MiB

And I disabled the Opus codec from being extracted, as it was causing problems with old devices/APIs.
(https://github.com/TeamNewPipe/NewPipeExtractor/blob/master/services/youtube/YoutubeStreamExtractor.java#L153)

But I think I'll enable it again, just to be able to download these files as they have a 160kpbs bitrate, and I guess that's the highest bitrate that you can find in most of the videos (it does have higher, but only some videos have) without having to convert anything.

[epitron]

@mauriciocolli I like the solution of displaying a no-sound icon beside the formats when an external player is selected.

I'm not an Android developer, but does anyone know if it's technically possible to pass two streams to MX Player, if it added support for that? If so, maybe I could try begging the devs to add a feature.

Another possible fix: add a config option that says "use internal player when audio is in a separate stream"

[mauriciocolli]

@epitron that's a thing I've been searching for since I take a look at this issue.

I found that this feature is already listed in the high-priority list (nº 14):
https://forum.xda-developers.com/apps/mx-player/feature-request-external-audio-streams-t2784474

---

I'll take that idea in consideration when implementing the "final" solution.

And what do you think about putting a warning/summary in the option itself (which I already did)?
I believe that the summary and the no-sound icon would be pretty informative to the user (the summary would be something like "Some resolutions will NOT have audio when this option is enabled").

[KaKi87]

Would there be possible to make a player like mpv or VLC able to play separate video and audio simultaneously ?

[doodhout]

I know youtube-dl using ffmpeg to mux the video and audio stream together.

Could NewPipe technically do this, mux the streams on the fly? I don't know the limitations of ffmpeg though, i.e. is it possible to start muxing at any point of the streams instead of just the beginning, etc.

[fritzrehde]

I know youtube-dl using ffmpeg to mux the video and audio stream together.

Could NewPipe technically do this, mux the streams on the fly? I don't know the limitations of ffmpeg though, i.e. is it possible to start muxing at any point of the streams instead of just the beginning, etc.

Has this idea by @doodhout been given any thought? Without this video+audio the external video player is useless for most use-cases, so this feature would be amazing to have.

[fritzrehde]

I think this issue (or a similar/new one related to audio missing in some streams) should be open. As has been pointed out, it might technically be possible to implement/fix this in the future (see the yt-dlp reference others have made).

[opusforlife2]

You can open a new issue.