Add all Itags

[ghost]

Currently the NewPipeExtractor doesn't handle all Itags. Some because "it's not well supported in older devices" (e.g. Opus). Some because we "Don't add VideoOnly streams that have normal variants". Some because we don't have added HDR support. Some others are just missing. IMHO we should just add all Itags, since NewPipeExtractor is just an extractor. We should let the frontend (e.g. NewPipe most of the time) handle choosing the stream from the available ones. Also the FPS is often incorrect (30 instead of 24).

[theScrabi]

Yes that sounds good.

[ghost]

@TheAssassin suggested to automatically parse youtube-dl's list

[TheAssassin]

... to benefit from their maintenance effort. I think this should be done during build, not during the runtime. The list won't be updated too often, I guess.

[theScrabi]

I'm with @@TheAssassin here. I don't know how big that list is but we could copy it. ... Mabe even with code.

[TheAssassin]

@theScrabi the idea is to have some script or so that generates Java source code from that list during builds. This way, the list is kept up to date.

The file can even be committed from time to time on changes. At linuxdeployqt, this was even necessary to allow for generating complete source bundles that can be built on systems like OBS, which don't allow network access during builds for various valid reasons.

[ghost]

@TheAssassin: The only thing is that youtube-dl's list doesn't contain all properties (like HDR), but somehow `youtube-dl -F` still lists what's HDR iirc, so I'm not sure how they're doing that. Op za 28 jul. 2018 19:55 schreef TheAssassin <[email protected]>:

…

@theScrabi <https://github.com/theScrabi> the idea is to have some script or so that generates Java source code from that list during builds. This way, the list is kept up to date. The file can even be committed from time to time on changes. At linuxdeployqt <https://github.com/probonopd/linuxdeployqt>, this was even necessary to allow for generating complete source bundles that can be built on systems like OBS <https://build.opensuse.org>, which don't allow network access during builds for various valid reasons. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#71 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJU76ertSPnH-D1Uky3GNTXaKtrx-zETks5uLKWLgaJpZM4SG9xY> .

[TheAssassin]

Well, this list might also be generated by calling such a command, and parsing the output.

[ghost]

@TheAssassin: The thing is that not every video has every format, so we probably still need to maintain the what-ids-are-hdr list. Op za 28 jul. 2018 20:01 schreef TheAssassin <[email protected]>:

…

Well, this list might also be generated by calling such a command, and parsing the output. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#71 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJU76ZFAB9-UjHIMjZ1nJlmIvrk8JR9Mks5uLKblgaJpZM4SG9xY> .

[TheAssassin]

@wb9688 you just said youtube-dl -F provides the correct information. So why not parse the output of that command? Just download the latest version of youtube-dl, call it like that, and generate the Java code.

[ghost]

@TheAssassin: No, it provides the formats (with all needed details iirc) available for a specific video, but not all videos have all formats. ;)

[theScrabi]

@TheAssassin: The thing is that not every video has every format, so we probably still need to maintain the what-ids-are-hdr list.

Thats the reason I would not do it full automatized, otherwhise we will end up with bugs and issues.

[theScrabi]

A good list of itags can be found here:
https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/youtube.py#L379

[theScrabi]

also we might want to speak about splitting codec (for audio and video) and container.

[TheAssassin]

@TheAssassin: The thing is that not every video has every format, so we probably still need to maintain the what-ids-are-hdr list.

Thats the reason I would not do it full automatized, otherwhise we will end up with bugs and issues.

A good list of itags can be found here:
https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/youtube.py#L379

You must be kidding. See #71 (comment).

I still haven't got what the problem with the HDR tags is. I was told two contradictory versions by @wb9688: either the source code doesn't provide it but youtube-dl -F does, or the other way round.

@wb9688 can you please tell if either version would work for NewPipe?

[TheAssassin]

@theScrabi as this is a semi-automatic process (the build tools generate the list, but some dev has to actually commit it, and I always read diffs before committing stuff), the chance for mistakes is quite small.

[ghost]

I still haven't got what the problem with the HDR tags is

@TheAssassin: We need to know which Itags are HDR, so we'll only play HDR videos on devices that support it, but the full list from youtube-dl doesn't contain the HDR ones (for example 337 is 2160p HDR), while youtube-dl -F does somehow (@theScrabi: do you know where they come from? DASH manifest?), which you can check with for example https://www.youtube.com/watch?v=WW2DKBGCvEs. (Btw we don't even have HDR support yet, though that shouldn't be a big change.)

[theScrabi]

(Btw we don't even have HDR support yet, though that shouldn't be a big change.

Its a good idea to make the backend ready for that.

I don't know where they got the itags from. I used to take them from a website, but this is the most complete list I could find so far.