New required parameter for ANDROID client

[unixfox]

It was discovered that the ANDROID client will require a new parameter in order to return correct streams URLs, see: TeamNewPipe/NewPipe#8713

Should we add that parameter in the results of the README: https://github.com/zerodytrash/YouTube-Internal-Clients#results?

[unixfox]

{
    "contentCheckOk": True,
    "context": {
        "client": {
            "androidSdkVersion": 31,
            "clientName": "ANDROID",
            "clientVersion": "17.31.35",
            "gl": "US",
            "hl": "en-GB",
            "osName": "Android",
            "osVersion": "12",
            "platform": "MOBILE"
        },
        "user": {
            "lockedSafetyMode": False
        }
    },
    "racyCheckOk": True,
    "videoId": video_id
}

You need to send androidSdkVersion now and recently the correct user agent: iv-org/invidious#3230 (comment)

[unixfox]

I meant to say that YouTube servers is A/B testing this new requirement. You may not have any issue on your IP address, but some do, see the first comments on this issue: iv-org/invidious#3230

[zerodytrash]

When I have time I will run the scripts again to update the version numbers as well.
Would definitely make sense to place the sometimes required parameter and a matching user-agent in the result table. I am surprised that Youtube is still in the A/B test phase after more than a month and not even half of the users are affected by the "experiment". Have tried to reproduce this from several IP locations. Without success.

[zerodytrash]

I just made this request with no problem

Youtube sometimes rolls out updates only for certain users or regions. This makes it difficult to reproduce such things. I believe that is the case here.

[unixfox]

When I have time I will run the scripts again to update the version numbers as well.
Would definitely make sense to place the sometimes required parameter and a matching user-agent in the result table. I am surprised that Youtube is still in the A/B test phase after more than a month and not even half of the users are affected by the "experiment". Have tried to reproduce this from several IP locations. Without success.

You can easily reproduce this issue on UK IPv4 public addresses, that's why I asked the country here: iv-org/invidious#3230 (comment). I've successfully reproduced the issue on digitalocean droplets hosted in UK.

Here is a script that try to reproduce the issue: https://gist.github.com/coletdjnz/cc0cbce42067b77c628d68c7b10a96c5/c495de26e470e20e7bf082f7d1e3716972106d8c

It's not the latest revision of the script because the latest revision test the unavailability when the user-agent header is incorrect. In this case, take the latest revision.

Anyway, you guys can believe or do anything you want, I'm just sharing the fact that this issue has been reported multiple times on popular projects:

• [FIXED IN 0.23.2] [YouTube] "The following content is not available on this app." message shown in video/5 minutes of silence instead of a sound TeamNewPipe/NewPipe#8713
• [Bug] Content not available in this app. upgrade to latest version of youtube. iv-org/invidious#3230
• "The following content is not available on this app." Tyrrrz/YoutubeExplode#647

And these popular projects have pushed multiple fixes for that, just an extract of them:

• [YouTube] Update clients version and improve extraction of API key and client version of WEB client TeamNewPipe/NewPipeExtractor#890
• For android client send sdk version to youtube iv-org/invidious#3255
• yt-dlp/yt-dlp@c7dcf0b

I'm not here to spread disinformation nor to be belittled by others in the community.

[unixfox]

You have to do multiple requests until the issue appears. Please use the script I sent in my previous message.

[zerodytrash]

Also, no one should run this script. As can be seen, its poorly written

For demonstration and testing purposes, the script is sufficient. I have no doubt that the androidSdkVersion param is sometimes required. The issue was only created to improve the documentation of the clients (thanks @unixfox). Any comment that doubts the existence of the problem or criticizes the test scripts is a waste of time lol

[zerodytrash]

it appears you are not looking at the suggestion critically.

It appears that you have not looked at the issue at all.
Your go script only checks the playability status. This is always "OK" even if the described problem occurs. Youtube sometimes returns a placeholder video which displays an error message to the user (for whatever reason). You need to check if the requested video ID matches the video ID from the response.

> go run .\youtube.go
&{City:London Region:England}
&{PlayabilityStatus:{Status:OK}}
&{PlayabilityStatus:{Status:OK}}

In the Python test script this video-id check is already present.

pr_video_id = data['videoDetails']['videoId']
if video_id != pr_video_id:
    # print(f"{t_name} Video id {video_id} does not match {pr_video_id}")

[coletdjnz]

Please use the script I sent in my previous message.

Also, no one should run this script. As can be seen, its poorly written:

https://gist.github.com/coletdjnz/cc0cbce42067b77c628d68c7b10a96c5/c495de26e470e20e7bf082f7d1e3716972106d8c#file-unavailabletest-py-L89-L107

no delay at all between requests. Thats a good way to get yourself IP blocked.

youtube doesn't care with innertube. thats the single threaded version anyways, which takes forever to reproduce.

[unixfox]

I feel like this discussion leads to nowhere.

What I propose in order to satisfy everyone, is to have a new column for each client with the recommended parameters to send to YouTube servers, like androidSdkVersion for the ANDROID client.

This way at least when someone will read the documentation, whenever they are doing extreme API load or not, they will be aware of these potential parameters that may or may not apply to their use case.

The ideal case would be to show an example request with all the recommended parameters for each client (see here: https://github.com/TeamNewPipe/NewPipeExtractor/blob/943b7c033bb9d07ead63ddab4441c287653e4384/extractor/src/main/java/org/schabi/newpipe/extractor/services/youtube/YoutubeParsingHelper.java#L1174), but that may be out of scope of this repository.

[zerodytrash]

• All Android clients now have a hint about the androidSdkVersion.
• There is now a sample request that contains the parameter.
• In a table are described beside the androidSdkVersion also a few other parameters.

Suggestions for improvement are welcome.

[zerodytrash]

well, Needs => Might require