Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ParsingException: Could not get ytInitialData #5977

Closed
Mattrazol opened this issue Apr 1, 2021 · 33 comments
Closed

ParsingException: Could not get ytInitialData #5977

Mattrazol opened this issue Apr 1, 2021 · 33 comments
Labels
ASAP Issue needs to be fixed as soon as possible bug Issue is related to a bug waiting for author If the author doesn't respond, the issue will be auto-closed. Otherwise the label will be removed. youtube Service, https://www.youtube.com/

Comments

@Mattrazol
Copy link

Mattrazol commented Apr 1, 2021

Exception

  • User Action: requested channel
  • Request: Start loading: http://www.youtube.com/c/TheMoonCarl
  • Content Country: US
  • Content Language: en
  • App Language: en
  • Service: YouTube
  • Version: 0.21.0
  • OS: Linux Android 11 - 30
Crash log

org.schabi.newpipe.extractor.exceptions.ParsingException: Could not get ytInitialData
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getInitialData(YoutubeParsingHelper.java:270)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.extractClientVersionAndKey(YoutubeParsingHelper.java:288)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getKey(YoutubeParsingHelper.java:366)
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeChannelExtractor.getNextPageFrom(YoutubeChannelExtractor.java:303)
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeChannelExtractor.getInitialPage(YoutubeChannelExtractor.java:251)
	at org.schabi.newpipe.extractor.utils.ExtractorHelper.getItemsPageOrLogError(ExtractorHelper.java:19)
	at org.schabi.newpipe.extractor.channel.ChannelInfo.getInfo(ChannelInfo.java:83)
	at org.schabi.newpipe.extractor.channel.ChannelInfo.getInfo(ChannelInfo.java:48)
	at org.schabi.newpipe.util.ExtractorHelper.lambda$getChannelInfo$4(ExtractorHelper.java:123)
	at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$BOLWstv98dC8pFAG_uir5gPXYwY.call(Unknown Source:4)
	at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
	at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15753)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
	at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:923)
Caused by: org.schabi.newpipe.extractor.utils.Parser$RegexException: failed to find pattern "var\s*ytInitialData\s*=\s*(\{.*?\});"
	at org.schabi.newpipe.extractor.utils.Parser.matchGroup(Parser.java:74)
	at org.schabi.newpipe.extractor.utils.Parser.matchGroup(Parser.java:63)
	at org.schabi.newpipe.extractor.utils.Parser.matchGroup1(Parser.java:54)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getInitialData(YoutubeParsingHelper.java:266)
	... 33 more


I'm also having problem fetching my Subscriptions 76/86 don't get updated.

@bauspanderu
Copy link

I'm having the same issue, I think it's because of a new cookie site that opens before the actual channel.

@TobiGr TobiGr pinned this issue Apr 1, 2021
@Neeqstock
Copy link

Same issue: Could not get ytInitialData

@TobiGr
Copy link
Contributor

TobiGr commented Apr 1, 2021

We are on it.
See TeamNewPipe/NewPipeExtractor#592

@TobiGr
Copy link
Contributor

TobiGr commented Apr 1, 2021

Can you please test this debug version and tell me whether the problem is solved?
app-debug.zip

@TobiGr TobiGr added ASAP Issue needs to be fixed as soon as possible bug Issue is related to a bug youtube Service, https://www.youtube.com/ labels Apr 1, 2021
@Mattrazol
Copy link
Author

@TobiGr Yes it seems to work now.

@noahjutz
Copy link

noahjutz commented Apr 1, 2021

Can you please test this debug version and tell me whether the problem is solved?
app-debug.zip

Works for me, too.

@Neeqstock
Copy link

Neeqstock commented Apr 1, 2021

Can you please test this debug version and tell me whether the problem is solved?
app-debug.zip

This test version works for me also (using it from Italy, Europe).

As a temporary workaround on the main version (0.21.0), since the issue affects only Europe, if I connect through a VPN to a server outside Europe, the app works.

@TobiGr
Copy link
Contributor

TobiGr commented Apr 2, 2021

Here is another debug APK. This time with no consent given (PENDING+999). Please test again.
Note: No cookie set, using the CONSENT header....
app-debug.zip

@Mattrazol
Copy link
Author

@TobiGr yes, this last one seems to work as well for me.

@Gaffen
Copy link

Gaffen commented Apr 2, 2021

I tried a version earlier today and it worked, then I uninstalled it. I reinstalled just now and got prompted to do the cookie thing. I completed it successfully and returned to newpipe but search wouldn't work; I get the ytinitialdata error again.

@darrenmothersele
Copy link

#5977 (comment)

Works for me. Thanks.

@Gaffen
Copy link

Gaffen commented Apr 3, 2021

I tried a version earlier today and it worked, then I uninstalled it. I reinstalled just now and got prompted to do the cookie thing. I completed it successfully and returned to newpipe but search wouldn't work;

My bad; I uninstalled uninstalled and reinstalled the most recently linked debug version; I now appear to be able to use the search facility.

However, when I import my old settings it no longer works

@deltragon
Copy link

deltragon commented Apr 4, 2021

However, when I import my old settings it no longer works

Same problem here, however it only affects the settings - if I clear the data, and import only subscriptions, playlist and history, but decline settings, it keeps working.

@Mattrazol
Copy link
Author

After the latest update it got worse. Before if I did the challenge it would work for a few videos, but now it doesn't work at all. It keeps asking to solve the challenge over and over without reproducing any videos whatsoever.

@Neeqstock
Copy link

Same here (internet access from Italy).
Version 0.21.1 gives the same issue as before. Cannot search for any video, although it can reproduce videos from links.

Strangely enough, the Debug Version posted by TobiGr (0.21.0 Debug, app-debug.zip) still works perfectly without giving the ytInitialData issue.

@darrenmothersele
Copy link

I can confirm that search isn't working with version 0.21.1
Error: org.schabi.newpipe.extractor.exceptions.ParsingException: Could not get ytInitialData

But version 0.21.0 Debug from above is working fine.

@straves16

This comment has been minimized.

@12people
Copy link

Now I'm having the same problem again, but the option to clear reCaptcha no longer appears in settings.

I use an EU VPN and turning off the VPN fixes the problem.

@AudricV
Copy link
Member

AudricV commented May 26, 2021

@12people Did you try the RC of the 0.21.3 version or my APK using the link I mentioned above? Can you reproduce the bug with these two versions?

@martinambrus
Copy link

Latest debug version from the linked build page works for me in Europe - Slovakia.

@Ignacy-s
Copy link

Ignacy-s commented Jun 18, 2021

Now I'm having the same problem again, but the option to clear reCaptcha no longer appears in settings.

I use an EU VPN and turning off the VPN fixes the problem.

This is a working workaround for me! Thanks much! (I connected from EU and therefore search didn't work, vids started normally).

WORKAROUND:
VPN trough non-EU state (tested with USA and Nord).
Search works now.

@AudricV AudricV mentioned this issue Jun 28, 2021
4 tasks
@AudricV
Copy link
Member

AudricV commented Jul 6, 2021

Can you try the 3rd RC of the 0.21.6 version (see the pinned issue) and let us know if it works? Thank you in advance.

@Ignacy-s
Copy link

Ignacy-s commented Jul 6, 2021

Can you try the 3rd RC of the 0.21.6 version (see the pinned issue) and let us know if it works? Thank you in advance.

Tested and works from Poland, both mobile network and family Wi-Fi from local provider.

However, if client is force-closed from app-info, and then VPN is turned on, networking won't work. But if NewPipe was run first without VPN, and then after connecting successfully, trough VPN, it works.

Error log when resetting the app and trying to connect trough VPN, even suggested videos don't work (which worked before when search didn't work). Tried using NordVPN and countries both in EU and North America.

Exception

  • User Action: searched
  • Request: grobix
  • Content Country: US
  • Content Language: en-US
  • App Language: en_US
  • Service: YouTube
  • Version: 0.21.6
  • OS: Linux Android 11 - 30
Crash log

java.net.UnknownHostException: Unable to resolve host "www.youtube.com": No address associated with hostname
	at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:156)
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Inet6AddressImpl.java:103)
	at java.net.InetAddress.getAllByName(InetAddress.java:1152)
	at okhttp3.Dns$1.lookup(Dns.java:40)
	at okhttp3.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:185)
	at okhttp3.internal.connection.RouteSelector.nextProxy(RouteSelector.java:149)
	at okhttp3.internal.connection.RouteSelector.next(RouteSelector.java:84)
	at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:215)
	at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
	at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
	at okhttp3.RealCall.execute(RealCall.java:93)
	at org.schabi.newpipe.DownloaderImpl.execute(DownloaderImpl.java:264)
	at org.schabi.newpipe.extractor.downloader.Downloader.get(Downloader.java:70)
	at org.schabi.newpipe.extractor.downloader.Downloader.get(Downloader.java:54)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.isHardcodedClientVersionValid(YoutubeParsingHelper.java:295)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getClientVersion(YoutubeParsingHelper.java:368)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.addClientInfoHeaders(YoutubeParsingHelper.java:719)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.addYouTubeHeaders(YoutubeParsingHelper.java:705)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getJsonResponse(YoutubeParsingHelper.java:668)
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeSearchExtractor.onFetchPage(YoutubeSearchExtractor.java:60)
	at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:54)
	at org.schabi.newpipe.extractor.search.SearchInfo.getInfo(SearchInfo.java:29)
	at org.schabi.newpipe.util.ExtractorHelper.lambda$searchFor$0(ExtractorHelper.java:81)
	at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$qyxpuXgomWa-cbONQns-pd7zxm0.call(Unknown Source:8)
	at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
	at java.lang.Thread.run(Thread.java:923)
Caused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
	at libcore.io.Linux.android_getaddrinfo(Native Method)
	at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:73)
	at libcore.io.BlockGuardOs.android_getaddrinfo(BlockGuardOs.java:202)
	at libcore.io.ForwardingOs.android_getaddrinfo(ForwardingOs.java:73)
	at java.net.Inet6AddressImpl.lookupHostByName(Inet6AddressImpl.java:135)
	... 46 more


@martinambrus
Copy link

martinambrus commented Jul 6, 2021

all features of the 3rd RC version of 0.21.6 work for me im Europe - Slovakia (cellular operator, 4G data as well as 3G data roamed through another EU operator), no VPN

@hp8wvvvgnj6asjm7

This comment has been minimized.

@Outlet2048
Copy link

Having the same issue
2 of my subscriptions cannot be loaded

@Stypox
Copy link
Member

Stypox commented Aug 4, 2021

Is it fixed in 0.21.8? Maybe switching to youtubei solved something, I don't know.

@AudricV AudricV added the waiting for author If the author doesn't respond, the issue will be auto-closed. Otherwise the label will be removed. label Sep 26, 2021
@opusforlife2 opusforlife2 unpinned this issue Sep 26, 2021
@github-actions
Copy link

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ASAP Issue needs to be fixed as soon as possible bug Issue is related to a bug waiting for author If the author doesn't respond, the issue will be auto-closed. Otherwise the label will be removed. youtube Service, https://www.youtube.com/
Projects
None yet
Development

No branches or pull requests