Skip to content

Commit

Permalink
Use the youtubei API for YouTube searches + update mocks + ignore a test
Browse files Browse the repository at this point in the history
Add getSearchParameter, a new method in YoutubeSearchQueryHandlerFactory class which returns the params field for a search, or an empty string if there is no one.
Update mocks of YoutubeSearchExtractorTest.
Ignore StreamSegmentsTestOstCollection test of YoutubeStreamExtractorDefaultTest because the video is now only available in China.
  • Loading branch information
AudricV committed Apr 11, 2021
1 parent 918512b commit 0e69831
Show file tree
Hide file tree
Showing 50 changed files with 4,422 additions and 4,746 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import java.util.List;
import java.util.Map;

import static org.schabi.newpipe.extractor.services.youtube.linkHandler.YoutubeSearchQueryHandlerFactory.getSearchParameter;
import static org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.*;
import static org.schabi.newpipe.extractor.utils.Utils.UTF_8;
import static org.schabi.newpipe.extractor.utils.Utils.isNullOrEmpty;
Expand Down Expand Up @@ -55,11 +56,33 @@ public YoutubeSearchExtractor(final StreamingService service, final SearchQueryH

@Override
public void onFetchPage(@Nonnull final Downloader downloader) throws IOException, ExtractionException {
final String url = getUrl() + "&pbj=1";
final String query = super.getSearchString();

// Get the search parameter of the request
final List<String> contentFilters = super.getLinkHandler().getContentFilters();
final String params;
if (!isNullOrEmpty(contentFilters)) {
final String searchType = contentFilters.get(0);
params = getSearchParameter(searchType);
} else {
params = "";
}

final JsonArray ajaxJson = getJsonResponse(url, getExtractorLocalization());
final byte[] body;
if (!isNullOrEmpty(params)) {
body = JsonWriter.string(prepareJsonBuilder()
.value("query", query)
.value("params", params)
.done())
.getBytes(UTF_8);
} else {
body = JsonWriter.string(prepareJsonBuilder()
.value("query", query)
.done())
.getBytes(UTF_8);
}

initialData = ajaxJson.getObject(1).getObject("response");
initialData = getJsonPostResponse("search", body, getExtractorLocalization());
}

@Nonnull
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import java.util.List;

import static org.schabi.newpipe.extractor.utils.Utils.UTF_8;
import static org.schabi.newpipe.extractor.utils.Utils.isNullOrEmpty;

public class YoutubeSearchQueryHandlerFactory extends SearchQueryHandlerFactory {

Expand Down Expand Up @@ -72,4 +73,27 @@ public String[] getAvailableContentFilter() {
// MUSIC_ARTISTS
};
}

public static String getSearchParameter(final String contentFilter) {
if (!isNullOrEmpty(contentFilter)) {
switch (contentFilter) {
case VIDEOS:
return "EgIQAQ%3D%3D";
case CHANNELS:
return "EgIQAg%3D%3D";
case PLAYLISTS:
return "EgIQAw%3D%3D";
case ALL:
case MUSIC_SONGS:
case MUSIC_VIDEOS:
case MUSIC_ALBUMS:
case MUSIC_PLAYLISTS:
case MUSIC_ARTISTS:
default:
return "";
}
} else {
return "";
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@ public static void setUp() throws Exception {
// @formatter:on
}

@Ignore("Test broken, video is only available in China")
public static class StreamSegmentsTestOstCollection extends DefaultStreamExtractorTest {
// StreamSegment example with single macro-makers panel
private static final String ID = "2RYrHwnLHw0";
Expand Down

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Loading

0 comments on commit 0e69831

Please sign in to comment.