From 8cb13c8d302a850a0e03c4a2e5fa747c14b7e01e Mon Sep 17 00:00:00 2001 From: Sultan Iman <354868+sultaniman@users.noreply.github.com> Date: Mon, 13 May 2024 19:22:52 +0200 Subject: [PATCH] Add notes about pagination authentication methods --- docs/website/docs/walkthroughs/create-a-pipeline.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/website/docs/walkthroughs/create-a-pipeline.md b/docs/website/docs/walkthroughs/create-a-pipeline.md index eeb4e465e7..0695f04bdc 100644 --- a/docs/website/docs/walkthroughs/create-a-pipeline.md +++ b/docs/website/docs/walkthroughs/create-a-pipeline.md @@ -103,6 +103,11 @@ def githubapi_resource(api_secret_key: str = dlt.secrets.value): yield page ``` +Since we're using a personal access token, it's necessary to use `BearerTokenAuth` for passing the correct authentication details. +GitHub returns the next page link through the header parameter `link`. +Consequently, our `HeaderLinkPaginator` pagination strategy enables the rest client to seamlessly navigate through the pages and retrieve data. +For additional details, consult https://docs.github.com/en/rest/using-the-rest-api/using-pagination-in-the-rest-api?apiVersion=2022-11-28 + ## 4. Load the data Uncomment the commented out code in `main` function in `githubapi.py`, so that running the