Prefect-meemoo Index / Prefect Meemoo / Triplydb / Pagination
Auto-generated documentation for prefect_meemoo.triplydb.pagination module.
Show source in pagination.py:108
Common logic for the run_saved_query and run_sparql functions
def _run_query(send_request_fn: Callable[[int], Response]) -> Iterable: ...Show source in pagination.py:202
def add_params_to_uri(uri: str, params: dict[str, Any]) -> str: ...Show source in pagination.py:192
def get_triply_headers(
bearer_token: str, extra_headers: Union[dict[str, str], None] = None
) -> dict[str, str]: ...Show source in pagination.py:144
Send a GET request including the triply token to the given endpoint.
@task
def request_triply_get(endpoint: str, triplydb_block_name: str) -> Response: ...Show source in pagination.py:165
Send a POST request including the triply token to the given endpoint.
@task()
def request_triply_post(
endpoint: str, body: str, triplydb_block_name: str
) -> Response: ...Show source in pagination.py:15
Execute a saved query.
Unlike a simple GET request to a saved query endpoint, this function takes care of pagination. Requires prefect. Results are yielded back as an Iterable. This means that only part of the results are kept in memory at any given time.
results = run_saved_query(...)
for r in results:
...One caveat with the offset parameter is that all results up to #offset + limit are fetched from the triple store,
including the first #offset results which are discarded afterwards.
Use a negative limit to fetch all results.
@flow(name="Run a Triply saved query with pagination")
def run_saved_query(
saved_query_uri: str, triplydb_block_name: str, limit: int = 10000, offset: int = 0
) -> Iterable: ...Show source in pagination.py:56
Execute a sparql SELECT query using the given endpoint.
Unlike a simple POST request to a tripple store, this function takes care of pagination. Requires prefect. Results are yielded back as an Iterable. This means that only part of the results are kept in memory at any given time.
results = run_sparql_select(...)
for r in results:
...One caveat with the offset parameter is that all results up to #offset + limit are fetched from the triple store,
including the first #offset results which are discarded afterwards.
Use a negative limit to fetch all results.
@flow(name="Run a sparql SELECT with pagination")
def run_sparql_select(
endpoint: str,
sparql: str,
triplydb_block_name: str,
limit: int = 10000,
offset: int = 0,
) -> Iterable: ...