Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rest_api: allow any type in incremental cursor values #1757

Open
willi-mueller opened this issue Aug 28, 2024 · 0 comments
Open

rest_api: allow any type in incremental cursor values #1757

willi-mueller opened this issue Aug 28, 2024 · 0 comments
Assignees

Comments

@willi-mueller
Copy link
Collaborator

Feature description

Right now, we require the initial_value, end_value, and last_value_func in IncrementalArgs to be str.

Changing existing test cases to int also proves that this is supported. Probably datetime is supported too.

Are you a dlt user?

Yes, I'm already a dlt user.

Use case

No response

Proposed solution

The types would look as follows:

class IncrementalArgs(TypedDict, Generic[TCursorValue], total=False):
    cursor_path: str
    initial_value: Optional[TCursorValue]
    last_value_func: Optional[LastValueFunc[TCursorValue]]
    primary_key: Optional[TTableHintTemplate[TColumnNames]]
    end_value: Optional[TCursorValue]
    row_order: Optional[TSortOrder]
    allow_external_schedulers: Optional[bool]
class IncrementalRESTArgs(IncrementalArgs[Any], total=False):
    convert: Optional[Callable[..., Any]]

The validation for the RESTAPIConfig needs to be adapted to support generic types.

Related issues

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Todo
Development

No branches or pull requests

2 participants