Decouple SimpleRetriever and HttpStream classes #14012
Labels
CDK
Connector Development Kit
CDKv2
team/extensibility
technical-debt
issues to fix code smell
type/enhancement
New feature or request
Tell us about the problem you're trying to solve
The
SimpleRetriever
class extendsHttpStream
, which allows us to easily reuse a lot of the non-trivial mechanics related to sending HTTP requests, pagination, and error extraction. This however comes at the cost of losing flexibility, both in terms of control flow, and in terms of how we can send requests as HttpStream does not support other protocols.Describe the solution you’d like
I'd like to decouple the 2 classes, and move the Http specific logic to the HttpRequester so the retriever can focus exclusively on orchestrating the requester, stream_slicer, and extractor.
Note that low code connectors will still need to use the
HttpAvailabilityStrategy
by default. That should probably be set in the HttpRequester's config.Describe the alternative you’ve considered or used
A clear and concise description of any alternative solutions or features you've considered or are using today.
Additional context
Add any other context or screenshots about the feature request here.
Are you willing to submit a PR?
Remove this with your answer :-)
The text was updated successfully, but these errors were encountered: