Mid-level ArrowFlight
Client
#3371
Labels
arrow-flight
Changes to the arrow-flight crate
enhancement
Any new improvement worthy of a entry in the changelog
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I want to improve the FlightSQL client contributed by @avantgardnerio in #3207
Specifically FlightSQL is in terms of RecordBatches but at the moment to get record batches out out of a stream of
FlightData
s is non trivial (especially as the schema and dictionaries are sent individually)I think this is a common need as I found similar code in IOx for our own flight API as was in the FlightSQL client from #3207
@xudong963 also mentions there is similar code in DataBend: https://github.com/datafuselabs/databend/tree/main/src/query/service/src/api/rpc (uses the crate https://github.com/DataEngineeringLabs/arrow-format)
Describe the solution you'd like
I would like a mid-level flight client that handles these details - the FlightSQL client can then be built on top of this mid level client, and I think the mid level client is useful on its own (for writing clients for other Arrow Flight based APIs).
Describe alternatives you've considered
Additional context
We have a version of this code in https://github.com/influxdata/influxdb_iox/pull/6427
The text was updated successfully, but these errors were encountered: