You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'd like to process data that is coming from outside arrow using arrow compute kernels without copying the data.
The FFI api does not seem to support this use case at the moment. The only way I found to create an FFI_ArrowArray was with an already existing ArrayData struct. Creating ArrayData requires at least one Buffer, and creating a Buffer from foreign memory again needs an FFI_ArrowArray.
Describe the solution you'd like
The Buffer already supports two ways of deallocating memory.
Via arrows cache-aligned allocator
With a callback in an FFI_ArrowArray
The second option can be generalized so that the callback can be any type using a dyn trait.
Describe alternatives you've considered
An alternative could be to make it easier to create an FFI_ArrowArray from a single memory region, but that seems to be more complex and also less general.
Additional context
This functionality could also allow zero-copy conversion from arrow2.
The text was updated successfully, but these errors were encountered:
alamb
changed the title
Allow creating buffers from externally owned memory like Vec or String
Support creating arrays from externally owned memory like Vec or StringApr 15, 2022
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
I'd like to process data that is coming from outside arrow using arrow compute kernels without copying the data.
The FFI api does not seem to support this use case at the moment. The only way I found to create an
FFI_ArrowArray
was with an already existingArrayData
struct. CreatingArrayData
requires at least oneBuffer
, and creating aBuffer
from foreign memory again needs anFFI_ArrowArray
.Describe the solution you'd like
The
Buffer
already supports two ways of deallocating memory.FFI_ArrowArray
The second option can be generalized so that the callback can be any type using a dyn trait.
Describe alternatives you've considered
An alternative could be to make it easier to create an
FFI_ArrowArray
from a single memory region, but that seems to be more complex and also less general.Additional context
This functionality could also allow zero-copy conversion from arrow2.
The text was updated successfully, but these errors were encountered: