You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
GpuExpandExec sadly keeps a ColumnarBatch around between calls. Not a SpillableGpuColumnarBatch or anything like that.
We should update it to use a SpillableColumnarBatch. Support tiered project so that we can get the retry easily.
We also have a lot of code in there for handling columnaEval returning a null, but it should never return a null it would return a GpuScalar, so we should rip out that code. It was a premature optimization that also could cache data in memory that is not spillable. So two wrongs are making a right here.
This really only runs right before a hash aggregate, but with the retry logic in hash aggregate it is kind of scary and I think we should fix this sooner than later.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
GpuExpandExec sadly keeps a ColumnarBatch around between calls. Not a SpillableGpuColumnarBatch or anything like that.
We should update it to use a SpillableColumnarBatch. Support tiered project so that we can get the retry easily.
We also have a lot of code in there for handling columnaEval returning a
null
, but it should never return a null it would return a GpuScalar, so we should rip out that code. It was a premature optimization that also could cache data in memory that is not spillable. So two wrongs are making a right here.This really only runs right before a hash aggregate, but with the retry logic in hash aggregate it is kind of scary and I think we should fix this sooner than later.
The text was updated successfully, but these errors were encountered: