You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe. GpuBaseLimitExec.sliceRDD is the basis for GpuGlobalLimitExec and GpuLocalLimitExec. The implementation is mostly deciding if a batch of data should be passed through unchanged, not at all, or sliced.
If the data is unchanged then there is no need for a retry because it is essentially a noop. If it is not passed through, then we never see the data, we just say in the iterator that there are no more batches. If we have to slice it, then we can put a retry block around the slice. But we cannot really support SplitAndRetry here. We are slicing the data into a subset of the input. If that does not work how do we expect to be able to split the input into multiple chunks that add up to the total size of the input before we slice it and throw away parts of it.
I guess if we had fragmentation..., but practically it is just not worth even trying here.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
GpuBaseLimitExec.sliceRDD
is the basis forGpuGlobalLimitExec
andGpuLocalLimitExec
. The implementation is mostly deciding if a batch of data should be passed through unchanged, not at all, or sliced.If the data is unchanged then there is no need for a retry because it is essentially a noop. If it is not passed through, then we never see the data, we just say in the iterator that there are no more batches. If we have to slice it, then we can put a retry block around the slice. But we cannot really support SplitAndRetry here. We are slicing the data into a subset of the input. If that does not work how do we expect to be able to split the input into multiple chunks that add up to the total size of the input before we slice it and throw away parts of it.
I guess if we had fragmentation..., but practically it is just not worth even trying here.
The text was updated successfully, but these errors were encountered: