-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-15745: [Java] Deprecate redundant iterable of ScanTask #14168
Conversation
@@ -68,6 +68,15 @@ ArrowReader execute() { | |||
} | |||
|
|||
@Override | |||
public NativeScanTask scanTask() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We just shouldn't have ScanTask at all, there is no such thing on the C++ side anymore.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, let me delete that, this scan()
method is exposed to the client and will cause break compatibilities
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The deprecation makes sense to me, I think we should consider what the new interface is, though. I would think something like ArrowReader scanBatchesUnordered()
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That would mirror the C++ API
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed
java/dataset/src/main/java/org/apache/arrow/dataset/scanner/Scanner.java
Outdated
Show resolved
Hide resolved
* | ||
* @return a {@link ArrowReader}. | ||
*/ | ||
ArrowReader scanBatchesUnordered(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, it looks like it's actually scanBatches
in the implementation:
ARROW_ASSIGN_OR_RAISE(auto batch_itr, scanner->ScanBatches()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, updated
java/dataset/src/main/java/org/apache/arrow/dataset/jni/NativeScanner.java
Outdated
Show resolved
Hide resolved
Co-authored-by: David Li <[email protected]>
Benchmark runs are scheduled for baseline = d7258aa and contender = fc98c95. fc98c95 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
…14168) Deprecate redundant iterable ScanTask since there are no more ScanTasks on the C++ side Authored-by: david dali susanibar arce <[email protected]> Signed-off-by: David Li <[email protected]>
…14168) Deprecate redundant iterable ScanTask since there are no more ScanTasks on the C++ side Authored-by: david dali susanibar arce <[email protected]> Signed-off-by: David Li <[email protected]>
Deprecate redundant iterable ScanTask since there are no more ScanTasks on the C++ side