-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Simplify TableFunctionSplitProcessor interface #17032
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hate the idea of another rebase but this looks valid
21e38d8
to
cbb1418
Compare
@@ -32,7 +33,7 @@ default TableFunctionDataProcessor getDataProcessor(ConnectorTableFunctionHandle | |||
* This method returns a {@code TableFunctionSplitProcessor}. All the necessary information collected during analysis is available | |||
* in the form of {@link ConnectorTableFunctionHandle}. It is called once per each split processed by the table function. | |||
*/ | |||
default TableFunctionSplitProcessor getSplitProcessor(ConnectorSession session, ConnectorTableFunctionHandle handle) | |||
default TableFunctionSplitProcessor getSplitProcessor(ConnectorSession session, ConnectorTableFunctionHandle handle, ConnectorSplit split) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* @return {@link TableFunctionProcessorState} including the processor's state and optionally a portion of result. | ||
* After the returned state is {@code FINISHED}, the method will not be called again for the split currently being processed | ||
* and may be called with a new split. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was actually incorrect, broken in #16955
core/trino-spi/src/main/java/io/trino/spi/ptf/TableFunctionSplitProcessor.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/LeafTableFunctionOperator.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/LeafTableFunctionOperator.java
Outdated
Show resolved
Hide resolved
The function |
cbb1418
to
3fe52ff
Compare
(just rebased) |
3fe52ff
to
a8ae043
Compare
(just rebased, for real this time, undoing some other stuff pushed together with rebase) |
a8ae043
to
cb0d0e6
Compare
AC |
cb0d0e6
to
305adfa
Compare
March 31 is in the past.
All table functions processing interfaces are experimental (for example `TableFunctionProcessorProvider`).
`ArrayList` should not be used for cases where we remove elements from the list start.
The `TableFunctionSplitProcessor` is used for one split only and then new instance of `TableFunctionSplitProcessor` is created. Provide the split directly to `TableFunctionProcessorProvider.getSplitProcessor` and do not provide it to `TableFunctionProcessor.process`. This removes conditionality of argument presence in the `process` method and so makes things simpler.
305adfa
to
f2ab11c
Compare
The
TableFunctionSplitProcessor
is used for one split only and thennew instance of
TableFunctionSplitProcessor
is created.Provide the split directly to
TableFunctionProcessorProvider.getSplitProcessor
and do not provide itto
TableFunctionProcessor.process
. This removes conditionality ofargument presence in the
process
method and so makes things simpler.