Add getCompletedPositions to ConnectorPageSource #8524
Conversation
core/trino-spi/src/main/java/io/trino/spi/connector/ConnectorPageSource.java
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/operator/TableScanWorkProcessorOperator.java
Show resolved
Hide resolved
| @@ -150,6 +150,21 @@ public void testCustomMetricsScanOnly() | |||
| .getMetrics(); | |||
| } | |||
|
|
|||
| @Test(timeOut = 30_000) | |||
| public void testPhysicalInputPositions() | |||
There was a problem hiding this comment.
Could you add similar test for ORC in Hive connector?
There was a problem hiding this comment.
that's tricky to test, requires hive3 setup for acid and assertions on operator stats outside of memory connector are quite flaky
|
lgtm % @martint comment. I think we should either have explicit method for completed positions OR we should move other methods (like completed bytes) to metrics for consistency. However, I think we can start with former |
This allows connectors to report input rows processed by ConnectorPageSource rather than relying solely on the positions count of the page returned from getNextPage
raunaqmorarka
force-pushed
the
completed-positions
branch
from
8e6e2d5 to
b506f0a
Compare
I'm avoiding use of |
This allows connectors to report input rows processed
by ConnectorPageSource rather than relying solely on
the positions count of the page returned from getNextPage