You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For fixed width type you can assume here that there will be OUTPUT_PAGE_SIZE positions (type check on instaceof FixedWidthType.
For variable length type, (instanceof VariableWidthType) you can assume OUTPUT_PAGE_SIZE positions, but can also assume some skew via io.trino.spi.block.BlockUtil#calculateBlockResetBytes. You can then use io.trino.spi.type.Type#createBlockBuilder(io.trino.spi.block.BlockBuilderStatus, int, int) to construct builder of right size.
This should save memory compared to existing code. I wouldn't be surprised to see aggregation mem usage drop by 10-20%
I think we can extend io.trino.spi.PageBuilder#reset to accept expectedEntries.
@skrzypo987 No, the issue is that VariableWIdthBlockBuilder needs to be reset/recreated with a similar (currently 25% higher) expectedBytes. So we either do something as I did in #12512 or we need to expose
Follow up after #12336
For fixed width type you can assume here that there will be OUTPUT_PAGE_SIZE positions (type check on instaceof FixedWidthType.
For variable length type, (instanceof VariableWidthType) you can assume OUTPUT_PAGE_SIZE positions, but can also assume some skew via io.trino.spi.block.BlockUtil#calculateBlockResetBytes. You can then use io.trino.spi.type.Type#createBlockBuilder(io.trino.spi.block.BlockBuilderStatus, int, int) to construct builder of right size.
This should save memory compared to existing code. I wouldn't be surprised to see aggregation mem usage drop by 10-20%
I think we can extend io.trino.spi.PageBuilder#reset to accept expectedEntries.
https://github.com/trinodb/trino/pull/12072/files#r861792201
The text was updated successfully, but these errors were encountered: