Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use getSizeInBytes rather than getRetainedSizeInBytes #21653

Merged
merged 1 commit into from
Apr 22, 2024

Conversation

sopel39
Copy link
Member

@sopel39 sopel39 commented Apr 22, 2024

For processed data getSizeInBytes should be used

Description

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

For processed data getSizeInBytes should be used
@sopel39 sopel39 added the no-release-notes This pull request does not require release notes entry label Apr 22, 2024
@cla-bot cla-bot bot added the cla-signed label Apr 22, 2024
@@ -171,6 +171,6 @@ private long sendPageToPartition(Consumer<Page> buffer, Page pageSplit)
long retainedSizeInBytes = pageSplit.getRetainedSizeInBytes();
memoryManager.updateMemoryUsage(retainedSizeInBytes);
buffer.accept(pageSplit);
return retainedSizeInBytes;
return pageSplit.getSizeInBytes();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's wrong about the existing code ?
We're calling page.compact() or page.copyPositions before sendPageToPartition, so what difference will this change make ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getRetainedSizeInBytes does not equal getSizeInBytes after compact or copyPositions. An example is dictionary. getSizeInBytes was always used to measure flow of data

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is just about measuring data processed, then why don't we call getSizeInBytes once on the full Page before splitting it rather than calling it on each partitioned Page and then adding that up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, #21668

@sopel39 sopel39 merged commit 8c776bd into trinodb:master Apr 22, 2024
95 checks passed
@sopel39 sopel39 deleted the ks/use_get_size branch April 22, 2024 20:53
@github-actions github-actions bot added this to the 446 milestone Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed no-release-notes This pull request does not require release notes entry
Development

Successfully merging this pull request may close these issues.

3 participants