-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set maxOutputLength correctly #16604
Conversation
@arhimondr now clue how to test it though for now. The |
@@ -337,7 +337,7 @@ private void decompress() | |||
blockSize, | |||
sink.getSlice().byteArray(), | |||
sink.getSlice().byteArrayOffset() + bytesPreserved, | |||
sink.getSlice().length()); | |||
sink.getSlice().length() - bytesPreserved); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually code below:
System.arraycopy(
source.getSlice().byteArray(),
source.getSlice().byteArrayOffset() + source.getPosition(),
sink.getSlice().byteArray(),
sink.getSlice().byteArrayOffset() + bytesPreserved,
blockSize);
seems broken too. Do we know that blockSize
will always fit in the targetSlice given some of the data is taken by bytesPreserved
@arhimondr ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
int bufferSize = blockSizeInBytes
// to guarantee a single long can always be read entirely
+ Long.BYTES;
buffers[0] = new ReadBuffer(Slices.allocate(bufferSize))
The buffer size is allocated according to blockSize
with an extra 8
bytes (one long) for carry-overs.
The carry over is guaranteed to never be higher than 8
bytes as this is the most what can be requested to be read in one shot.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
e671f64
to
d0898ef
Compare
@arhimondr I tried to add regression test (attempt is here) but with no luck. I cannot trigger the code flow which results in check failure. |
Previously we were not accounting for bytesPreserved. With recent check added to aircompressor code (airlift/aircompressor@127b7f3) it started manifesting as java.lang.IllegalArgumentException. This is strictly validation fix. The passed array was not filled beyond allocated size.
d0898ef
to
f50081c
Compare
@arhimondr updated with test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. Great idea with constructing a specialized block encoder to cover the problematic code path. Love it!
Previously we were not accounting for bytesPreserved.
With recent check added to aircompressor code
(airlift/aircompressor@127b7f3)
it started manifesting as java.lang.IllegalArgumentException.
This is strictly validation fix. The passed array was not filled beyond
allocated size.
Description
fixes: #16541
Additional context and related issues
Release notes
( ) This is not user-visible or docs only and no release notes are required.
(x) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text: