-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix overflow exception in BitArray #8494
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -32,10 +32,17 @@ struct BitArray | |
|
||
def ==(other : BitArray) | ||
return false if size != other.size | ||
# NOTE: If BitArray implements resizing, there may be more than 1 binary | ||
# representation and their hashes for equivalent BitArrays after a downsize as the | ||
# discarded bits may not have been zeroed. | ||
return LibC.memcmp(@bits, other.@bits, malloc_size) == 0 | ||
return true if size == 0 | ||
return false if LibC.memcmp(@bits, other.@bits, malloc_size - 1) != 0 | ||
last = @bits[malloc_size - 1] | ||
other_last = other.@bits[malloc_size - 1] | ||
return true if last == other_last | ||
trailing = 32 - size % 32 | ||
if trailing != 32 | ||
last << trailing == other_last << trailing | ||
else | ||
false | ||
end | ||
end | ||
|
||
def ==(other) | ||
|
@@ -133,18 +140,18 @@ struct BitArray | |
bits = @bits[0] | ||
|
||
bits >>= start | ||
bits &= (1 << count) - 1 | ||
bits &= (1 << count) &- 1 | ||
|
||
BitArray.new(count).tap { |ba| ba.@bits[0] = bits } | ||
elsif size <= 64 | ||
# Original fits in int64, we can use bitshifts | ||
bits = @bits.as(UInt64*)[0] | ||
|
||
bits >>= start | ||
bits &= (1 << count) - 1 | ||
bits &= (1 << count) &- 1 | ||
|
||
if count <= 32 | ||
BitArray.new(count).tap { |ba| ba.@bits[0] = bits.to_u32 } | ||
BitArray.new(count).tap { |ba| ba.@bits[0] = bits.to_u32! } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see when this can overflow if
Usages of |
||
else | ||
BitArray.new(count).tap { |ba| [email protected](UInt64*)[0] = bits } | ||
end | ||
|
@@ -162,7 +169,7 @@ struct BitArray | |
bits = @bits[start_bit_index + i + 1] | ||
|
||
high_bits = bits | ||
high_bits &= (1 << start_sub_index) - 1 | ||
high_bits &= (1 << start_sub_index) &- 1 | ||
high_bits <<= 32 - start_sub_index | ||
|
||
ba.@bits[i] = low_bits | high_bits | ||
|
@@ -239,7 +246,13 @@ struct BitArray | |
# See `Object#hash(hasher)` | ||
def hash(hasher) | ||
hasher = size.hash(hasher) | ||
hasher = to_slice.hash(hasher) | ||
bytes, bits = @size.divmod(8) | ||
if bytes > 0 | ||
hasher = Slice.new(@bits.as(Pointer(UInt8)), bytes).hash(hasher) | ||
end | ||
if bits != 0 | ||
hasher = (@bits.as(Pointer(UInt8))[bytes] << 8 - bits).hash(hasher) | ||
end | ||
hasher | ||
end | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you explain why this entire change is needed? I don't understand why the last bit needs special handling. And if it does, there should be a comment in the code explaining why.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It needs special handling since unused bits at the end of the array are not guaranteed to be in a defined state, so they have to be masked out (i.e., set to zero) before comparing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, that's not correct.
Pointer.malloc
makes sure everything is zero. The comparison is broken becauseLibC.memcmp
expects the number of bytes but it usesmalloc_size
which is in terms of UInt32. Changing it to usemalloc_size * sizeof(UInt32)
makes it work.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see. The method
[]
will copy bits from an existing bit array and just copy them over without zeroing the non-interesting part, that's whymemcmp
doesn't work.I think we should fix
[]
to guarantee the invariant that unused bits are always zero.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't really care which method is used, this PR is certainly a valid method.
Nice catch on the memcmp size being wrong though.