Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Array#unshift for large arrays #11656

Merged

Conversation

HertzDevil
Copy link
Contributor

@HertzDevil HertzDevil commented Dec 27, 2021

#unshift still assumes the new capacity is exactly twice the old one whenever a reallocation is needed, which is no longer true for large arrays after #11482, so everything past around the first new_capacity / 2 elements would be referring to invalid storage, leading to potential heap corruption:

x = Array.new(300, &.itself)
x.@capacity # => 300
x           # => [0, 1, 2, ..., 282, 283, 284, 285, ..., 298, 299]

x.unshift 1000
x.@capacity # => 567
x           # => [1000, 0, 1, 2, ..., 282, 283, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

x = Array.new(300) { [] of Int32 }
x.unshift [1]
x # => [[1], [], [], ..., [Invalid memory access (signal 11) at address 0x4

Pushes will also no longer reallocate:

x = Array.new(300, &.itself)
x.@capacity # => 300
x.size      # => 300
x.unshift 1000
x.@capacity # => 567
x.size      # => 301
50000.times { x << 2000 }
x.@capacity # => 567
x.size      # => 50301

This PR fixes it.

@HertzDevil HertzDevil added kind:regression Something that used to correctly work but no longer works topic:stdlib:collection kind:bug A bug in the code. Does not apply to documentation, specs, etc. labels Dec 27, 2021
a = Array.new(300, &.itself)
[email protected] eq(300)
a.unshift 1
[email protected] be > 300
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not test the actual value?

Copy link
Member

@beta-ziliani beta-ziliani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@beta-ziliani beta-ziliani added this to the 1.3.0 milestone Dec 30, 2021
@straight-shoota straight-shoota merged commit 2c1a38d into crystal-lang:master Dec 31, 2021
@HertzDevil HertzDevil deleted the bug/array-unshift-large branch January 1, 2022 03:33
rdp pushed a commit to rdp/crystal that referenced this pull request Jan 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug A bug in the code. Does not apply to documentation, specs, etc. kind:regression Something that used to correctly work but no longer works topic:stdlib:collection
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants