Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix _checked_mul_dims in the presence of 0s and overflow. #54255

Merged
merged 6 commits into from
Apr 26, 2024
Merged

fix _checked_mul_dims in the presence of 0s and overflow. #54255

merged 6 commits into from
Apr 26, 2024

Conversation

oscardssmith
Copy link
Member

fixes #54244.

@oscardssmith oscardssmith added arrays [a, r, r, a, y, s] bugfix This change fixes an existing bug backport 1.11 Change should be backported to release-1.11 labels Apr 25, 2024
@oscardssmith oscardssmith changed the title fix _checked_mul_dims fix _checked_mul_dims in the presence of 0s and overflow. Apr 25, 2024
@oscardssmith oscardssmith mentioned this pull request Apr 25, 2024
Closed
vtjnash
vtjnash requested changes Apr 25, 2024
View reviewed changes
Copy link
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to compute overflow wrong in the case where some of the values are negative

@oscardssmith
Copy link
Member Author

I haven't changed anything there, have I?

vtjnash
vtjnash reviewed Apr 25, 2024
View reviewed changes
Copy link
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aren't you masking the result unconditionally?

@oscardssmith
Copy link
Member Author

oscardssmith commented Apr 25, 2024 •
edited
Loading

Oh, this code is awful. I didn't realize you were checking for negatives by using unsigned checks on signed ints. Is that not UB?

@oscardssmith oscardssmith requested a review from vtjnash April 25, 2024 20:11
@oscardssmith
Copy link
Member Author

Turns out there was already a bug here!

julia> zeros(-4, 2, 0)
-4×2×0 Array{Float64, 3}

@nsajko
Copy link
Contributor

nsajko commented Apr 25, 2024

I think this code allows typemax(Int) as the size along a dimension, if the size along some other dimension is zero, and no size is negative. Is that intentional? FTR this fails with ArgumentError on all recent Julia releases: Array{Int}(undef, 0, typemax(Int)), from v1.8 to the v1.11 beta1, but succeeds on v1.7.

@oscardssmith
Copy link
Member Author

It does not allow typemax(Int) (because of the ule_int check). If we wanted to allow them, we would need to use ult_int, but I don't think we do.

giordano
giordano reviewed Apr 25, 2024
View reviewed changes
test/core.jl Outdated Show resolved Hide resolved
vtjnash
vtjnash approved these changes Apr 26, 2024
View reviewed changes
Copy link
Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I specifically wanted maxdim+1 to be representable, since otherwise other off-by-one math representations become much harder, and that makes boundschecking slower.

@oscardssmith oscardssmith merged commit aeac289 into master Apr 26, 2024
7 checks passed
@oscardssmith oscardssmith deleted the os/check-dims-redux branch April 26, 2024 03:00
@giordano giordano mentioned this pull request May 1, 2024
Closed
@LilithHafner LilithHafner mentioned this pull request May 1, 2024
Open
KristofferC pushed a commit that referenced this pull request May 6, 2024
@oscardssmith @KristofferC
fix _checked_mul_dims in the presence of 0s and overflow. (#54255)
92db099 
fixes #54244.

(cherry picked from commit aeac289)
@KristofferC KristofferC mentioned this pull request May 6, 2024
Merged
59 tasks
KristofferC added a commit that referenced this pull request May 28, 2024
@KristofferC
Backports for 1.11.0-beta2 (#54112)
f7295bf 
Backported PRs:
- [x] #53665 <!-- use afoldl instead of tail recursion for tuples -->
- [x] #53976 <!-- LinearAlgebra: LazyString in interpolated error
messages -->
- [x] #54005 <!-- make `view(::Memory, ::Colon)` produce a Vector -->
- [x] #54010 <!-- Overload `Base.literal_pow` for `AbstractQ` -->
- [x] #54069 <!-- Allow PrecompileTools to see MI's inferred by foreign
abstract interpreters -->
- [x] #53750 <!-- inference correctness: fields and globals can revert
to undef -->
- [x] #53984 <!-- Profile: fix heap snapshot is valid char check -->
- [x] #54102 <!-- Explicitly compute stride in unaliascopy for SubArray
-->
- [x] #54070 <!-- Fix integer overflow in `skip(s::IOBuffer,
typemax(Int64))` -->
- [x] #54013 <!-- Support case-changes to Annotated{String,Char}s -->
- [x] #53941 <!-- Fix writing of AnnotatedChars to AnnotatedIOBuffer -->
- [x] #54137 <!-- Fix typo in docs for `partialsortperm` -->
- [x] #54129 <!-- use correct size when creating output data from an
IOBuffer -->
- [x] #54153 <!-- Fixup IdSet docstring -->
- [x] #54143 <!-- Fix `make install` from tarballs -->
- [x] #54151 <!-- LinearAlgebra: Correct zero element in
`_generic_matvecmul!` for block adj/trans -->
- [x] #54213 <!-- Add `public` statement to `Base.GC` -->
- [x] #54222 <!-- Utilize correct tbaa when emitting stores of unions.
-->
- [x] #54233 <!-- set MAX_OS_WRITE on unix -->
- [x] #54255 <!-- fix `_checked_mul_dims` in the presence of 0s and
overflow. -->
- [x] #54259 <!-- Fix typo in `readuntil` -->
- [x] #54251 <!-- fix typo in gc_mark_memory8 when chunking a large
array -->
- [x] #54276 <!-- Fix solve for complex `Hermitian` with non-vanishing
imaginary part on diagonal -->
- [x] #54248 <!-- ensure package callbacks are invoked when no valid
precompile file exists for an "auto loaded" stdlib -->
- [x] #54308 <!-- Implement eval-able AnnotatedString 2-arg show -->
- [x] #54302 <!-- Specialised substring equality for annotated strs -->
- [x] #54243 <!-- prevent `package_callbacks` to run multiple time for a
single package -->
- [x] #54350 <!-- add a precompile signature to Artifacts code that is
used by JLLs -->
- [x] #54331 <!-- correctly track freed bytes in
jl_genericmemory_to_string -->
- [x] #53509 <!-- revert moving "creating packages" from Pkg.jl -->
- [x] #54335 <!-- When accessing the data pointer for an array, first
decay it to a Derived Pointer -->
- [x] #54239 <!-- Make sure `fieldcount` constant-folds for `Tuple{...}`
-->
- [x] #54288
- [x] #54067
- [x] #53715 <!-- Add read/write specialisation for IOContext{AnnIO} -->
- [x] #54289 <!-- Rework annotation ordering/optimisations -->
- [x] #53815 <!-- create phantom task for GC threads -->
- [x] #54130 <!-- inference: handle `LimitedAccuracy` in
`handle_global_assignment!` -->
- [x] #54428 <!-- Move ConsoleLogging.jl into Base -->
- [x] #54332 <!-- Revert "add unsetindex support to more copyto methods
(#51760)" -->
- [x] #53826 <!-- Make all command-line options documented in all
related files -->
- [x] #54465 <!-- typeintersect: conservative typevar subtitution during
`finish_unionall` -->
- [x] #54514 <!-- typeintersect: followup cleanup for the nothrow path
of type instantiation -->
- [x] #54499 <!-- make `@doc x` work without REPL loaded -->
- [x] #54210 <!-- attach finalizer in `mmap` to the correct object -->
- [x] #54359 <!-- Pkg REPL: cache `pkg_mode` lookup -->

Non-merged PRs with backport label:
- [ ] #54471 <!-- Actually setup jit targets when compiling
packageimages instead of targeting only one -->
- [ ] #54457 <!-- Make `String(::Memory)` copy -->
- [ ] #54323 <!-- inference: fix too conservative effects for recursive
cycles -->
- [ ] #54322 <!-- effects: add new `@consistent_overlay` macro -->
- [ ] #54191 <!-- make `AbstractPipe` public -->
- [ ] #53957 <!-- tweak how filtering is done for what packages should
be precompiled -->
- [ ] #53882 <!-- Warn about cycles in extension precompilation -->
- [ ] #53707 <!-- Make ScopedValue public -->
- [ ] #53452 <!-- RFC: allow Tuple{Union{}}, returning Union{} -->
- [ ] #53402 <!-- Add `jl_getaffinity` and `jl_setaffinity` -->
- [ ] #53286 <!-- Raise an error when using `include_dependency` with
non-existent file or directory -->
- [ ] #52694 <!-- Reinstate similar for AbstractQ for backward
compatibility -->
- [ ] #51479 <!-- prevent code loading from lookin in the versioned
environment when building Julia -->
@KristofferC KristofferC removed the backport 1.11 Change should be backported to release-1.11 label May 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@giordano giordano giordano left review comments

@vtjnash vtjnash vtjnash approved these changes

Assignees
No one assigned
Labels
arrays [a, r, r, a, y, s] bugfix This change fixes an existing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

whether Core.checked_dims decides to throw depends on dimension order (for empty arrays)
5 participants
@oscardssmith @nsajko @vtjnash @giordano @KristofferC