-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Backports for 1.10.3 #53714
Backports for 1.10.3 #53714
Conversation
Co-authored-by: Keno Fischer <[email protected]> (cherry picked from commit b8540d1)
(cherry picked from commit 71f68b4)
This regression at a large number of GC threads still shows up on a large aarch64 machine (80-core ARM Neoverse N1). ``` bench = "many_refs.jl" (gcthreads, threads) = (1, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 3807 │ 2826 │ 2633 │ 193 │ 960 │ 13 │ 892 │ 74 │ │ median │ 3810 │ 2826 │ 2633 │ 193 │ 961 │ 15 │ 892 │ 74 │ │ maximum │ 3810 │ 2831 │ 2638 │ 193 │ 962 │ 27 │ 892 │ 74 │ │ stdev │ 2 │ 3 │ 3 │ 0 │ 1 │ 7 │ 0 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (2, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2455 │ 1476 │ 1350 │ 126 │ 488 │ 20 │ 892 │ 60 │ │ median │ 2459 │ 1478 │ 1352 │ 126 │ 489 │ 23 │ 892 │ 60 │ │ maximum │ 2465 │ 1479 │ 1352 │ 126 │ 489 │ 23 │ 893 │ 60 │ │ stdev │ 5 │ 1 │ 1 │ 0 │ 1 │ 2 │ 0 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (4, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2173 │ 1194 │ 1115 │ 77 │ 463 │ 18 │ 892 │ 54 │ │ median │ 2187 │ 1194 │ 1116 │ 78 │ 463 │ 19 │ 893 │ 55 │ │ maximum │ 2217 │ 1208 │ 1130 │ 78 │ 463 │ 19 │ 893 │ 55 │ │ stdev │ 22 │ 8 │ 8 │ 1 │ 0 │ 1 │ 0 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (8, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2191 │ 1212 │ 1149 │ 63 │ 539 │ 19 │ 901 │ 55 │ │ median │ 2199 │ 1219 │ 1156 │ 63 │ 540 │ 20 │ 901 │ 55 │ │ maximum │ 2201 │ 1222 │ 1157 │ 65 │ 540 │ 20 │ 901 │ 56 │ │ stdev │ 5 │ 5 │ 4 │ 1 │ 1 │ 1 │ 0 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (16, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 3897 │ 2916 │ 2854 │ 60 │ 1423 │ 20 │ 902 │ 75 │ │ median │ 4012 │ 3032 │ 2972 │ 62 │ 1491 │ 21 │ 904 │ 75 │ │ maximum │ 4026 │ 3106 │ 3044 │ 62 │ 1511 │ 22 │ 904 │ 77 │ │ stdev │ 70 │ 96 │ 96 │ 1 │ 46 │ 1 │ 1 │ 1 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ ``` This patch seems to improve performance at 16 GC threads though we still have some negative scaling for this benchmark beyond 8 GC threads (others seem fine). I didn't test whether this is indeed the optimal struct layout for the queue in this benchmark. ``` bench = "many_refs.jl" (gcthreads, threads) = (1, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 3807 │ 2822 │ 2630 │ 193 │ 959 │ 15 │ 892 │ 74 │ │ median │ 3824 │ 2823 │ 2630 │ 193 │ 959 │ 15 │ 892 │ 74 │ │ maximum │ 4307 │ 3194 │ 2974 │ 220 │ 1086 │ 19 │ 892 │ 74 │ │ stdev │ 284 │ 215 │ 199 │ 16 │ 74 │ 2 │ 0 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (2, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2459 │ 1476 │ 1349 │ 126 │ 488 │ 19 │ 892 │ 60 │ │ median │ 2460 │ 1479 │ 1352 │ 127 │ 488 │ 21 │ 893 │ 60 │ │ maximum │ 2770 │ 1661 │ 1526 │ 135 │ 570 │ 22 │ 893 │ 60 │ │ stdev │ 179 │ 106 │ 101 │ 5 │ 47 │ 1 │ 0 │ 0 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (4, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2215 │ 1233 │ 1155 │ 77 │ 485 │ 20 │ 892 │ 53 │ │ median │ 2320 │ 1248 │ 1171 │ 78 │ 492 │ 21 │ 893 │ 54 │ │ maximum │ 2372 │ 1259 │ 1176 │ 83 │ 493 │ 22 │ 893 │ 56 │ │ stdev │ 80 │ 13 │ 11 │ 3 │ 5 │ 1 │ 1 │ 1 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (8, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2204 │ 1220 │ 1156 │ 64 │ 544 │ 23 │ 901 │ 53 │ │ median │ 2362 │ 1249 │ 1181 │ 64 │ 553 │ 23 │ 901 │ 53 │ │ maximum │ 2374 │ 1261 │ 1196 │ 68 │ 558 │ 25 │ 901 │ 55 │ │ stdev │ 95 │ 21 │ 20 │ 2 │ 7 │ 1 │ 0 │ 1 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ (gcthreads, threads) = (16, 1) [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback [ Info: Setting GC memory pressure callback ┌─────────┬────────────┬─────────┬───────────┬────────────┬──────────────┬───────────────────┬──────────┬────────────┐ │ │ total time │ gc time │ mark time │ sweep time │ max GC pause │ time to safepoint │ max heap │ percent gc │ │ │ ms │ ms │ ms │ ms │ ms │ us │ MB │ % │ ├─────────┼────────────┼─────────┼───────────┼────────────┼──────────────┼───────────────────┼──────────┼────────────┤ │ minimum │ 2502 │ 1519 │ 1458 │ 62 │ 721 │ 22 │ 902 │ 58 │ │ median │ 2511 │ 1524 │ 1461 │ 63 │ 728 │ 23 │ 903 │ 61 │ │ maximum │ 2664 │ 1554 │ 1486 │ 68 │ 741 │ 25 │ 905 │ 61 │ │ stdev │ 91 │ 19 │ 16 │ 3 │ 10 │ 1 │ 2 │ 1 │ └─────────┴────────────┴─────────┴───────────┴────────────┴──────────────┴───────────────────┴──────────┴────────────┘ ``` (cherry picked from commit 973b4ff)
(cherry picked from commit e9c84c8)
Cherry-pick the parts of #53355 which are relevant to the 1.10 GC.
This shouldn't have any cost on smaller images because the only thing that gets put into ldata is the system image data, which is only reference via `dlsym`. This allows for images larger than 2gb (tested by putting a 2gb array in the base image) I did not test how this might be handled in other platforms (Windows doesn't support it). (cherry picked from commit 0f04b33)
(cherry picked from commit abd00d0)
~~Apparently somewhere in codegen inside `emit_codeinst`some piece of code is relying on the implicit try catch gcstack restoring. I haven't got the analyzer working on that file yet (it has hundreds of issues and it doesn't like C++ code that much + the file is tens of thousands of lines after includes so it struggles).~~ This fixes the compileall segfault in apple-aarch64 ci. #53811 (cherry picked from commit 52fc796)
…41232) This PR fixes ```julia julia> r = reshape(Base.IdentityUnitRange(3:4), 2, 1) 2×1 reshape(::Base.IdentityUnitRange{UnitRange{Int64}}, 2, 1) with eltype Int64: 3 4 julia> collect(r) == r false julia> collect(r) 2×1 Matrix{Int64}: 3258125826116431922 3688512103538242609 ``` After this PR, ```julia julia> collect(r) 2×1 Matrix{Int64}: 3 4 ``` (cherry picked from commit 6e3044d)
It'd be great to get 1.10.3 out (with the threading gc segfault fix that's on this branch) |
Closes #54015. `diff -r` on the source tree for the original archive vs. the archive that gets downloaded now shows no changes.
This reduces dynamic dispatch and makes JET happier. Testing on v1.11: ```julia julia> import LinearAlgebra: checksquare julia> using JET julia> @report_opt checksquare(rand(2,2), rand(2,2)) ═════ 4 possible errors found ═════ ┌ checksquare(::Matrix{Float64}, ::Matrix{Float64}) @ LinearAlgebra /cache/build/builder-amdci4-1/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/LinearAlgebra/src/LinearAlgebra.jl:307 │┌ string(::String, ::Tuple{Int64, Int64}) @ Base ./strings/io.jl:189 ││┌ print_to_string(::String, ::Tuple{Int64, Int64}) @ Base ./strings/io.jl:150 │││┌ _unsafe_take!(io::IOBuffer) @ Base ./iobuffer.jl:494 ││││┌ wrap(::Type{Array}, m::MemoryRef{UInt8}, l::Int64) @ Base ./array.jl:3101 │││││ failed to optimize due to recursion: wrap(::Type{Array}, ::MemoryRef{UInt8}, ::Int64) ││││└──────────────────── │││┌ print_to_string(::String, ::Vararg{Any}) @ Base ./strings/io.jl:143 ││││ runtime dispatch detected: Base._str_sizehint(%17::Any)::Int64 │││└──────────────────── │││┌ print_to_string(::String, ::Vararg{Any}) @ Base ./strings/io.jl:148 ││││ runtime dispatch detected: print(%59::IOBuffer, %97::Any)::Any │││└──────────────────── │││┌ string(::String, ::Int64, ::String, ::Tuple{Int64}, ::String, ::Int64, ::String, ::Int64, ::String) @ Base ./strings/io.jl:189 ││││ failed to optimize due to recursion: string(::String, ::Int64, ::String, ::Tuple{Int64}, ::String, ::Int64, ::String, ::Int64, ::String) │││└──────────────────── julia> function checksquare(A...) # This PR sizes = Int[] for a in A size(a,1)==size(a,2) || throw(DimensionMismatch(lazy"matrix is not square: dimensions are $(size(a))")) push!(sizes, size(a,1)) end return sizes end checksquare (generic function with 2 methods) julia> @report_opt checksquare(rand(2,2), rand(2,2)) No errors detected ``` (cherry picked from commit d505c8c)
This is a part of issue #52725. --------- (cherry picked from commit 15e2af2) Co-Authored-By: Steven G. Johnson <[email protected]> Co-Authored-By: Max Horn <[email protected]>
…626b0 (#54049) Stdlib: Pkg URL: https://github.com/JuliaLang/Pkg.jl.git Stdlib branch: release-1.10 Julia branch: backports-release-1.10 Old commit: 70525539d New commit: f487626b0 Julia version: 1.10.2 Pkg version: 1.10.0(Does not match) Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Pkg.jl@7052553...f487626 ``` $ git log --oneline 70525539d..f487626b0 f487626b0 Merge pull request #3868 from JuliaLang/backports-release-1.10 e4a6078a2 prune manifest after the set of some deps have been "demoted" to weakdeps (#3864) 72ba57759 Report failures to download artifacts as failures (#3860) 4ee732f1b why: show more when package is both a direct and indirect dep (#3771) ``` Co-authored-by: Dilum Aluthge <[email protected]>
d561ec2
to
e386554
Compare
feceefe appears to have introduced a LinearAlgebra failure https://buildkite.com/julialang/julia-release-1-dot-10/builds/245#018ecd32-81ec-4189-99ed-8fcbb37d3b6c/634-688 which seems odd |
#54063) Stdlib: SHA URL: https://github.com/JuliaCrypto/SHA.jl.git Stdlib branch: release-0.7 Julia branch: backports-release-1.10 Old commit: 2d1f84e New commit: e1af7dd Julia version: 1.10.2 SHA version: 0.7.0(Does not match) Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaCrypto/SHA.jl@2d1f84e...e1af7dd ``` $ git log --oneline 2d1f84e..e1af7dd e1af7dd Hardcode doc edit backlink aaf2df6 [test] fix typo 0f70e8b [test] test `AbstractString` input 335322a [test] test branch in `digest!` 5992826 [test] Testing branch within the `HMAC_CTX` constructor 5e3a676 Protect against re-using digested context ``` Co-authored-by: Dilum Aluthge <[email protected]>
…5 to 8eec5cb (#54065) Stdlib: NetworkOptions URL: https://github.com/JuliaLang/NetworkOptions.jl.git Stdlib branch: master Julia branch: backports-release-1.10 Old commit: 0bd3345 New commit: 8eec5cb Julia version: 1.10.2 NetworkOptions version: 1.2.0(Does not match) Bump invoked by: @IanButterworth Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/NetworkOptions.jl@0bd3345...8eec5cb ``` $ git log --oneline 0bd3345..8eec5cb 8eec5cb Hardcode doc edit backlink (#32) ``` Co-authored-by: Dilum Aluthge <[email protected]>
1e28984
to
97b0571
Compare
I confirmed locally that this error was introduced by
which is weird for just a change switching to lazy strings? diff --git a/stdlib/LinearAlgebra/src/LinearAlgebra.jl b/stdlib/LinearAlgebra/src/LinearAlgebra.jl
index 494572b1e8dd8..7b3ff8f0db53d 100644
--- a/stdlib/LinearAlgebra/src/LinearAlgebra.jl
+++ b/stdlib/LinearAlgebra/src/LinearAlgebra.jl
@@ -238,14 +238,14 @@ julia> LinearAlgebra.checksquare(A, B)
"""
function checksquare(A)
m,n = size(A)
- m == n || throw(DimensionMismatch("matrix is not square: dimensions are $(size(A))"))
+ m == n || throw(DimensionMismatch(lazy"matrix is not square: dimensions are $(size(A))"))
m
end
function checksquare(A...)
sizes = Int[]
for a in A
- size(a,1)==size(a,2) || throw(DimensionMismatch("matrix is not square: dimensions are $(size(a))"))
+ size(a,1)==size(a,2) || throw(DimensionMismatch(lazy"matrix is not square: dimensions are $(size(a))"))
push!(sizes, size(a,1))
end
return sizes Specifically, reverting just this one makes tests pass
Also leaving it lazy and adding a |
I'll just revert feceefe for now but we should probably open an issue to track this since it might indicate some deeper issue. |
@nanosoldier |
@nanosoldier |
Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. |
The package evaluation job you requested has completed - possible new issues were detected. |
Seeing some weird stuff in the PkgEval logs for a few packages:
|
This reverts commit 7536f35.
@nanosoldier |
The package evaluation job you requested has completed - possible new issues were detected. |
Are there things remaining for this? |
Backported PRs:
LazyString
inLinearAlgebra.checksquare
error message #53961Need manual backport:
Contains multiple commits, manual intervention needed:
UnionAll
unaliasing bug caused by innervars. #53553Non-merged PRs with backport label:
Base.literal_pow
forAbstractQ
#54010