Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports release 1.10 #51563

Merged
merged 46 commits into from
Nov 2, 2023
Merged

Backports release 1.10 #51563

merged 46 commits into from
Nov 2, 2023

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Oct 3, 2023

Backported PRs:

Contains multiple commits, manual intervention needed:

Non-merged PRs with backport label:

Keno and others added 9 commits October 3, 2023 15:21
When running under `rr`, it needs to patch out `rdtsc` to record the
values returned. If this is not possible, `rr` falls back to an
expensive signal-based emulation. As of rr master, a specific `nopl;
rdtsc` sequence may be used to guarantee that `rdtsc` patching is always
possible. Use this sequence for uses of rdtsc in our runtime.

(cherry picked from commit ce3f97c)
The guard instruction for unreachables and other crashes in aarch64 is
`brk`, in macos there isn't a distinction between a brk for a breakpoint
and one for a crash, as an attempt we check the value of `pc` when the
signal is triggered, if it is
`brk #0x1` we say that it is a crash and go into the sigdie_handler.

We should probably do the same in aarch64 linux, though I haven't dug
too deep into what values it uses for traps, and if what compiler used
matters, on apple I assumed we use clang/LLVM

It might be possible to test this by calling some inline assembly.

This means that something like
#51267 actually crashes with
```c
[16908] signal (5): Trace/BPT trap: 5
in expression starting at /Users/gabrielbaraldi/julia/test.jl:2
_collect at ./array.jl:768
collect at ./array.jl:757
top-level scope at /Users/gabrielbaraldi/julia/test.jl:5
_jl_invoke at /Users/gabrielbaraldi/julia/src/gf.c:2892
jl_toplevel_eval_flex at /Users/gabrielbaraldi/julia/src/toplevel.c:925
jl_toplevel_eval_flex at /Users/gabrielbaraldi/julia/src/toplevel.c:877
ijl_toplevel_eval at /Users/gabrielbaraldi/julia/src/toplevel.c:943 [inlined]
ijl_toplevel_eval_in at /Users/gabrielbaraldi/julia/src/toplevel.c:985
eval at ./boot.jl:383 [inlined]
include_string at ./loading.jl:2070
_jl_invoke at /Users/gabrielbaraldi/julia/src/gf.c:2873
ijl_apply_generic at /Users/gabrielbaraldi/julia/src/gf.c:3074
_include at ./loading.jl:2130
include at ./Base.jl:494
jfptr_include_46486 at /Users/gabrielbaraldi/julia/usr/lib/julia/sys.dylib (unknown line)
_jl_invoke at /Users/gabrielbaraldi/julia/src/gf.c:2873
ijl_apply_generic at /Users/gabrielbaraldi/julia/src/gf.c:3074
exec_options at ./client.jl:317
_start at ./client.jl:552
jfptr__start_83179 at /Users/gabrielbaraldi/julia/usr/lib/julia/sys.dylib (unknown line)
_jl_invoke at /Users/gabrielbaraldi/julia/src/gf.c:2873
ijl_apply_generic at /Users/gabrielbaraldi/julia/src/gf.c:3074
jl_apply at /Users/gabrielbaraldi/julia/src/./julia.h:1970 [inlined]
true_main at /Users/gabrielbaraldi/julia/src/jlapi.c:582
jl_repl_entrypoint at /Users/gabrielbaraldi/julia/src/jlapi.c:731
Allocations: 570978 (Pool: 570031; Big: 947); GC: 1
fish: Job 1, './julia test.jl' terminated by signal SIGTRAP (Trace or breakpoint trap)
```
instead of hanging silently

---------

Co-authored-by: Jameson Nash <[email protected]>
(cherry picked from commit d51ad06)
This PR adds an optional field to the existing `Xoshiro` struct to be
able to faithfully copy the task-local RNG state.

Fixes #51255
Redo of #51271

Background context: #49110 added an additional state to the task-local
RNG. However, before this PR `copy(default_rng())` did not include this
extra state, causing subtle errors in `Test` where `copy(default_rng())`
is assumed to contain the full task-local RNG state.

(cherry picked from commit 41b41ab)
The work I did in #41099 introduced code which, if method information
could not be found for an inlined frame, would fall back to use the
module of the next-higher stack frame. This often worked there because
the only frames that would not be assigned a module at this point would
be e.g., `macro expansion` frames.

However, due to the performance impact of the way method roots are
currently encoded, the extra method roots were removed in #50546.

The result is that inlined frames were being assigned a potentially
incorrect module, rather than being left blank.

Example:
```
julia> @Btime plot([1 2 3], seriestype = :blah)
...
 [13] #invokelatest#2
    @ BenchmarkTools ./essentials.jl:901 [inlined]
 [14] invokelatest
    @ BenchmarkTools ./essentials.jl:896 [inlined]
...
```

(cherry picked from commit ed891d6)
Co-authored-by: Valentin Churavy <[email protected]>
(cherry picked from commit ac8246f)
@KristofferC KristofferC added the release Release management and versioning. label Oct 3, 2023
@rfourquet
Copy link
Member

I can backport #51376 if it helps.

@KristofferC
Copy link
Member Author

Sure, thanks.

rfourquet and others added 10 commits October 3, 2023 18:44
As pointed out in
#51594 (comment),
this is necessary for the assertion added in
#49447 to be valid.

Fix #51594

(cherry picked from commit c18e485)
This helps with type-stability, as the flag `tA` is usually known from
the type of the matrix.
On master,
```julia
julia> f(A) = LinearAlgebra.wrap(A, 'N')
f (generic function with 1 method)

julia> @code_typed f([1;;])
CodeInfo(
1 ─ %1 = invoke LinearAlgebra.wrap(A::Matrix{Int64}, 'N'::Char)::Union{Adjoint{Int64, Matrix{Int64}}, Hermitian{Int64, Matrix{Int64}}, Symmetric{Int64, Matrix{Int64}}, Transpose{Int64, Matrix{Int64}}, Matrix{Int64}}
└──      return %1
) => Union{Adjoint{Int64, Matrix{Int64}}, Hermitian{Int64, Matrix{Int64}}, Symmetric{Int64, Matrix{Int64}}, Transpose{Int64, Matrix{Int64}}, Matrix{Int64}}
```
This PR
```julia
julia> @code_typed f([1;;])
CodeInfo(
1 ─     return A
) => Matrix{Int64}
```

(cherry picked from commit 0fd7f72)
Fixes #51576 on a simple
snapshot I collected on my machine.

(cherry picked from commit 5bdc1b3)
This seems to be the right combination of annotations to fix both #50562
and an inference regression in PropertyDicts.jl on the 1.10 release
branch. When backported the `@noinline` should be restored for 1.10.

(cherry picked from commit f7e8f92)
@IanButterworth
Copy link
Member

I accidentally updated Pkg on release-1.10 directly, so updated this branch

KristofferC and others added 7 commits October 11, 2023 14:41
Fix #51194

This PR fixes a regression introduced in
#49294, so I believe it should be
backported to v1.10.
In the current code, completion of `qux(foo, bar.` is detected by
parsing `foo(qux, bar` as an incomplete expression, and then looking for
the sub-expression to complete (here, `bar.`). This approach fails
however for infix calls, since completing `foo + bar.` starts by parsing
`foo + bar`, which is a complete call expression, and so the code
behaves as if completing `(foo + bar).` instead of `bar.`. This leads to
the current problematic behaviour:
```julia
julia> Complex(1, 3) + (4//5).#TAB
im
re
```
which would be correct for `(Complex(1, 3) + (4//5)).#TAB`, but here we
expect
```julia
julia> Complex(1, 3) + (4//5).#TAB
den
num
```

This PR fixes that by trying to detect infix calls. In the long term,
all this ad-hoc and probably somewhat wrong string processing should be
replaced by proper use of `JuliaSyntax` (as mentioned in
#49294 (comment),
#50817 (comment)
and probably other places), but for now at least this fixes the
regression.

(cherry picked from commit e949236)
The 1.10 GC heuristics introduced in
#50144 have been a source of
concerning issues such as
#50705 and
#51601. The PR also doesn't
correctly implement the paper on which it's based, as discussed in
#51498.

Test whether the 1.8 GC heuristics are a viable option.
Handling `AbstractQ`s in concatenation got lost in the overhaul. This
brings it back (though it seemed to not be used anywhere), and even
better: pre-1.9 `Q`s where handled via `AbstractMatrix` fallbacks, so
elementwise. Now, it first materializes the matrix, and then copies to
the right place in the destination array.

(cherry picked from commit 50146b2)
IanButterworth and others added 6 commits October 23, 2023 12:37
ccall was not creating roots for the contents of struct values which
contained roots on the stack, as expected to align with `GC.@preserve`,
and causing many segfaults for #51319

(cherry picked from commit e36f65f)
Disallow some type combinations that don't make sense.

---------

Co-authored-by: Ben Baumgold <[email protected]>
(cherry picked from commit 8a889ff)
This fixes a string-indexing bug introduced in #24713 (Julia 0.7).
Sometimes, this would cause `parse(Complex{T}, string)` to throw a
`StringIndexError` rather than an `ArgumentError`, e.g. for
`parse(ComplexF64, "3 β+ 4im")` or `parse(ComplexF64, "3 + 4αm")`. (As
far as I can tell, it can never cause parsing to fail for valid
strings.)

The source of the error is that if `i` is the index of an ASCII
character in a string `s`, then `s[i+1]` is valid (even if the next
character is non-ASCII) but `s[i-1]` is invalid if the previous
character is non-ASCII.

(cherry picked from commit f71228d)
@KristofferC
Copy link
Member Author

@nanosoldier runtests()

@nanosoldier
Copy link
Collaborator

Your job failed: error when preparing/pushing to report repo.

@maleadt
Copy link
Member

maleadt commented Oct 24, 2023

Your job failed: error when preparing/pushing to report repo.

You can find the report here: https://github.com/JuliaCI/NanosoldierReports/tree/d25a7aacfae7ad645bc3d794b038afce31584248/pkgeval/by_hash/e862940_vs_0ba6ec2

KristofferC and others added 4 commits October 24, 2023 10:43
Thanks ScottPJones for finding this.

Co-authored-by: Lilith Hafner <[email protected]>
(cherry picked from commit b197197)
[Docs](https://docs.julialang.org/en/v1/manual/control-flow/#else-Clauses)
state:

> The try, catch, else, and finally clauses each introduce their own
> scope blocks.

But it is currently not the case for `else` blocks

```julia
julia> try
       catch
       else
           z = 1
       end
1

julia> z
1
```

This change actually makes `else` blocks have their own scope block:

```julia
julia> try
       catch
       else
           z = 1
       end
1

julia> z
ERROR: UndefVarError: `z` not defined
```

(cherry picked from commit 17a36ee)
Fixes #51771

The convert method that asserts in #51771 is arguably still faulty
though.

(cherry picked from commit cf00550)
@KristofferC
Copy link
Member Author

@nanosoldier runtests(["BayesianQuadrature", "AcousticRayTracers", "MonthlyDates", "QDates", "CompEcon", "PauliPopRec", "TimeDataFrames", "Tapestree", "FrankWolfe", "MathML", "EnsembleKalmanProcesses", "DataDrivenSparse", "MEDYANSimRunner", "TMLE", "SpatialAccessTrees", "DataDrivenDMD", "DIVAnd", "ChargeTransport", "OptimizationPRIMA", "ReactionNetworkImporters", "Intervals", "UnixTimes", "SOCRATESSingleColumnForcings", "PolySignedDistance", "DPFEHM", "Electrum", "TableTransforms", "ControlSystemsMTK", "PALEOocean", "QuantumOptics", "FastAI", "EditorsRepo", "Test", "ODE", "MomentClosure", "HybridArrays", "Modia", "SteadyStateDiffEq", "QuantumPropagators", "Symbolics", "MCPTrajectoryGameSolver", "PeriodicalDates", "JutulDarcy", "CategoricalDistributions", "VoronoiFVM"])

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

d-netto and others added 4 commits October 24, 2023 20:28
Otherwise `--heap-size-hint` will become a no-op.

Likely a merge bug from #51661.
It seems this case has already been fixed by other improvements, so we
no longer need this hack, which is now causing problems.

Fixes #51694
…ion. (#51840)

This allows other users of LLVM to use opaque pointers with their
contexts.

Co-authored-by: Jameson Nash <[email protected]>
Restores the method whose removal was probably causing problems.

(cherry picked from commit 14d9c7c)
@JeffBezanson
Copy link
Member

@nanosoldier runtests(["BayesianQuadrature", "AcousticRayTracers", "MonthlyDates", "QDates", "CompEcon", "PauliPopRec", "TimeDataFrames", "Tapestree", "FrankWolfe", "MathML", "EnsembleKalmanProcesses", "DataDrivenSparse", "MEDYANSimRunner", "TMLE", "SpatialAccessTrees", "DataDrivenDMD", "DIVAnd", "ChargeTransport", "OptimizationPRIMA", "ReactionNetworkImporters", "Intervals", "UnixTimes", "SOCRATESSingleColumnForcings", "PolySignedDistance", "DPFEHM", "Electrum", "TableTransforms", "ControlSystemsMTK", "PALEOocean", "QuantumOptics", "FastAI", "EditorsRepo", "Test", "ODE", "MomentClosure", "HybridArrays", "Modia", "SteadyStateDiffEq", "QuantumPropagators", "Symbolics", "MCPTrajectoryGameSolver", "PeriodicalDates", "JutulDarcy", "CategoricalDistributions", "VoronoiFVM"])

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

vchuravy and others added 3 commits October 27, 2023 08:59
Restores the method whose removal was probably causing problems.

(cherry picked from commit f6f1ee9)
@KristofferC
Copy link
Member Author

@nanosoldier runtests(["PauliPopRec", "UnixTimes", "HybridArrays", "MEDYANSimRunner", "Electrum", "CompEcon", "CategoricalDistributions", "EnsembleKalmanProcesses", "DIVAnd", "FrankWolfe", "TableTransforms"])

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

KristofferC and others added 2 commits October 31, 2023 12:39
This shouldn't be needed because `ldd` should do it itself.

(cherry picked from commit 5b34cdf)
@KristofferC KristofferC merged commit 1391444 into release-1.10 Nov 2, 2023
4 of 6 checks passed
@KristofferC KristofferC deleted the backports-release-1.10 branch November 2, 2023 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.