DO NOT MERGE: v1.12 branch for comparison to master #202
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nickrobinson251
force-pushed
the
v1.12.0-DEV+RAI
branch
4 times, most recently
from
966538b to
9da665d
Compare
d-netto
force-pushed
the
v1.12.0-DEV+RAI
branch
from
9da665d to
48a5871
Compare
nickrobinson251
force-pushed
the
v1.12.0-DEV+RAI
branch
from
48a5871 to
5974904
Compare
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
5 times, most recently
from
50775dd to
fd6934f
Compare
I couldn't find `time_ns` when I was looking for it, nice to make clear the "monotonic" clock is also available in Base. (cherry picked from commit a52de83)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
fd6934f to
1cff7d7
Compare
…uliaLang#57241) The `fork()` we do here relies on `SIGCHLD` to make sure that we don't race against the child. This is easy to see in an embedding application that dynamically links `libjulia`: ```c int main(int argc, char *argv[]) { signal(SIGCHLD, SIG_IGN); void *handle = dlopen("path/to/libjulia.so", RTLD_LAZY); return 0; } ``` Without this change, this fails with an error message: ``` Error during libstdcxxprobe in parent process: waitpid: No child processes ``` Resolves JuliaLang#57240 (cherry picked from commit daf865e)
Restores JuliaLang#57035, undo JuliaLang#57089 for non-FreeBSD. While I suggested doing this change for all platforms, I forgot that means non-FreeBSD platforms become vulnerable again to the very deadlock problems that JuliaLang#57035 was required to prevent. That fix seems to not be viable on FreeBSD due to known libc implementation problems on that platform. However, upon closer inspection of the questionable design implementation decisions they seem to have made here, the platform is likely not currently vulnerable to this libunwind bug in the first place: https://github.com/lattera/freebsd/blob/master/libexec/rtld-elf/rtld_lock.c#L120 (cherry picked from commit 2f0a523)
…57211) It looks like these methods were just missed while overloading for BufferStream. There's also `readbytes!` where the current implementation will fallback to the `LibuvStream` implementation that is currently not threadsafe. What's the best approach there since the implementation is quite a bit more involved? Just duplicate the code but for BufferStream? Should we take the BufferStream lock and invoke the LibuvStream method? Open to ideas there. Also open to suggestions for having tests here? Not easy to simulate the data race of writing and calling readavailable. The fix here will unblock JuliaWeb/HTTP.jl#1213 (I'll probably do some compat shim there until this is fully released). Thanks to @oscardssmith for rubber ducking this issue with me. Probably most helpfully reviewed by @vtjnash. --------- Co-authored-by: Jameson Nash <[email protected]> (cherry picked from commit ffc96bc)
(cherry picked from commit 99fd5d9)
…ce (JuliaLang#57226) (cherry picked from commit ca7cf30)
This is the final PR in the binding partitions series (modulo bugs and tweaks), i.e. it closes JuliaLang#54654 and thus closes JuliaLang#40399, which was the original design sketch. This thus activates the full designed semantics for binding partitions, in particular allowing safe replacement of const bindings. It in particular allows struct redefinitions. This thus closes timholy/Revise.jl#18 and also closes JuliaLang#38584. The biggest semantic change here is probably that this gets rid of the notion of "resolvedness" of a binding. Previously, a lot of the behavior of our implementation depended on when bindings were "resolved", which could happen at basically an arbitrary point (in the compiler, in REPL completion, in a different thread), making a lot of the semantics around bindings ill- or at least implementation-defined. There are several related issues in the bugtracker, so this closes JuliaLang#14055 closes JuliaLang#44604 closes JuliaLang#46354 closes JuliaLang#30277 It is also the last step to close JuliaLang#24569. It also supports bindings for undef->defined transitions and thus closes JuliaLang#53958 closes JuliaLang#54733 - however, this is not activated yet for performance reasons and may need some further optimization. Since resolvedness no longer exists, we need to replace it with some hopefully more well-defined semantics. I will describe the semantics below, but before I do I will make two notes: 1. There are a number of cases where these semantics will behave slightly differently than the old semantics absent some other task going around resolving random bindings. 2. The new behavior (except for the replacement stuff) was generally permissible under the old semantics if the bindings happened to be resolved at the right time. With all that said, there are essentially three "strengths" of bindings: 1. Implicit Bindings: Anything implicitly obtained from `using Mod`, "no binding", plus slightly more exotic corner cases around conflicts 2. Weakly declared bindings: Declared using `global sym` and nothing else 3. Strongly declared bindings: Declared using `global sym::T`, `const sym=val`, `import Mod: sym`, `using Mod: sym` or as an implicit strong global declaration in `sym=val`, where `sym` is known to be global (either by being at toplevle or as `global sym=val` inside a function). In general, you always allowed to syntactically replace a weaker binding by a stronger one (although the runtime permits arbitrary binding deletion now, this is just a syntactic constraint to catch errors). Second, any implicit binding can be replaced by other implicit bindings as the result of changing the `using`'ed module. And lastly, any constants may be replaced by any other constants (irrespective of type). We do not currently allow replacing globals, but may consider changing that in 1.13. This is mostly how things used to work, as well in the absence of any stray external binding resolutions. The most prominent difference is probably this one: ``` set_foo!() = global foo = 1 ``` In the above terminology, this now always declares a "strongly declared binding", whereas before it declared a "weakly declared binding" that would become strongly declared on first write to the global (unless of course somebody had created a different strongly declared global in the meantime). To see the difference, this is now disallowed: ``` julia> set_foo!() = global foo = 1 set_foo! (generic function with 1 method) julia> const foo = 1 ERROR: cannot declare Main.foo constant; it was already declared global Stacktrace: [1] top-level scope @ REPL[2]:1 ``` Before it would depend on the order of binding resolution (although it just crashes on current master for some reason - whoops, probably my fault). Another major change is the ambiguousness of imports. In: ``` module M1; export x; x=1; end module M2; export x; x=2; end using .M1, .M2 ``` the binding `Main.x` is now always ambiguous and will throw on access. Before which binding you get, would depend on resolution order. To choose one, use an explicit import (which was the behavior you would previously get if neither binding was resolved before both imports). (cherry picked from commit 888cf03)
…#57273) Fixes JuliaLang#56864. Ref https://github.com/JuliaLang/julia/blob/99fd5d9a92190e826bc462d5739e7be948a3bf44/stdlib/REPL/src/LineEdit.jl#L504 --------- Co-authored-by: Ian Butterworth <[email protected]> (cherry picked from commit f0446c6)
- Update `JuliaSyntax.jl` to v1.0.1 - Revert workaround changes, use original test case `n37134` Fix JuliaLang#57223 (cherry picked from commit 97c920d)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
1cff7d7 to
1e6e20d
Compare
Addresses review comment in JuliaLang#57212 (comment). The key is that the hand-off of responsibility for verification between the loading code and the ordinary backedge mechanism happens under the world counter lock to ensure synchronization. (cherry picked from commit 34aceb5)
Updating mmtk-julia version to include mmtk/mmtk-julia#228 and fix the MMTk CI. I've also changed the allocation profiler tests to skip all tests instead of just a few since I've seen some spurious errors - they should all be related though, we need to make sure the profiler accounts for fastpath allocation (see JuliaLang#57103) This should fix JuliaLang#57306. (cherry picked from commit 72f8a10)
Similar to JuliaLang#57229, this commit ensures that `Compiler.finish!` properly synchronizes the operations to set `max_world` for cached `CodeInstance`s by holding the world counter lock. Previously, `Compiler.finish!` relied on a narrow timing window to avoid race conditions, which was not a robust approach in a concurrent execution environment. This change ensures that `Compiler.finish!` holds the appropriate lock (via `jl_promote_ci_to_current`). (cherry picked from commit 4ebb50b)
(cherry picked from commit 132057c)
Fixes JuliaLang#54560 (comment) (cherry picked from commit 79e98e3)
This adds a warning for the auto-import of types cases (JuliaLang#25744) that we have long considered a bit of a bug, but didn't want to change because it is too breaking. The reason to do it now is that the binding rework has made this case more problematic (see JuliaLang#57290). To summarize, the question is what happens when the compiler sees `f(x) = ...` and `f` is currently and implicitly imported binding. There are two options: 1. We add a method to the generic function referred to by `f`, or 2. We create a new generic function `f` in the current module. Historically, case 1 has the additional complication that this error'd unless `f` is a type. It is my impression that a lot of existing code did not have a particularly good understanding of the resolved-ness dependence of this behavior. However, because case 1 errors for generic functions, it appears that existing code generally expects case 2. On the other hand, for types, there is existing code in both directions (JuliaLang#57290 is an example of case 2; see JuliaLang#57302 for examples of case 1). That said, case 1 is more common (because types tend to be resolved because they're used in signatures at toplevel). Thus, to retain compatibility, the current behavior on master (where resolvedness is no longer available) is that we always choose case 2 for functions and case 1 for types. This inconsistency is unfortunate, but I tried resolving this in either way (making all situations case 1 or all case 2) and the result was too breaking. Nevertheless, it is problematic that there is existing code that expects case 2 beavior for types and we should help users to know what the correct way to fix it is. The proposed resolution is thus: 1. Retain case 1 behavior for types 2. Make it a warning to use, encouraging people to explicitly import, since we generally consider the JuliaLang#25744 case a bug. Example: ``` julia> module Foo String(i::Int) = i end WARNING: Type Core.String was auto-`import`ed in `Foo`. NOTE: This behavior is deprecated and may change in future Julia versions. NOTE: This behavior may have differed in Julia versions prior to 1.12 depending on binding resolution. Hint: To retain the current behavior, add an explicit `import Core: String` in Foo. Hint: To create a new generic function of the same name use `function String end`. Main.Foo ``` (cherry picked from commit 8c62f42)
introduced in JuliaLang#56144 --------- Co-authored-by: Mosè Giordano <[email protected]> (cherry picked from commit dd13878)
…uliaLang#57426) An upcoming optimization will skip most binding validation if no binding replacement has taken place in (sysimage, pkgimage) modules. However, as a special case, we would like to treat `Main` as a non-sysimage module because the addition of new bindings in `Main` is common and we would like this to not ruin the optimization. To make this legal, we have to prohibit `import`ing or `using` any `Main` bindings in pkgimages. I don't think anybody actually does this, particularly, since `Main` is not considered loading during precompile (so you have to use the main binding via (Core|Base|).Main), and I can't think of any good semantic reason to want to do this, but regardless, it does add additional restrictions to `using`/`import`, so I wanted to break it out into its own PR. (cherry picked from commit 726c816)
…dules (JuliaLang#57433) This implements the optimization proposed in JuliaLang#57426 by keeping track of whether any bindings were replaced in image modules (excluding `Main` as facilitated by JuliaLang#57426). In addition, we augment serialization to keep track of whether a method body contains any GlobalRefs that point to a loaded (system or package) image. If both of these flags are true, we can skip scanning the body of the method, since we know that we neither need to add any additional backedges nor were any of the referenced bindings invalidated. The performance impact on end-to-end load time is small, but measurable. Overall `@time using ModelingToolkit` consistently improves about 5% using this PR. However, I should note that using time is still about 40% slower than 1.11. This is not necessarily an Apples-to-Apples comparison as there were substantial other changes on 1.12 (as well as current load-time-tunings targeting older versions), but I wanted to put the number context. (cherry picked from commit f6e2b98)
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
02e0f68 to
35024c5
Compare
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
35024c5 to
8756d93
Compare
I was playing with strengthening the semantics around JuliaLang#57290. However, the particular change I was trying turned out too breaking (I may try a weaker version of it). Still, there were a number of good changes found in two categories: 1. We should explicitly import types when defining constructors. This has been allowed for a long time, but we may want to consider removing it, especially given the new binding semantics which make it more confusing as in JuliaLang#57290. 2. There were a couple of places where I don't think it was intended for generic functions in question not to extend Base. (cherry picked from commit 778e079)
…JuliaLang#57445) (cherry picked from commit d43a5ad)
Other backends (in this case NVPTX) require that `invariant.load` metadata is maintained to generate non-coherent loads. Currently, we unconditionally strip that metadata from all loads, since our other uses of it may have become invalid. x-ref: llvm/llvm-project#112834 JuliaGPU/CUDA.jl#2531 --------- Co-authored-by: Gabriel Baraldi <[email protected]> (cherry picked from commit 29da86b)
…ng#57453) This reverts a portion of commit 50833c8. This algorithm is not able to handle simple cases where there is any internal padding, such as the example of: ``` struct LotsBytes a::Int8 b::NTuple{256,Int} c::Int end ``` Unfortunately fixing it is a bit of a large project right now, so reverting now to fix correctness while working on that. Fixes JuliaLang#55513 (indirectly, by removing broken code) Maybe reopens JuliaLang#54109, although the latency issue it proposes to fix doesn't occur on master even with this revert (just the mediocre looking IR result output returns) (cherry picked from commit a65c2cf)
…ang#57389) also add nsw/nuw flags whenever possible. (cherry picked from commit b9a8d46)
…liaLang#57467) Fix JuliaLang#57316 (cherry picked from commit a70818c)
…uliaLang#57471) The buffer may end up reallocated by the additional writes performed to it in this function. (cherry picked from commit 0fb5fa0)
Fixes JuliaLang#56904. The associated PR (JuliaLang#55876) compiles a finally block, then compiles a renumbered version of it. This works if `compile` doesn't mutate its input, but in reality, lambda bodies were being `set!` when linearized. The "invalid syntax" error was a result of attempting to linearize a lambda twice. (cherry picked from commit 414aca2)
…al (JuliaLang#57447) The `_DECLARED` partition kind used to be considered `guard`, but we now consider it equivalent to an Any-typed `_GLOBAL` (but with weaker redefinition properties). That said, its `->restriction` is NULL, so add it to the list of bindings that should return `nothing` here (and thus `Any` from the bulitin) to fix JuliaLang#57446. (cherry picked from commit 0163991)
This reverts commit f15d417.
…s from packages succeed. TODO: remove this once alpha/beta is released
Prevent transparent huge pages (THP) overallocating pysical memory. Co-authored-by: Adnan Alhomssi <[email protected]>
Prepend `[signal (X) ]thread (Y) ` to each backtrace line that is displayed. Co-authored-by: Diogo Netto <[email protected]>
Also show the signal number when we have it.
DelveCI
force-pushed
the
v1.12.0-DEV+RAI
branch
from
8756d93 to
b0a0ed0
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See also RAICode PR https://github.com/RelationalAI/raicode/pull/22602/commits