orchestrate_diagnostics has lots of inference failures
#82
Labels
Comments
|
I noticed that one of the failures in https://buildkite.com/clima/climaatmos-ci/builds/20805#01922a42-bcbe-49b0-ae2d-2247cb3b2a49 was an inference failure in ClimaDiagnostics.jl/src/clima_diagnostics.jl Lines 252 to 253 in c1cc7c3 diagnostics_handler.
|
Closed
|
I re-ran this with some package updates (and with more type info) and it seems like the main issue is with the So, we may be able to fix this when we move to using LazyBroadcast. I can't imagine that we don't have the same issue with the function, too. |
|
This is similar to the manual dispatch example: using BenchmarkTools
func(x::Int) = 1;
func(x::Float64) = 1;
# The type of the arrays is `Vector{Any}`, so that the compiler cannot choose the
# required method for `func` at compile time.
const x_int = Any[1 for i in 1:1000];
const x_float = Any[1.0 for i in 1:1000];
const x_mixed = Any[iseven(i) ? 1 : 1.0 for i in 1:1000];
function ex_union_split(x)
for y in x
if y isa Int
func(y)
elseif y isa Float64
func(y)
end
end
end
function ex_runtime_dispatch(x)
for y in x
func(y)
end
end
function main(x_int,x_float,x_mixed)
print("Manual union split with array of Int: "); @btime ex_union_split(x_int)
print("Runtime dispatch with array of Int: "); @btime ex_runtime_dispatch(x_int)
print("Manual union split with array of Float64: "); @btime ex_union_split(x_float)
print("Runtime dispatch with array of Float64: "); @btime ex_runtime_dispatch(x_float)
print("Manual union split with mixed array: "); @btime ex_union_split(x_mixed)
print("Runtime dispatch with mixed array: "); @btime ex_runtime_dispatch(x_mixed)
end
main(x_int,x_float,x_mixed) |
|
A function-analogy might be: using BenchmarkTools
foo(x::Int) = 1;
bar(x::Float64) = 1;
# The type of the arrays is `Vector{Any}`, so that the compiler cannot choose the
# required method for `func` at compile time.
const x_int = Any[foo for i in 1:1000];
const x_float = Any[bar for i in 1:1000];
const x_mixed = Any[iseven(i) ? foo : bar for i in 1:1000];
function ex_union_split(x::X) where {X}
for f in x
ft = typeof(f)
if ft <: typeof(foo)
z = f(1)
elseif ft <: typeof(bar)
z = f(1.0)
end
end
end
function ex_runtime_dispatch(x)
for f in x
a = typeof(f) <: typeof(foo) ? 1 : 1.0
f(a)
end
end
# function main(x_int,x_float,x_mixed)
# ex_union_split(x_int)
# ex_runtime_dispatch(x_int)
# ex_union_split(x_float)
# ex_runtime_dispatch(x_float)
# ex_union_split(x_mixed)
# ex_runtime_dispatch(x_mixed)
# print("Manual union split with array of Int: "); @btime ex_union_split(x_int)
# print("Runtime dispatch with array of Int: "); @btime ex_runtime_dispatch(x_int)
# print("Manual union split with array of Float64: "); @btime ex_union_split(x_float)
# print("Runtime dispatch with array of Float64: "); @btime ex_runtime_dispatch(x_float)
# print("Manual union split with mixed array: "); @btime ex_union_split(x_mixed)
# print("Runtime dispatch with mixed array: "); @btime ex_runtime_dispatch(x_mixed)
# end
# main(x_int,x_float,x_mixed)
using JET
@test_opt ex_union_split(x_int)
@test_opt ex_union_split(x_float)
@test_opt ex_union_split(x_mixed)But this didn't seem to work, I had to put the function into a wrapper object for it to work: using BenchmarkTools
foo(x::Int) = 1;
bar(x::Float64) = 1;
struct FunctionWrapper{F}
f::F
end
@inline (fw::FunctionWrapper)(args...; kwargs...) = fw.f(args...; kwargs...)
# The type of the arrays is `Vector{Any}`, so that the compiler cannot choose the
# required method for `func` at compile time.
const x_int = Any[FunctionWrapper(foo) for i in 1:1000];
const x_float = Any[FunctionWrapper(bar) for i in 1:1000];
const x_mixed = Any[iseven(i) ? FunctionWrapper(foo) : FunctionWrapper(bar) for i in 1:1000];
function ex_union_split(x::X) where {X}
for f in x
if f isa FunctionWrapper{typeof(foo)}
z = f(1)
elseif f isa FunctionWrapper{typeof(bar)}
z = f(1.0)
end
end
end
function ex_runtime_dispatch(x)
for f in x
a = f isa FunctionWrapper{typeof(foo)} ? 1 : 1.0
f(a)
end
end
function main(x_int,x_float,x_mixed)
ex_union_split(x_int)
ex_runtime_dispatch(x_int)
ex_union_split(x_float)
ex_runtime_dispatch(x_float)
ex_union_split(x_mixed)
ex_runtime_dispatch(x_mixed)
print("Manual union split with array of Int: "); @btime ex_union_split(x_int)
print("Runtime dispatch with array of Int: "); @btime ex_runtime_dispatch(x_int)
print("Manual union split with array of Float64: "); @btime ex_union_split(x_float)
print("Runtime dispatch with array of Float64: "); @btime ex_runtime_dispatch(x_float)
print("Manual union split with mixed array: "); @btime ex_union_split(x_mixed)
print("Runtime dispatch with mixed array: "); @btime ex_runtime_dispatch(x_mixed)
end
main(x_int,x_float,x_mixed) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
orchestrate_diagnosticshas lots of inference failures, and I've noticed that compilation time is significantly reduced when diagnostics are disabled. This very well may be the reason.Found in https://buildkite.com/clima/climaatmos-ci/builds/20805#01922a42-bcbe-49b0-ae2d-2247cb3b2a49
The text was updated successfully, but these errors were encountered: