Benchmark 0.5 vs 0.6 [do not merge]

[KristofferC]

Thought it would be a good time to start checking the performance vs 0.5. Easiest to have this in a branch since AFAIK there is no way to run a benchmark between two commits from an issue.

vs 0.5.1:

@nanosoldier runbenchmarks(ALL, vs = "@6445c82d0060dbe82b88436f0f4371a4ee64d918")

Note to self, but [ci skip] on the commit for the rebases to come.

[nanosoldier]

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here. cc @jrevels

[KristofferC]

Not bad.

Summary of stuff that got slower:

Arrays:

• ["array","index",("sumcolon","1.0:1.0:100000.0")]
• ["array","index",("sumeach_view","BitArray{2}")]
• ["array","index",("sumlinear_view","BitArray{2}")]
• ["array","index",("sumrange","1.0:1.0:100000.0")]
• Some of the concatenations.

Date accessors in general

For example: ["dates","accessor","millisecond"]

Fixed by #20853?

Factorizations

For example: ["linalg","factorization",("eig","Matrix",256)]

div with BigNumbers

For example: ["scalar","arithmetic",("div","BigFloat","Complex{Float32}")]

Maybe some new promotion paths that leads to multiple allocations of big numbers?

Yes, changed from:

/(a::Real, z::Complex) = a*inv(z)

to

/{R<:Real,S<:Complex}(a::R, z::S) = (T = promote_type(R,S); a*inv(T(z)))

Parallell remotecall

For example ["parallel","remotecall",("identity",2)]

Sorting in general

For example ["sort","insertionsort",("sort forwards","ascending")]

I think this happened when we upgraded LLVM

Sparse indexing

For example ["sparse","index",("spmat","col","array",10)]

Other

• ["problem","laplacian","laplace_sparse_matvec"]
• ["misc","bitshift",("UInt32","UInt32")]
• ["shootout","revcomp"]

[jebej]

Is it possible to test against 0.4 as well?

[KristofferC]

Should be, let's try it!

[KristofferC]

vs v0.4.7:

@nanosoldier runbenchmarks(ALL, vs = "@ae26b25d43317d7dd3ca05f60b70677aab9c0e08")

[nanosoldier]

Something went wrong when running your job:

NanosoldierError: failed to run benchmarks against comparison commit: failed process: Process(`sudo cset shield -e su nanosoldier -- -c ./benchscript.sh`, ProcessExited(1)) [1]

Logs and partial data can be found here
cc @jrevels

[tkelman]

probably need an older copy of the saved data for 0.4

[KristofferC]

Eeh what? I just rebased this branch and force pushed to it...?

[yuyichao]

There's no more commit to merge on this anymore.