Performance regressions on blake

[jewatkins]

There's a couple there we need to investigate: https://sandialabs.github.io/ali-perf-data/ali/blake_nightly_data/Ali_PerfTestsBlake_02_01_2023.html

[bartgol]

@jewatkins do you think it's doable/interesting to add the commit first line to the overlay window that appears when scrolling over a point in the perf history graph? I know we can take the printed sha, and go see what it was, but it feels like it would me much more immediate to understand (to some extent) what happened at any point in time...

To be clear I'm thinking of upgrading the overlay window

Date: 2022-11-02
Albany commit: f6db667
Trilinos commit: 409aaf2
Mean: ...
Ratio: ...

to something like

Date: 2022-11-02
Albany commit: f6db667
    Correcting path in script.
Trilinos commit: 409aaf2
    Merge Pull Request #11210 from gsjaardema/Trilinos/SEACAS-Nemesis-cmake-fix
Mean: ...
Ratio: ...

[jewatkins]

Yeah we could probably do that and it makes sense. Maybe even a link would be useful. The commit ids are read/stored from simulation output but the python script that reads/stores the data could probably query the commit id and also store the first line description. I'll add it as an issue.

[ikalash]

Do you guys know why there are not run performance tests on Blake? https://sems-cdash-son.sandia.gov/cdash/viewTest.php?onlynotrun&buildid=45031 . The error is failed dependency: https://sems-cdash-son.sandia.gov/cdash/test/3264091 .

[bartgol]

It looks like the tests that are creating the populated mesh are crashing. E.g. this

[bartgol]

Seems like an MPI error. It could be a temporary issue of the testbed.

[mcarlson801]

Hmm, this same error showed up before (Issue #904). I wasn't able to replicate it when it showed up originally and I think it went away the next day. Seems like some sort of system issue.

[ikalash]

Hmmm could be a system issue. I will keep an eye on it. Thanks!

[jewatkins]

this is a different issue. I will create a new issue for the failing tests that need to be reconfigured.