ci(port): various ccache and matrix improvements

[scarf005]

Summary

SUMMARY: Build "Ported various ccache and build improvements from DDA"

Purpose of change

speed up test loop by porting various improvements from DDA

Describe the solution

blamed and copied relevant lines. individual referenced commits and PRs are written in commit message.

Commits are cherry-picked from the works of: @BrettDong @kevingranade @irwiss @jbytheway

Testing

I'd be surprised if this doesn't cause mayhem.

Additional context

tests marked as [slow] were disabled to prevent build failure

[olanti-p]

2 failing tests on GCC Curses build, the 1st one seems legit, 2nd may be RNG-dependent

([slow] ~starting_items)=> Aiming at a target partially covered by a wall
([slow] ~starting_items)=> -------------------------------------------------------------------------------
([slow] ~starting_items)=> ../tests/ranged_aiming_test.cpp:226
([slow] ~starting_items)=> ...............................................................................
([slow] ~starting_items)=> 
([slow] ~starting_items)=> ../tests/ranged_aiming_test.cpp:266: FAILED:
([slow] ~starting_items)=>   CHECK( unseen == 0 )
([slow] ~starting_items)=> with expansion:
([slow] ~starting_items)=>   2500 (0x9c4) == 0
([slow] ~starting_items)=> with messages:
([slow] ~starting_items)=>   unseen := 2500 (0x9c4)
([slow] ~starting_items)=>   failed := {  }

([slow] ~starting_items)=> grenade_lethality
([slow] ~starting_items)=> -------------------------------------------------------------------------------
([slow] ~starting_items)=> ../tests/explosion_balance_test.cpp:151
([slow] ~starting_items)=> ...............................................................................
([slow] ~starting_items)=> 
([slow] ~starting_items)=> ../tests/explosion_balance_test.cpp:103: FAILED:
([slow] ~starting_items)=>   CHECK( victims.avg() == Approx( lethality ).margin( margin ) )
([slow] ~starting_items)=> with expansion:
([slow] ~starting_items)=>   0.0 == Approx( 0.400000006 )
([slow] ~starting_items)=> with messages:
([slow] ~starting_items)=>   margin := 0.06f
([slow] ~starting_items)=>   grenade_act
([slow] ~starting_items)=>   range 15
([slow] ~starting_items)=>   96 survivors out of 96 targets.
([slow] ~starting_items)=>   (41,41,0) 80, (19,41,0) 80, (19,19,0) 80, (41,19,0) 80, (42,20,0) 80, (42,21,
([slow] ~starting_items)=>   0) 80, (42,39,0) 80, (42,40,0) 80, (40,42,0) 80, (39,42,0) 80, (21,42,0) 80,
([slow] ~starting_items)=>   (20,42,0) 80, (18,40,0) 80, (18,39,0) 80, (18,21,0) 80, (18,20,0) 80, (20,18,
([slow] ~starting_items)=>   0) 80, (21,18,0) 80, (39,18,0) 80, (40,18,0) 80, (43,21,0) 80, (43,22,0) 80,
([slow] ~starting_items)=>   (43,38,0) 80, (43,39,0) 80, (39,43,0) 80, (38,43,0) 80, (22,43,0) 80, (21,43,
([slow] ~starting_items)=>   0) 80, (17,39,0) 80, (17,38,0) 80, (17,22,0) 80, (17,21,0) 80, (21,17,0) 80,
([slow] ~starting_items)=>   (22,17,0) 80, (38,17,0) 80, (39,17,0) 80, (44,23,0) 80, (44,24,0) 80, (44,36,
([slow] ~starting_items)=>   0) 80, (44,37,0) 80, (37,44,0) 80, (36,44,0) 80, (24,44,0) 80, (23,44,0) 80,
([slow] ~starting_items)=>   (16,37,0) 80, (16,36,0) 80, (16,24,0) 80, (16,23,0) 80, (23,16,0) 80, (24,16,
([slow] ~starting_items)=>   0) 80, (36,16,0) 80, (37,16,0) 80, (45,25,0) 80, (45,26,0) 80, (45,27,0) 80,
([slow] ~starting_items)=>   (45,28,0) 80, (45,29,0) 80, (45,30,0) 80, (45,31,0) 80, (45,32,0) 80, (45,33,
([slow] ~starting_items)=>   0) 80, (45,34,0) 80, (45,35,0) 80, (35,45,0) 80, (34,45,0) 80, (33,45,0) 80,
([slow] ~starting_items)=>   (32,45,0) 80, (31,45,0) 80, (30,45,0) 80, (29,45,0) 80, (28,45,0) 80, (27,45,
([slow] ~starting_items)=>   0) 80, (26,45,0) 80, (25,45,0) 80, (15,35,0) 80, (15,34,0) 80, (15,33,0) 80,
([slow] ~starting_items)=>   (15,32,0) 80, (15,31,0) 80, (15,30,0) 80, (15,29,0) 80, (15,28,0) 80, (15,27,
([slow] ~starting_items)=>   0) 80, (15,26,0) 80, (15,25,0) 80, (25,15,0) 80, (26,15,0) 80, (27,15,0) 80,
([slow] ~starting_items)=>   (28,15,0) 80, (29,15,0) 80, (30,15,0) 80, (31,15,0) 80, (32,15,0) 80, (33,15,
([slow] ~starting_items)=>   0) 80, (34,15,0) 80, (35,15,0) 80, 
([slow] ~starting_items)=>   Wounded survivors: 0
([slow] ~starting_items)=>   average hp of survivors: 80

[scarf005]

2 failing tests on GCC Curses build, the 1st one seems legit, 2nd may be RNG-dependent

are they caused by #3258 ?

[olanti-p]

These tests are marked [slow], and this PR enables them. They've most likely been broken for some time.

[scarf005]

should i disable slow tests and re-introduce them in later PR? i think fixing CCache should be done quickly

[olanti-p]

That could work.

[olanti-p]

I can confidently say I don't understand much of it, but here's matrix.yml

[scarf005]

applied all of the suggestions, let's hope this works...

[olanti-p]

I completely forgot, but Catch2 has a special tag [!mayfail] we could use to acknowledge that the test sometimes fails.
https://github.com/catchorg/Catch2/blob/v2.x/docs/test-cases-and-sections.md#special-tags
Adding it to failing slow tests may be a better solution that outright disabling all of them.

[olanti-p]

I don't understand Mac parts, but I didn't see anything suspicious with them.

[scarf005]

Adding it to failing slow tests may be a better solution that outright disabling all of them.

makes sense.

[scarf005]

• marked ranged aiming tests as flaky (related: fix: crash in ranged accuracy test related to dead NPC cleanup #3309)
• marked lab generation test as flaky
• rebalanced grenade lethality test as its values were for DDA

[scarf005]

@olanti-p do you think this is ready?

[scarf005]

@olanti-p i think it's ready for reviewing.

[scarf005]

wait, errrr

of course, yes, the mac builds.

[olanti-p]

MacOS build failed on this PR, but it's legit related failure this time.

Makefile:536: *** "SDL2 framework not found".  Stop.

[scarf005]

i haven't changed anything on macos related code, not really sure what's happening.

[olanti-p]

It's complaining about missing dependency, so obviously something here changed how dependencies are detected for the build job.

[scarf005]

failing item display test will be fixed in #3407

[olanti-p]

The caching didn't work for a couple builds when I tested it, and a couple other builds received only partial speedup - don't know what's up with that. Either way, some caching is better than no caching, and a couple builds went down to sub 3 minutes, which is definitely an improvement.