Populate caches first

[pgrange]

We restructure the C.I. to have one job that will build and populate all the caches for the next jobs to run.

It shall improve our resources usage in C.I. as we ensure that most of the stuff are built only once.

It might improve our total job execution although... well it depends on so many things that it is not clear. But it will bring us more determinism in job execution that can pave the way to improvement.

The next big thing eating time in the job is downloading all the nix artifacts. It gives us some leads for future improvements in C.I. runtime.

[github-actions]

Test Results

319 tests  ±0   313 ✔️ ±0   18m 53s ⏱️ -39s
108 suites ±0       6 💤 ±0 
    6 files   ±0       0 ❌ ±0 

Results for commit 30f5c21. ± Comparison against base commit a563fe5.

♻️ This comment has been updated with latest results.

[github-actions]

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2023-05-17 09:08:45.168136273 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	e5cc20df4a2216e706b3d00b59fbb15a7cf12dbd28d271d4a8cf6d04	4336
νCommit	47c102d5f95a0648b4065f2b8bff59d3e34536a82ee7b0d42df73123	2124
νHead	9fe3a5c4d826f9475368e1e24c15bf22f4df19893cce2689d3c0564a	9492
μHead	7bec671467e923281c92e94257931913be106e217270a7b97076cb9b*	4148

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4746	12.09	4.71	0.49
2	4946	17.32	6.79	0.56
3	5152	18.79	7.31	0.58
5	5559	24.17	9.36	0.66
10	6590	34.25	13.10	0.81
37	12123	97.64	36.99	1.74

Cost of Commit Transaction

Currently only one UTxO per commit allowed (this is about to change soon)

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	603	15.29	5.93	0.34

Cost of CollectCom Transaction

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	816	27.63	10.77	0.49
2	113	1146	43.15	16.96	0.67
3	170	1469	60.91	24.10	0.88
4	227	1783	81.71	32.48	1.12

Cost of Close Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	640	18.88	8.43	0.39
2	805	20.28	9.68	0.42
3	969	21.67	10.94	0.45
5	1300	24.16	13.33	0.50
10	2124	31.43	19.73	0.64
50	5424	65.03	29.46	1.13

Cost of Contest Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	675	24.35	10.47	0.45
2	867	26.51	12.19	0.49
3	1006	28.20	13.38	0.52
5	1344	31.20	15.95	0.58
10	2160	40.20	22.97	0.74
45	7938	99.74	70.74	1.82

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4929	23.17	9.81	0.63
2	5180	36.25	15.44	0.78
3	5218	44.31	18.59	0.87
4	5826	72.64	31.38	1.22
5	6148	95.54	41.39	1.49

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4761	8.67	3.57	0.46
5	1	57	4801	10.06	4.39	0.47
5	5	285	4940	15.64	7.69	0.55
5	10	569	5125	22.61	11.82	0.64
5	20	1137	5485	36.57	20.08	0.83
5	30	1705	5841	50.53	28.34	1.02
5	40	2278	6200	64.49	36.60	1.21
5	50	2844	6561	78.46	44.87	1.40
5	65	3698	7095	99.42	57.28	1.68

[ch1bo]

Good idea, let's try this. Some changes required though

[ch1bo]

I think we can drop this step as build errors would already appear in the previous job and cabal test would also build anything that was not yet built (e.g. benchmarks are not built when we do cabal build all)

[pgrange]

I'd like to keep it that way to limit inter jobs dependencies.

This P.R. proposes an improvement by just preparing the caches before running the existing job. So if we decide to drop the warm up the caches job this job still works and make sense.

/cc @ch1bo

[ch1bo]

Do you want to continue this journey? I think it does not provide any benefit and instead we should focus on #867. WDYT?

[pgrange]

It is providing some improvements although not as much as we hope to get from #867, I agree. @ch1bo I'd like to see #867 work first before considering throwing out this one.

[pgrange]

Canceled in favor of #867