Feature/benchmarks

[mikemimik]

What / Why

We want benchmarking to run on the current state of the repo. This is done by sending a dispatch event to the https://github.com/npm/benchmark repository. It has a set of actions which will run all the benchmarked scenarios and then report back to the pull-request with some results. Those results are specifically from the ubuntu-latest/[email protected] matrix run.

References

• Epic: Add monitoring/benchmarking tools statusboard#3

[mikemimik]

hello world

[npm-deploy-user]

	angular-quickstart			app-large			app-medium			ember-quickstart			react-app
	prev	current	status	prev	current	status	prev	current	status	prev	current	status	prev	current	status
initial install	38.4s	37.9s	✅	39.6s	38.6s	✅	32.4s	32.9s	🛑	24.9s	25.6s	🛑	31.4s	30.8s	✅
repeat install	9s	8.2s	✅	8s	7.1s	✅	7.7s	7.9s	🛑	6.9s	6.3s	✅	8.4s	7.7s	✅
with warm cache	28.9s	28.9s	✅	32.6s	32.3s	✅	27.3s	28.7s	🛑	21.4s	20.6s	✅	29.4s	28.1s	✅
with node_modules	8.7s	7.9s	✅	7.3s	6.8s	✅	8.3s	7.2s	✅	7.4s	6.3s	✅	8.3s	7.5s	✅
with lockfile	31.8s	27.6s	✅	27.8s	29.3s	🛑	26.5s	26.2s	✅	18.4s	18.4s	✅	24.4s	25.9s	🛑
with warm cache and node_modules	9s	7.9s	✅	8.3s	6.8s	✅	7.7s	14s	🛑	8.9s	6.4s	✅	10.3s	7.7s	✅
with warm cache and lockfile	25.2s	22.7s	✅	24.9s	26.8s	🛑	24s	21.2s	✅	15.2s	15.5s	🛑	20.2s	19.9s	✅
with node_modules and lockfile	9.3s	9.1s	✅	8.6s	7.7s	✅	8.3s	7.6s	✅	7.5s	6.8s	✅	8.8s	8.3s	✅

[isaacs]

Note from @mikemimik in slack, posted here so I don't forget it:

Please don't be alarmed by the negative results. The "prev" results were a run of 6.13.0 from my local machine. Where the "current" results are what ran in the ubuntu-latest vm, running [email protected]. When this pull-request gets merged in, they will become the "latest" results. Which is what we want. All results moving forward will be output from the VM's not local machines.

[mikemimik]

Updated the latest.json in the https://github.com/npm/benchmarks repo. The above results are "prev" which is the HEAD commit of the latest branch run from ubuntu-latest/[email protected] against "current" which is a commit branched from 6.12.1 run from ubuntu-latest/[email protected]. Compared to the last set that existed this is much more reasonable. Some of these up's and down's can be attributed to network performance, or the metal the VM is running on.