Vue.js SPA displaying benchmark results - e.g. website 2.0

[Akirathan]

Current solution (bad)

The current website with plots of all the benchmarks is a static website generated from a Jinja template by website_regen.py script hosted with GitHub pages on https://enso-org.github.io/engine-benchmark-results/.

This has many disadvantages:

• Hard to maintain. Especially with some Javascript code in the HTML template.
• The resulting web pages are huge and very slow to load, because they contain all the data inside Javascript embedded in the HTML.
• The Google charts JS library used by the template is ugly and difficult to work with and with insufficient documentation.
• It is very difficult to add any dynamic behavior like loading data on demand or comparing results of two branches.

Motivation

There are requests for new features in the current website:

• Plot also Enterprise edition benchmarks along with Community edition.
• Compare results from a custom branch to the results of the develop branch.
  • Currently handled only by the bench_download.py cmd line script.
• Display standard deviation.

It is easier to implement the current features along with the requested ones from scratch using VueJS than to add them to the current template solution.

VueJS proposal

Let's implement a single page application with VueJS and use some more responsive, sophisticated and better documented plotting library, like ChartJS. The data will be fetched from the cache directory in the same repo. We only need some index.json file that maps names of the files to their timestamps. There is no need for any sophisticated backend or a database yet. Just build static single page application with VueJS and host it on GitHub pages. Access to the cache directory in the same repo should be straightforward. The application won't be needed to regenerate on a daily basis, just when new features are added.

A prototype is being developed in https://github.com/enso-org/engine-benchmark-results/tree/vue

Benchmark data upload

The data for the benchmarks are uploaded daily by the Benchmarks Upload GH Action into the directory https://github.com/enso-org/engine-benchmark-results/tree/main/cache as JSON files. Note that this task does not need any change to the data uploading functionality.

Related

• Improve Engine benchmark reporting #5714
• Ensure that bench downloading script downloads artifacts from correct job #9355

There is a branch with working (to some extent) UI. Let's finish this and make the new UI visible. These are the minimal steps that have to be done:

Tasks

Preview Give feedback

1. modify the main benchmarking page to contain three links (not just two). The new link will point to experimental Vue.js based SPA.
2. Merge vue branch into main.
3. Generate static HTML pages from Vue with something like vite generate static. Put them into view subdirectory. Ensure that our manually maintained index.html points to the index as generated by vite.
4. Generate index.json by executing construct_index.py by an CI job
5. verify the SPAs display some data from the generated indexes

No need for any further automation right now. Being able to see latest benchmark results in the Vue.js SPA (preferably also from other than develop branch) is good enough starting point.

[Akirathan]

Priority is high because the results of Oracle GraalVM (Enterprise Edition) benchmarks, implemented on 2024-03-12 in #9322 are stored only as GH artifacts so far that are dropped after 3 months without any recovery or backup. The initial results are already dropped. Let's make sure they are stored permanently in engine-benchmark-results repo.

[enso-bot]

Pavel Marek reports a new STANDUP for the provided date (2024-06-18):

Progress: - Prototype of a new benchmark website in VueJS. It should be finished by 2024-07-11.

[JaroslavTulach]

Let's finish integration of this new, cool, Vue.js based SPA!

Added three essential tasks to finish this issue.