bakerydemo GOLD benchmark

Reference Wagtail site implementation for energy usage benchmarking.

Project setup

This project is based upon Wagtail’s official bakerydemo, with tweaks to make it more suitable for local energy consumption benchmarking.

Differences with vanilla bakerydemo

With the overall goal of making the project more representative of a well-maintained real-world production site:

• Full Debian base image rather than -slim
• Gunicorn instead of uWSGI
• Python 3.11 instead of 3.9
• Latest WhiteNoise 6.2.0 for Python 3.11 compatibility
• Up-to-date production-ready Django configuration

Differences with real-world production sites

With the overall goal of making it simpler to get the project running locally with Docker as the only requirement:

• Running locally rather than in a data center
• No reverse-proxy CDN caching requests
• Serving media files with Django (bad practice) rather than dedicated object storage service (S3, Google Cloud Storage, etc)
• Running over HTTP rather than HTTPS

Getting started

Requirement: Docker.

# Download a copy of https://github.com/thibaudcolas/bakerydemo-gold-benchmark.
cd bakerydemo-gold-benchmark
docker compose build app
docker compose up app
# The site is up and running but still needs its database initialised.
docker compose exec app ./manage.py migrate
docker compose exec app ./manage.py load_initial_data

The site is now up and running with its demo content, but still needs cache warming to be representative of a real-world production site. In particular, Wagtail generates optimised images on the fly the first time an image is requested.

Here is a sample wget command to warm up the cache:

wget --recursive --spider --no-directories http://localhost:8000/ -o warmup.log

From there, the site can be accessed at http://localhost:8000/.

Benchmark scenarios

Our scenarios cover different aspects of a site, representing real-world user journeys, all spanning multiple pages:

• homepage-landing.js: The simplest user journey. Landing on the homepage, navigating to the bread listing page, and finally to a bread detail page.
• blog-filtering.js: Another simple user journey. Landing on the blogs listing, filtering it by tag, and arriving on a blog page.
• contact-us.js: Successful submission of a simple Wagtail form.
• search.js: Usage of a search form. From the homepage, arriving on the search results for "bread", and opening one result.
• admin.js: A simple journey through the Wagtail admin. Logging in, editing a blog page, and checking the results in the live preview.

Note although the navigation through the sequence of pages represents a real-world journey, the scenarios’ duration isn’t representative of real usage. Average times spent across different page types are:

• Homepage: 2min
• Listing page: 1min
• Search results: 30s
• Blog post: 1min30s to 8min

Puppeteer scenarios

Puppeteer test scripts are in benchmark/puppeteer. To run those scenarios (requirement: Node 18),

cd benchmark/puppeteer
npm install
# Then run each scenario with `node`:
node homepage-landing.js

Playwright scenarios for Greenframe

See Playwright – Migrating from Puppeteer for differences between the two APIs. Those are the same scenarios, but written with Playwright for compatibility with Greenframe. First install Greenframe, then use the following test commands:

greenframe analyze http://localhost:8000/ homepage-landing.js --containers="bakerydemo-gold-benchmark-app-1" --databaseContainers="bakerydemo-gold-benchmark-db-1,bakerydemo-gold-benchmark-redis-1"
greenframe analyze http://localhost:8000/ search.js --containers="bakerydemo-gold-benchmark-app-1" --databaseContainers="bakerydemo-gold-benchmark-db-1,bakerydemo-gold-benchmark-redis-1"
greenframe analyze http://localhost:8000/ blog-filtering.js  --containers="bakerydemo-gold-benchmark-app-1" --databaseContainers="bakerydemo-gold-benchmark-db-1,bakerydemo-gold-benchmark-redis-1"
greenframe analyze http://localhost:8000/ contact-us.js --containers="bakerydemo-gold-benchmark-app-1" --databaseContainers="bakerydemo-gold-benchmark-db-1,bakerydemo-gold-benchmark-redis-1"
greenframe analyze http://localhost:8000/ admin.js --containers="bakerydemo-gold-benchmark-app-1" --databaseContainers="bakerydemo-gold-benchmark-db-1,bakerydemo-gold-benchmark-redis-1"0.

Or run all scenarios at once, based on the configuration in .greenframe.yml:

greenframe analyze

Here is what the result from a successful run looks like:

[…]
✅ homepage-landing completed
The estimated footprint is 0.093 g eq. co2 ± 6.1% (0.21 Wh).

✅ search completed
The estimated footprint is 0.041 g eq. co2 ± 4.9% (0.092 Wh).

✅ blog-filtering completed
The estimated footprint is 0.063 g eq. co2 ± 1.8% (0.142 Wh).

✅ contact-us completed
The estimated footprint is 0.035 g eq. co2 ± 6% (0.078 Wh).

✅ admin completed
The estimated footprint is 0.155 g eq. co2 ± 8.8% (0.352 Wh).

Static site setup

It’s interesting to compare the performance of Django and Wagtail to that of pre-generated HTML files. First, generate the site:

wget --mirror http://localhost:8000/
mv localhost:8000 static-bakerydemo
mv static-bakerydemo/static/wagtailfontawesome/fonts/fontawesome-webfont.woff2\?v=4.7.0 static-bakerydemo/static/wagtailfontawesome/fonts/fontawesome-webfont.woff2
mv bakerydemo/static/img/bread-favicon.ico static-bakerydemo/favicon.ico

Then, serve it with nginx:

docker compose up static_app

From there, the static site can be accessed at http://localhost:8001/.

The Greenframe test suite can run over this site as well with:

greenframe analyze --configFile .greenframe.static.yml

Sample results:

[…]
✅ homepage-landing completed
The estimated footprint is 0.038 g eq. co2 ± 3.2% (0.085 Wh).

✅ blog-filtering completed
The estimated footprint is 0.038 g eq. co2 ± 15.3% (0.085 Wh).