Finalize assignments: Chapter 14. CMS

[rviscomi]

Section	Chapter	Author	Reviewers
III. Content Publishing	14. CMS	@amedina	@westonruter @mor10 @sirjonathan

Due date: To help us stay on schedule, please complete the action items in this issue by June 3.

To do:

• Assign subject matter expert (author)
• Assign peer reviewers
• Finalize metrics

Current list of metrics:

Section | Metric description

• What are the top CMSs
  -- There are studies and reports classifying CMSes according to market share
  -- The WordPress community commonly cites W3Techs
  -- It would be interesting to validate such claims with HTTPArchive/CrUX data
  -- That is: would the sample space represented by these datasets correlate to the reported market shares elsewhere?

• AMP adoption: number WordPress-powered pages using the AMP plugin for WordPress
  -- Version of the plugin
  -- Number of AMP pages using the different Template mode used (reader/classic, transitional/paired, native).
  -- Suggestion: WordPress.com enables AMP by default so it would be interesting to see how many sites have disabled it in addition to how many self-hosted sites have enabled it. Not sure if this is possible or not but would also help
  -- The AMP plugin for WordPress generates the following meta tag:
  <meta name="generator" content="AMP Plugin v1.1.2; mode=native">

• Coupled vs. Decoupled CMS use: Headless CMSes
  -- There is a “trend” of using some CMSes in headless mode; it would be interesting to capture the prevalence of such uses
  -- Measuring this is not easily doable but we would like to keep this metric and analyze in terms of the metrics obtained for regular CMS usage (e.g. non Headless)

• Device Distribution
  -- With so much device fragmentation and the impact on performance of using low-end devices, it would be good to know where content powered by difference CMSes is being accessed from.
  -- Comparison against non-CMS cases would also shed light on demographics, geography (together with device usage per region)

• Connection distribution
  -- Connection types

• HTTPArchive/CrUX Metrics: We should capture a view of the ecosystem in terms of usability metrics
  -- Is it happening?: Has the navigation started successfully? has the server started responding? Metric: First Paint,TTFB (HTTPArchive/CrUX Metrics)
  -- Is it useful?: when you’ve painted text, an image or content that allows the user to derive value from the experience and engage with it. Metrics: First Contentful Paint, First Meaningful Paint, Speed Index (HTTPArchive/CrUX Metrics)
  -- Is it usable?: when a user can start meaningfully interacting with the experience and have something happen (e.g tapping on a button). This can be critical as users can get disappointed if they try using UI that looks ready but isn't. Metrics: Time to Interactive(lab), First CPU Idle, First Input Delay (field)
  -- Is it delightful?: delightfulness is about ensuring performance of the user experience remains consistent after page load. Can you smoothly scroll without janking? are animations smooth and running at 60fps? do other Long Tasks block any of these from happening?.

Check the brainstorming doc to explore ideas.

These metrics would paint a holistic, data-driven picture of the CMS landscape. The HTTP Archive does have its limitations and blind spots, so if there are metrics out of scope it's still good to identify them now during the brainstorming phase. We can make a note of them in the final report so readers understand why they're not discussed and the HTTP Archive team can make an effort to improve our telemetry for next year's Almanac.

Next steps: Over the next couple of months analysts will write the queries and generate the results, then hand everything off to you to write up your interpretation of the data.

Additional resources:

• All chapter assignments
• Project timeline
• About the project

[rviscomi]

Updated the list of reviewers. Only remaining AI is to finalize the metrics.

[mor10]

Heyyyy... If you need a non-googler to review this, I'm here for it.

[rviscomi]

Yes please! Thanks Morten!

[rviscomi]

@amedina @westonruter @mor10 @sirjonathan We're hoping to finalize the metrics for each chapter today. I've updated the list at #16 (comment) with what was in the brainstorming doc. If it looks good to everyone, please tick the last checkbox and close this issue. Thanks!

[amedina]

@mor10 Can you go over the CMS section here?

/cc @westonruter