Postmortem: Broken "Edit on GitHub" Links

[rogermparent]

This ticket contains a postmortem about docs "Edit on GitHub" links being broken.

High Level Summary

In PR #3170, the logic for generating "Edit on GitHub" links was refactored so that the GitHub repo pointed to would be configurable in a shadowed src/constants file. While the logic works fine, I had made a typo and put iterative/dvc as the repo instead of iterative/dvc.org, so the resulting links ended up broken across the repo.

Timeline

Times are written in UTC.

2022-01-31, 22:00: #3170, with the broken buttons, was merged into master after review
2022-02-01, 02:30: #3243 was opened as @shcheklein noticed the issue with Hexometer
2022-02-01, 12:30: #3244 was opened by @julieg18 with a fix
2022:02-01, 15:00: #3244 was approved and merged by me, fixing the issue

Perf indicators:

• Time to notice: Less than a day, roughly 14 hours.
• Time for websites team to start working on the issue: 10 hours since the initial issue was reported
• Fix Developed After: less than 1 hour, it was just a one-liner typo change
• Resolved After: 3 hours after the fix was developed to get an approval and merge

Impact

All "Edit on GitHub" buttons on docs pages would point to 404 pages, inconveniencing any users that may have wanted to use them and likely dinging our SEO while the problem was occurring due to having a broken link on the page.

Root Cause Analysis

I made a typo in #3170, and reviewers didn't catch the misconfiguration (mostly because the PR was pretty massive)

Prevention and next steps

• Try to break apart chunks when possible (like the docs template refactor in Move Documentation components to theme #3170) into linked PRs so they're easier for reviewers, making it more likely some of these issues will be caught.
• Implement some sort of link check that inspects the resulting page itself, as opposed to source files like our current checker does.
• Actually try and use features that are changed, even if it's just a refactor.

cc @iterative/websites