Skip to content

Review process

Jump to bottom
Richard van der Hoff edited this page Mar 30, 2023 · 3 revisions

Review process

This page outlines guidelines for both authors and reviewers of pull requests for Element on all platforms. We want the review process to be fair and transparent, acknowledging expectations, constraints and risks up front.

Triage

Pull Requests are triaged by the issue wrangler mirroring the triage process.

During review, it’s common to discover a need to assign more types of review or individuals as assignees. This should be carried forward by the review assignees, with no expectation from the initial triager.

Initial assessment

Triagers perform an initial assessment to determine what type(s) of review a request needs, based on the guidelines below. If you’re a core team member and require review from a known specific individual, it’s best to assign this directly when authoring your review.

Types of review

Technical review

As almost every pull request includes code contributions, most requests are reviewed technically. Each platform has individual guidelines for code contributions:

Design review

If a request impacts the user interface or user experience, design review should be assigned:

  • Assign to one of vector-im/design or matrix-org/design depending on the GitHub organisation
  • The active design maintainer will then assign the PR to an individual contributor to review, depending on domain expertise and availability
  • Design reviews are visualised in the design board, worked through with allocated time week on week

We, non-exhaustively, review for:

  • Visual & interaction design: Accessibility, spacing, sizing, surface, typography, iconography, colours, theming
  • User experience & content design: Usability, discoverability, UX writing, tone of voice, avoiding jargon, progressive disclosure
  • Product thinking: Solving the right problem, impacts to mental load for users, clashes with other goals

Product review

If a request materially impacts our strategy, viability or vision, product review should be assigned:

  • Assign to one of vector-im/product or matrix-org/product depending on the GitHub organisation
  • The active product maintainer will then assign the PR to an individual contributor to review, depending on domain expertise and availability
  • Product reviews are visualised in the product board, worked through with allocated time week on week

We, non-exhaustively, review for:

  • Product sense: User impact, solving the right problem, avoiding individual subjectivity, serving our wide & complex userbase
  • Strategy: Alignment with our short and long term goals, current focuses of the core team, roadmap
  • Viability: Feasibility over time, increasing debt, avoiding creating unnecessary requirements for future efforts

Contributing enhancements

If you’re contributing an enhancement, it’s important to remember that we review requests based on the review guidelines above. In particular, it’s important to remember:

  1. Enhancements in issue trackers are just ideas. Without explicit approval of an enhancement in advance, we can’t guarantee we’ll merge enhancements into main.
  2. As contributors, we aren’t our users. Anyone reading this is both blessed and cursed with technical and domain knowledge, and we need to consider our own biases and individual subjectivities.
  3. We are constantly aiming to reduce debt. As a product with a rich technical legacy, we have a lot of debt in both usability and product thinking. We review enhancements with a view to be net reducing debt.

To help aid product or design review of enhancements, please describe:

  • The problem being solved
  • What alternatives were considered
  • How you know this is the right solution
  • Which types of users you think would benefit from this enhancement
  • Users context in the app before using the enhancement
  • Clear steps to use the enhancement

Discussing reviews

Please keep any discussion or bumps related to the review within the review itself. It helps carry context on any critical thinking, and is the most productive way to move a review forward.

Avoid direct messaging individual assignees or stakeholders with requests to bump reviews. It causes interruptions which further delays review, and there’s often a mismatch in when community contributors are active (often in their spare time, evenings and weekends) and when core team members are active (who need valued downtime).

FAQs

  • When will my review be reviewed?
    • We’re currently focused on improving processes and working through the existing backlog of reviews. Once that works well, we’ll be able to set better expectations of timeliness.
    • We try to act quickly if a review is easy to reverse. Reviews take longer if they’re ambiguous or misaligned with the current focus of the core team.
  • What are the responsibilities of the design & product maintainers?
    • Each maintainer actively monitors when reviews are assigned to their teams, and then assigns review to individuals based on domain expertise and availability.
    • We’re dogfooding the role currently. Once we’re confident in what works, we’ll formalise it in further docs.
  • What’s next?
    • Once we’re confident in these processes, we’ll add more guidelines when authoring reviews in GitHub.
    • Automating workflows to help manage design & product review.
    • Once the review backlog is healthier, we’ll explore how to get ahead of product & design thinking in open issues.
      • Considering moving enhancements, suggestions and ideas from GitHub issues to GitHub discussions.
      • Considering grooming ideas with functional & UI specs, labelled with ‘help wanted’.
    • We plan to add design & product review checklists, to increase transparency for review authors.
    • We plan to mature our design systems and design guidelines to ease implementing UI.
    • We plan to increase visibility on our product thinking in the open, to help align community contributions with review.
Clone this wiki locally