-
Notifications
You must be signed in to change notification settings - Fork 13
Triage process
Element apps have active user engagement which leads to many new issues (defects and enhancement requests) every week. We want to have a structure for triage which ensures new issues are labelled correctly and are actively processed. Our secondary, longer term, goal is to reduce the issue backlog for all platforms.
This triage process is currently used by the web, iOS, Android and product teams at Element.
The issue wrangler role rotates between developers on the app team every week. There may be one or more issue wrangler, depending on the team process. The responsibility of the issue wrangler is to:
- Do initial assessment on incoming issues
- Redirect issues to appropriate teams if needed (this can be with labels or raising them directly)
- Re-triage based on corporate priorities as indicated by management
- Move issues to correct repositories if needed
- Pick out highest priority issues and work on them for the allocated time every week
The web team, iOS team and Android team use triage boards to manage incoming issues.
The issue wrangler will:
- Label new issues with:
- One T-* label (usually T-Enhancement or T-Defect)
- One O-* label and the reasoning for it unless explicitly clear from previous comments
- One S-* label for issues labelled T-Defect (optionally also for any other type of issue) and the reasoning for it unless explicitly clear from previous comments
- X-, A- and any other labels as needed
- Request more information as needed on new issues
- Update new issue description and titles as needed to be useful
- Review any issues processed by anyone else (e.g. the team member who filed the issue)
- Review any issues updated with more information
- Check for duplicates and close the new issue, mentioning the duplicate issue so progress can be tracked in the original issue. Older issue should only be closed in preference to newer issue if the newer issue has more detail and relevant discussion.
- Refer issues to Engineering and Product Managers if priority is uncertain post-triage
- Raise regressions with the relevant team immediately
- Priority issues will be moved to the backlog for the relevant team. These are defects labelled with:
- A11y and S-Critical - web/desktop
- S‑Critical and O-Frequent - web/desktop
- S‑Critical and O-Occasional - web/desktop
- S‑Major and O-Frequent - web/desktop
- Defects with other labels which have been agreed by the team as being high priority
- Issues which do not fulfil the criteria for priority focus should not be added to the Web App Team's board
- If possible, confirm that the issue is reproducible
Issue priority is made up of a number of criteria such as risk, cost, impact, occurrence and severity. To keep the process simple, we use occurrence and severity to create a basic risk assessment matrix to use objective criteria to ease primary prioritisation.
| Occurrence (score) | 3 | 2 | 1 | |
|---|---|---|---|---|
| Severity (score) | Frequent | Occasional | Uncommon | |
| 4 | Critical | 12 | 8 | 4 |
| 3 | Major | 9 | 6 | 3 |
| 2 | Minor | 6 | 4 | 2 |
| 1 | Tolerable | 3 | 2 | 1 |
Note that scores correspond to the impact of the label. Combinations with a score of 8-12 will be prioritised for picking up by the team in the first instance because they are the most visible with the severest effect on the users.
| Labels | Equivalent priority | What it means |
|---|---|---|
| S‑Critical and O-Frequent S‑Critical and O-Occasional S‑Major and O-Frequent |
P1 | These issues should be worked on in this sprint or next sprint. If the backlog of issues is too long, we should reevaluate why the bugs are not caught earlier. |
| S‑Critical and O-Uncommon S‑Major and O-Occasional S‑Minor and O-Frequent |
P2 | When all the highest priority issues are done, this is the next set to tackle. Ideally we should be fixing a few issues from this group every week. |
| S‑Major and O‑Uncommon S‑Minor and O-Occasional S‑Tolerable and O-Frequent |
P3 | These issues are wishful thinking for now. We hope to get to them one day, but they are low priority. There are likely to be some good new contributor issues in here. |
| S‑Minor and O‑Uncommon S‑Tolerable and O‑Occasional S‑Minor and O‑Uncommon |
P4 and P5 | These issues are unlikely to be actively looked at by the webapp team, but may be picked up by community. |
Issues need sufficient information in them to be reproducible and therefore for developers to be able to fix them.
These issues are usually either personal notes or reminders or trackers for ongoing work.
This should be automated and done by a bot in the future (and by issue wrangler for now):
- Comment with “This needs more detail to be useful to other developers, assigning to issue author to update with more info or fix” and assign to developer
- Label with X-Needs-Info and revisit a long way down the line later
This part cannot be automated easily so will continue to be done manually for now:
- Update with more information if it’s obvious to you what the issue is about and that it’s not a personal note, and comment asking the developer to include more information next time.
- Label with any labels that you can, especially A-* labels
- Revisit issues and close them if they have been open for a long time
- Label with X-Needs-Info, comment with “
@<username>Thank you for opening an issue. Unfortunately there is not enough information there for me to be able to reproduce it. Please update the description with steps/screenshots/video/more details so our developers can have a look at it.” or a more appropriate comment - [Automation will move the issue to a follow up column if there is no comment for two weeks or to incoming if there is a comment within two weeks]
- If no update in two weeks, try to reproduce and update the description.
- If you can’t reproduce because you don’t have the right setup, label with X-Cannot-Reproduce, comment with “
@<username>Unfortunately I cannot reproduce your issue. Please update your issue with more information. If we don’t hear back from you soon, we may close the issue as we need to be able to reproduce it before we can fix it.” - If you can’t reproduce with the right setup (to the best of your knowledge), then tag with X-Cannot-Reproduce and close with a “Thank you for opening an issue. Unfortunately I wasn’t able to reproduce it. If you keep experiencing this defect, please update your description with steps/screenshots/video/more details and add a comment with
@<your own username>to let me know that it’s ready to be reopened.” - [If the issue is open and still in follow up column, automation will move issue to candidate for closing if there is no comment for two weeks, or to incoming if there is a comment within two weeks]
- If the issue is blocking on more information, then close it with a “Thank you for opening an issue. Unfortunately I wasn’t able to reproduce it. If you keep experiencing this defect, please update your description with steps/screenshots/video/more details and add a comment with
@<your own username>to let me know that it’s ready to be reopened.” message. - If the issue is not blocking on more information, then move it to triaged column.
See comment templates for more details
As a general rule,
- Platform specific defects and feature requests should live in the platform issue tracker (Android, iOS and web)
- Cross-platform defects should live in element-meta
- Cross-platform feature requests which don't require product input or discussion should live in element-meta issues
- Cross-platform features and requests which require product input or discussion should live in element-meta discussions
If you are triaging platform defects,
- If the defect is a valid platform bug, leave it in the platform tracker and consider opening an epic in element-meta to track it cross-platform
- If the defect mentions a bug in another platform, advise the author to file an issue in that platform or open the issue yourself if you can reproduce it
- If the defect is blocked by a fix in another repository, keep the issue as a tracker issue in the platform issue tracker, label it with
X-Blockedand link to the blocker issue. Bug or enhancement request in a specific platform, depends on fix in another repo -> tracker issue in platform bug tracker which references blocker, labelled X-Blocked
Developing...
Issues suggesting improvements to existing functionality are triaged as enhancements, mirroring our review process. When suggesting an enhancement, it's important to remember:
- Enhancements are just ideas: Issue trackers are a great place to file issues and foster discussion, but ideas usually come with non-obvious tradeoffs which may need to be considered when making changes.
- As contributors, we aren't our users: Even though we all use Element, anyone engaging with online issue trackers is both blessed and cursed with technical and domain knowledge. We need to consider our own biases and individual subjectivities.
- We are constantly aiming to reduce debt: As a product with a rich technical legacy, we have a lot of debt in both usability and product thinking. We triage enhancements with a view to be net reducing debt.
As we have thousands of open issues and hundreds of open enhancements, we're experimenting with triaging and prioritising based on confidence in things like:
- Impact: Is the core problem important enough to be solved? Does the proposed enhancement solve it?
- Reach: Is the enhancement universally useful? Does it avoid individual subjectivity?
- Alternatives: Can the same goal be achieved through an acceptable or existing alternative?
- Viability & feasibility: Is it aligned with our long term goals? Is this something we'd be happy to support and carry forward?
We're currently experimenting with using z-confidence-low, z-confidence-medium & z-confidence-high labels to better organise open enhancements and suggestions.
Features are new functionality that currently doesn't exist in the apps. We collect feature ideas in Discussions, in element-meta.
Features are normally cross-platform.
When we see a feature request filed against a platform, our triagers will move it to element-meta, and then into Discussions.