Skip to content

Commit

Permalink
docs: Add "Debugging DORA Issue Metrics" section to "Dashboard Troubl…
Browse files Browse the repository at this point in the history
…eshooting" guide (#631)

* Co-authored with: Louis.z <[email protected]>

docs: Add "Debugging DORA Issue Metrics" section to "Dashboard Troubleshooting" guide.
Supplied with .png diagrams for each of the DORA metrics

* docs: add a small troubleshooting section regarding webhook usage

* refact: move dashboard troubleshooting images to /docs/Configuration/images folder

* docs: include to v0.18

* docs: remove subfolders from image path
  • Loading branch information
KucherenkoSerhiy authored Aug 30, 2023
1 parent 22dd394 commit 0248b94
Show file tree
Hide file tree
Showing 10 changed files with 84 additions and 0 deletions.
Binary file added docs/Configuration/images/cfr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/Configuration/images/deployments.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/Configuration/images/lead_time.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/Configuration/images/mttr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 42 additions & 0 deletions docs/Troubleshooting/Dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,48 @@ description: >
Dashboard Troubleshooting
---

## Debugging DORA Issue Metrics

This section may help if `Median Time to Restore Service (MTTR)` or `Change Failure Rate (CFR)` do not appear on the dashboards or you want to learn more about how these issue-based metrics are built.

### DORA Validation Dashboard
Starting from DevLake v0.18 this dashboard can be found near the `DORA` dashboard. Also, it can be accessed by a direct link in the `Dashboard Instruction` panel in the `DORA` dashboard.

This dashboard is a step-by-step guide to check which step went wrong **for all 4 of the DORA metrics**. The sections are:
- Check "Deployment Frequency"
- Check "Median Lead Time for Changes"
- Check "Change Failure Rate" & "Median Time to Restore Service

### SQL scripts behind Grafana charts
Each chart has a hidden button in the top-right corner to access the context menu. In that menu, click `Edit` to open a more detailed view with the script that tells how exactly the data is queried.

### How issues and deployments are associated to projects

Following Entity-Relationship diagrams below represent how the data is mapped and used for each of the 4 DORA metrics.
They are based on the SQL queries for each of the charts.

Legend:
- Blue box: user data source, be it **deployments**, **pull requests** from the source code, or **issues**
- White box: a table or entity used by DevLake
- Connections: lines that tell how the tables are mapped, also specify which fields are used.

The `project_mapping` is responsible for mapping **deployments**, **pull requests** from the source code, or **issues**.
To do so, it must be filtered using either `table = 'cicd_scopes'`, `table = 'repos'`, or `table = 'boards'` when connecting to another table.

![](../Configuration/images/cfr.png)

![](../Configuration/images/deployments.png)

![](../Configuration/images/lead_time.png)

![](../Configuration/images/mttr.png)

### If you use webhooks

DevLake knows to which project an issue or a deployment belongs only by segregation between the webhooks.
I.e. **each project should have its own webhook**. A webhook used by multiple projects means that all the
issues or deployments published by that webhook **will be replicated among those projects**, as they belong to both of them.

WIP


Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
42 changes: 42 additions & 0 deletions versioned_docs/version-v0.18/Troubleshooting/Dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,48 @@ description: >
Dashboard Troubleshooting
---

## Debugging DORA Issue Metrics

This section may help if `Median Time to Restore Service (MTTR)` or `Change Failure Rate (CFR)` do not appear on the dashboards or you want to learn more about how these issue-based metrics are built.

### DORA Validation Dashboard
Starting from DevLake v0.18 this dashboard can be found near the `DORA` dashboard. Also, it can be accessed by a direct link in the `Dashboard Instruction` panel in the `DORA` dashboard.

This dashboard is a step-by-step guide to check which step went wrong **for all 4 of the DORA metrics**. The sections are:
- Check "Deployment Frequency"
- Check "Median Lead Time for Changes"
- Check "Change Failure Rate" & "Median Time to Restore Service

### SQL scripts behind Grafana charts
Each chart has a hidden button in the top-right corner to access the context menu. In that menu, click `Edit` to open a more detailed view with the script that tells how exactly the data is queried.

### How issues and deployments are associated to projects

Following Entity-Relationship diagrams below represent how the data is mapped and used for each of the 4 DORA metrics.
They are based on the SQL queries for each of the charts.

Legend:
- Blue box: user data source, be it **deployments**, **pull requests** from the source code, or **issues**
- White box: a table or entity used by DevLake
- Connections: lines that tell how the tables are mapped, also specify which fields are used.

The `project_mapping` is responsible for mapping **deployments**, **pull requests** from the source code, or **issues**.
To do so, it must be filtered using either `table = 'cicd_scopes'`, `table = 'repos'`, or `table = 'boards'` when connecting to another table.

![](../Configuration/images/cfr.png)

![](../Configuration/images/deployments.png)

![](../Configuration/images/lead_time.png)

![](../Configuration/images/mttr.png)

### If you use webhooks

DevLake knows to which project an issue or a deployment belongs only by segregation between the webhooks.
I.e. **each project should have its own webhook**. A webhook used by multiple projects means that all the
issues or deployments published by that webhook **will be replicated among those projects**, as they belong to both of them.

WIP


Expand Down

0 comments on commit 0248b94

Please sign in to comment.