Dashboard for MSSQL Metricbeat module

[sayden]

Because this is my first dashboard PR, I'll maybe need some guidance about what to do / include.

I'm gonna start with the dashboard that includes the key metrics from performance metricset. Then I'll do the same with the db metricset and then a single "big" dashboard containing both.

The PR just includes a single file Metricbeat-mssql-performance.json within _meta/kibana/7/dashboard.
It has been done using kibana-7.0.0-alpha2-linux-x86_64 and as far as you can see from the screenshots, it has some issues with the visualization of gauges.

Metrics from performance metricset that are being presented are:

• Batch Requests/sec: Average value. I'm not sure about the "look" of this metric as it seems a value that only grows, not a rate as describe in the docs.
• Buffer cache hit ratio: Represented as a gauge. There's some visualization issue so it's not properly rendered yet.
• Lock Waits/sec: Average value. Fixed in 3 because the testing database has no usage.
• Transactions: Average value. Fixed in 0 because the testing database has no usage.
• User Connections: Average value.

I could try to write some crazy script that load tests the database to generate a more dynamic output for the visualization.

[ruflin]

I tested this on my machine. After moving the dashboards to 6 it loads as expected. Unfortunately in Kibana I get the following errors:

I think the reason is that it tries to refer to an index pattern which does not exist with the id in the screenshot. Looking at the dashboard json it has quite a few lines like:

searchSourceJSON": "{\"filter\": [], \"index\": \"8a0c0250-18b9-11e9-9836-f37dedd3b411\", \"query\": {\"query\": \"\", \"language\": \"lucene\"}}"
                }, 

In most other dashboards these are empty or at least do not reference an index. I'm wondering where this is coming from. How did you load the index pattern and how did you export the dashboard? @jsoriano might now here more.

For the Batch Requests/sec there seems to be something off here, potentially on the ingestion side. Please investigate.

[jsoriano]

In most other dashboards these are empty or at least do not reference an index. I'm wondering where this is coming from. How did you load the index pattern and how did you export the dashboard? @jsoriano might now here more.

Yeah, this is weird, in the rest of dashboards we have there the index pattern itself, not its uuid, that is going to be always different.
If I try to export a dashboard now it also has the uuid, could it be a change in Kibana 7? This would require changes in our exporter.

[jsoriano]

@sayden the selection of the metrics for this dashboard looks good to me.

Some observations:

• +1 to review the Batch Requests/sec metric, this doesn't look good
• There is also something weird in the transactions graph, why it doesn't show any scale?
• Could the hit ratio be shown as a line graph over time? Apart of the visualization issue, I don't like gauges so much in this kind of dashboards, when you select an interval with historical data they are not so useful. I'd only use them in not-actionable dashboards with current data.
• Review the titles of the dashboard and the visualizations. The dashboard should be titled [Metricbeat MSSQL] Performance, and the visualizations should have [Metricbeat MSSQL] as suffix. You can look for examples at other dashboards.
• Once you have changed the titles in visualizations, notice that you can still edit the title in the dashboard. In the dashboard they don't need to contain the word MSSQL as this is a dashboard for this. You can give a more specific title in this context.

and then a single "big" dashboard containing both.

Regarding this, more than a big dashboard, create an overview one, it doesn't need to contain all the info of all metricsets, better give an overview using key metrics of the available metricsets, potentially linking to the other dashboards for details.

[cwurm]

In most other dashboards these are empty or at least do not reference an index. I'm wondering where this is coming from. How did you load the index pattern and how did you export the dashboard? @jsoriano might now here more.

Yeah, this is weird, in the rest of dashboards we have there the index pattern itself, not its uuid, that is going to be always different.
If I try to export a dashboard now it also has the uuid, could it be a change in Kibana 7? This would require changes in our exporter.

Seems it's a change in Kibana (see here, and here), I'm running into the same issue unfortunately.

I guess it would be possible for us to change the dashboard exporter to overwrite the ID in searchSourceJSON.index with a pattern (e.g. metricbeat-*). But I'm not sure how long Kibana would accept that format. I think we need a longer term solution supported by Kibana.

[cwurm]

So, as I commented in #10117 the index-pattern IDs probably stem from using a manual created index pattern. All should be fine if you load the generated index pattern from the Beat.

[sayden]

@cwurm Thanks for the help! Really appreciated.

New version:

Dashboard naming now also follows convention:

Visualizations also follows naming conventions:

More details that you can see in the screenshots:

• I have removed "units" from Y axis, the reason is that it was dedundant to see User connections in the axis and the title, so close to each other.
• I have seen in dashboards like the one of Nginx that the X axis title was per 10 seconds instead of @ timestamp per 30 seconds but I didn't understand the reasoning behind this difference so I left it untouched.
• Batch/request per sec metric: Finally I found that the metric returned by MSSQL is a cumulative rate so I have changed the graph to a derivative and now it looks much better.
• I have changed the Hit Ratio chart to line too. I wanted it to show as gauge because it was the only chart that could bring a bit more visual dynamism to the dashboard (all charts are plain, boring 😄 line charts).
• Titles of Visualizations have been changed. Titles inside the dashboards too. You can check in the screenshots.
• About the big dashboard: Yes, you can also call it overview but the idea is the same. I just didn't find the correct word when I wrote it but later I realized that the dashboards are commonly called Overview
• Transactions shows as 0 because it's the value that is being returned by the database, it's not a parsing nor visualization error. All in all, it would be even more weird that there are ongoing transactions in a testing database 😄

[jsoriano]

Thanks for addressing the issues, it looks better now 🙂

Visualizations also follows naming conventions:

The suffix for visualizations should be [Metricbeat MSSQL]

I have removed "units" from Y axis, the reason is that it was dedundant to see User connections in the axis and the title, so close to each other.

LGTM. It makes sense to me in these cases, but take into account that the title could be the thing measured (for example in the Locks/sec the title could be also "Lock contention"). Also think that the visualizations could be reused in other custom dashboards. And also that we may look for a coherent style between dashboards, what could include to always display the units (afaik this is not the case by now 🙂 )

Transactions shows as 0 because it's the value that is being returned by the database, it's not a parsing nor visualization error. All in all, it would be even more weird that there are ongoing transactions in a testing database

Yes, what looked weird to me is that nothing is shown in the Y-Axis, but this seems to be what happens when everything is 0.

[sayden]

Fail seems unrelated. Merging.