Confusion matrix plot doesn't display numbers

[mnrozhkov]

I follow docs to log/plot confusion matrix with DVCLive, but don't see the number of labels per class

How to reproduce:

y_true = [1, 1, 2, 2, 2, 1, 1]
y_pred = [2, 1, 1, 2, 1, 2, 2]
live.log_sklearn_plot("confusion_matrix", y_true, y_pred)

Output

Expected result

[mattseddon]

@mnrozhkov This is not an issue with the extension but the expected behaviour of the confusion matrix template.

Maybe @dberenbaum or @daavoo can give some background on the decision not to display those numbers.

Please note the details for each cell are available in a tooltip:

[daavoo]

Maybe @dberenbaum or @daavoo can give some background on the decision not to display those numbers.

iterative/dvc-render#127

I think it was about the YOLO repo where the text was rendering really bad

[shcheklein]

@mnrozhkov do you feel it would be substantially better to include numbers? Is it a high priority? I guess we can try to add an option to include numbers. Also, how TB, W&B, MLFlow show this by default?

[mnrozhkov]

@mnrozhkov do you feel it would be substantially better to include numbers? Is it a high priority? I guess we can try to add an option to include numbers. Also, how TB, W&B, MLFlow show this by default?

First, I followed the documentation (here, and here, and here). Everywhere - confusion matrixes with numbers! So, when I see my plot without numbers - I have a confusion :))))

I looked through your previous discussions. Agree, that for a large number of classes numbers bring a mess to confusion plots. When there are only a few of them, I would prefer to have numbers plotted! So, it would be cool to have option to turn them on/off.

Here is an example, 2 plots that look pretty similar. If you compare 3-4 experiments and see such plots in a row, you may miss differences.

However, F1 metrics difference is 14%. The difference of values btw cells is around 15-20%.
This example is artificial and subjective, but I tried )))

To sum it up

• it's not critical if there is an alignment btw docs and visualization
• I would prefer to have the option to turn on/off numbers on plots

Below are examples for TB (have numbers) and W&B (additional level of detalization)

• https://www.tensorflow.org/tensorboard/image_summaries
• https://wandb.ai/wandb/plots/reports/Confusion-Matrix-Usage-and-Examples--VmlldzozMDg1NTM

[dberenbaum]

A user also asked about this in discord.

Overall, I think it gets to the need to customize the plots more easily. I think we all agree at this point that custom templates are not the best way to configure this type of option, so hopefully we can find simpler ways to add toggles like this as we work on improving the plots implementation.

[mattseddon]

Overall, I think it gets to the need to customize the plots more easily. I think we all agree at this point that custom templates are not the best way to configure this type of option, so hopefully we can find simpler ways to add toggles like this as we work on improving the plots implementation.

Unpopular opinion: We have put in a massive amount of effort to make templates and plots customizable and we give all of this away for free. ATM we are small and stretched. Further help with customising plots should be part of a paid engagement. Either through a support contract or Studio.

[dberenbaum]

I don't disagree @mattseddon, and regardless of free/paid, I'm not sure additional plots refinement needs to be high priority right now. However, if/when we want to prioritize plots and update the implementation, easier customization should be one of the priorities IMO.

[mattseddon]

FWIW these are the equivalent diagrams in Plotly: https://plotly.com/javascript/heatmaps/

[mnrozhkov]

The source of the confusion is the inconsistency between plots shown in documentation, and one people see when they try to build their own. I think we may just update the documentation, use images people will see when they build plots, and point to additional details in a tooltip. WDYT?

[dberenbaum]

Good point @mnrozhkov! Opened iterative/dvc.org#4866.