F1 Macro score wrong(?) when label never appears

[ejohb]

🐛 Bug

When F1 (Macro) is calculated on a batch where one (or more) labels never appear, the result is <1.0 even when all predictions are correct. This seems very counterintuitive. Is it expected?

To Reproduce

import torch
from torchmetrics import F1

f1=F1(num_classes=2, average='macro')(preds=torch.tensor([0,0]), target=torch.tensor([0,0]))
print(f1)

Output: 0.5.

This occurs if one of my batches lacks one (or more) of the labels (my real use-case has many more classes).

Expected behavior

I'm not an expert on how F1 is calculated, but I would expect the output to be 1.0.

Environment

pytorch-lightning==1.3.1
torch==1.8.1+cu111
torchmetrics==0.2.0
torchvision==0.9.1+cu111

[github-actions]

Hi! thanks for your contribution!, great first issue!

[SkafteNicki]

seems related to #295