[Metrics] MetricCollection

[SkafteNicki]

What does this PR do?

Adds generic MetricCollection object to chain together multiple Metric objects.
I seen both on slack and here on github (#4255) that users are defining dictionaries of metrics, to easy evaluate multiple at once. This PR adds support for this by introducing the MetricCollection object that can wrap multiple metrics into a single callable metric.
Also adds a prefix argument to self.log_dict such that users easily can reuse the same MetricCollection in both train, val, test.

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?
• Did you verify new and existing tests pass locally with your changes?
• If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

• Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[Borda]

LGTM, just I think that have seen some other metric collection from @tchaton?

[s-rog]

Jury-rigged a similar class for my own project, glad to see a formalized API!

[tchaton]

Great job ! Small ideas to consider.

[chris-clem]

Hi,

it seems that the prefix argument to self.log_dict was not added?

Edit:
I went through the commits and it was removed in this commit: 6986376. Was there a reason why it was deleted? It is still in the docs and seems like a useful argument.

[SkafteNicki]

Hi @chris-clem,
The prefix arg was removed from the self.log_dict and instead added directly to the MetricCollection. Please see relevant section in the torchmetrics documentation https://torchmetrics.readthedocs.io/en/latest/pages/overview.html#metriccollection

[xfffrank]

Hi @chris-clem,
The prefix arg was removed from the self.log_dict and instead added directly to the MetricCollection. Please see relevant section in the torchmetrics documentation https://torchmetrics.readthedocs.io/en/latest/pages/overview.html#metriccollection

Hi @SkafteNicki,
Thank you! I just got the issue. This metric page is now outdated as the "prefix" argument has gone.

[SkafteNicki]

@xfffrank I see your point. We are going to release v1.3 of lightning soon and then stable will instead take you to this page https://pytorch-lightning.readthedocs.io/en/latest/extensions/metrics.html which will redirect you to the torchmetrics documentation.

[xfffrank]

But I need to manually install torchmetrics 0.3.0 to have the "prefix" argument.

[SkafteNicki]

But I need to manually install torchmetrics 0.3.0 to have the "prefix" argument.

Yes, torchmetrics 0.3.0 is in rc right now, which can be installed as:
pip install torchmetrics==0.3.0rc1
(0.3 is going to be out anytime soon, then you can just upgrade the package)