Add Modified Panoptic Quality metric

[marcocaccin]

🚀 Feature

Add Modified Panoptic Quality metric introduced in Seamless Scene Segmentation, Section 4 "Revisiting Panoptic Segmentation".

Motivation

The Modified Panoptic Quality is a subtle modification of the original, and it comes in handy to have it available when dealing with Panoptic Segmentation problems: it's not easy to say whether the original or the modified one is a better choice in any given situation, so one would ideally track both.
To prove the point, some public datasets such as nuScenes use both variants for evaluation on the leaderboard.

Pitch

I'm happy to continue the work on PQ to extend its functionality 😉

Alternatives

Additional context

Question to the maintainers: would you rather see this implemented in its own class and functional versions, or as a switch (e.g., implementation: Literal['original', 'modified'] = 'original') of the already existing metrics?
The code is almost identical besides a couple of extra if statements in the update() step, and so are the signatures, so I would lean on the latter option to avoid additional copy-paste.

[SkafteNicki]

Hi @marcocaccin, sound like a great enhancement.
After looking through the paper you linked, it looks to me that the easier solution would be to just have this as an argument implementation: Literal['original', 'modified'] = 'original' in the current PQ implementation as the authors does not really consider this a new metric but only an adjusted version of the original.

[stancld]

@SkafteNicki Just my two cents. On the other hand, we have BLEUScore and SacreBLEUScore, which basically differ in tokenization. But both options seem to be acceptable.

[SkafteNicki]

@stancld true. To me, it actually does not matter what option we go with, like you said but options are acceptable :]

[marcocaccin]

Thanks for the feedback! After thinking about it, maybe the least confusing solution is to have a common private module where all the functionality is implemented (torchmetrics/functional/detection/_panoptic_quality_common.py) and separate super-shallow modules containing the individual functional and class implementations.
The only reason I have for that is the docstring: I find that linking a single paper and writing a single docstring test is the least confusing.
Feel free to assign the issue to me btw 🙂

ps: first time I come across the Sacre bleu score, that was a good chuckle

[marcocaccin]

For a more concrete example, this is how the feature would shape up when having this feature as a separate metric: marcocaccin#1 (it's on my fork to avoid triggering CI for nothing).
Now that I wrote it (almost, missing all new tests), I have doubts whether it's the right way to go because of the amount of duplication. Let me know if the Literal argument is more or less desirable now