Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lite Unify Metric Formatting #803

Merged
merged 11 commits into from
Oct 17, 2024
Merged

Lite Unify Metric Formatting #803

merged 11 commits into from
Oct 17, 2024

Conversation

czaloom
Copy link
Collaborator

@czaloom czaloom commented Oct 16, 2024

Changes

  • IoU changed to IOU for uniformity across valor.
  • Reorganized Metric creation into <tasktype>/utilities.py.
  • No longer defining a dataclass per metric. Only a single Metric type per task with multiple classmethods that help with initialization.
  • Classification and Object Detection now specify maximum_number_of_examples in the ConfusionMatrix metric.
  • Classification now follows the same format as object detection and semantic segmentation.
    • A metric value can no longer be a list. A metric value can be either a int, float or dict.
  • Remove as_dict parameter to make output typing consistent.

Old Classification Format

{
    "type": "Accuracy",
    "value": [2 / 3, 1 / 3],
    "parameters": {
        "score_thresholds": [0.25, 0.75],
        "hardmax": True,
    },
},

New Classification Format

{
    "type": "Accuracy",
    "value": 2 / 3,
    "parameters": {
        "score_threshold": 0.25,
        "hardmax": True,
    },
},
{
    "type": "Accuracy",
    "value": 1 / 3,
    "parameters": {
        "score_threshold": 0.75,
        "hardmax": True,
    },
},

@czaloom czaloom self-assigned this Oct 16, 2024
@czaloom czaloom changed the title Lite Unify Metric Format Lite Unify Metric Formatting Oct 17, 2024
@czaloom czaloom marked this pull request as ready for review October 17, 2024 20:17
@czaloom czaloom merged commit 8aa7c02 into main Oct 17, 2024
15 checks passed
@czaloom czaloom deleted the czaloom-lite-unify-metric-format branch October 17, 2024 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants