Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix - Object Detection IOU #817

Merged
merged 2 commits into from
Nov 8, 2024
Merged

Bugfix - Object Detection IOU #817

merged 2 commits into from
Nov 8, 2024

Conversation

czaloom
Copy link
Collaborator

@czaloom czaloom commented Nov 8, 2024

valor version checks

  • I have confirmed this bug exists on the latest version of valor.

Reproducible Example

def test_iou_computation():

    detection = Detection(
        uid="uid",
        groundtruths=[
            BoundingBox(xmin=0, xmax=10, ymin=0, ymax=10, labels=["0"]),
            BoundingBox(xmin=100, xmax=110, ymin=100, ymax=110, labels=["0"]),
            BoundingBox(
                xmin=1000, xmax=1100, ymin=1000, ymax=1100, labels=["0"]
            ),
        ],
        predictions=[
            BoundingBox(
                xmin=1,
                xmax=11,
                ymin=1,
                ymax=11,
                labels=["0", "1", "2"],
                scores=[0.5, 0.25, 0.25],
            ),
            BoundingBox(
                xmin=105,
                xmax=116,
                ymin=105,
                ymax=116,
                labels=["0", "1", "2"],
                scores=[0.5, 0.25, 0.25],
            ),
        ],
    )

    loader = DataLoader()
    loader.add_bounding_boxes([detection])

    assert len(loader.pairs) == 1

    # show that three unique IOUs exist
    unique_ious = np.unique(loader.pairs[0][:, 3])
    assert np.isclose(
        unique_ious, np.array([0.0, 0.12755102, 0.68067227])
    ).all()

Issue Description

There are two serious bugs.

  1. IOU is being computed after label permutation which means that a ton of work is being redone.
  2. (1) was introduced at some point without updating the IOU allocation. This means that all IOUs were identical for every pair within an image. This did not affect the results between images.

Expected Behavior

IOU should be computed the minimum number of times and should be properly reported per groundtruth-prediction pair.

@czaloom czaloom added the bug Something isn't working label Nov 8, 2024
@czaloom czaloom self-assigned this Nov 8, 2024
@czaloom czaloom linked an issue Nov 8, 2024 that may be closed by this pull request
1 task
@czaloom czaloom marked this pull request as ready for review November 8, 2024 18:05
Copy link
Contributor

@rsbowman-striveworks rsbowman-striveworks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@czaloom czaloom merged commit aaf352c into main Nov 8, 2024
13 checks passed
@czaloom czaloom deleted the czaloom-objdet-iou-bugfix branch November 8, 2024 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Object Detection IOU Computation
2 participants