-
Notifications
You must be signed in to change notification settings - Fork 247
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update MOT17 metrics and add MOT20 results #202
Conversation
b46a665
to
ace7d2a
Compare
ace7d2a
to
ee81454
Compare
bd7bf31
to
51a65f8
Compare
Also change detections to use those from ByteTrack.
51a65f8
to
637b60b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just had one question regarding the filter
distance_threshold=DISTANCE_THRESHOLD, | ||
detection_threshold=DETECTION_THRESHOLD, | ||
pointwise_hit_counter_max=POINTWISE_HIT_COUNTER_MAX, | ||
hit_counter_max=HIT_COUNTER_MAX, | ||
filter_factory=FilterPyKalmanFilterFactory(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why are we still using filterpy? OptimizedFilter is faster and it shouldn't affect metrics too much.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried using OptimizedKalmanFilterFactory
, but the metrics were a bit worse. We saw this when implementing OptimizedKalmanFilterFactory
but we still haven't gotten to the bottom of it.
This PR recalculates MOT Challenge's metrics, more specifically it updates MOT17 metrics and adds MOT20 metrics. The biggest change in performance comes from using ByteTrack's YOLOX object detection model to get the detections instead of using the original detections from MOT Challenge. This is a common practice in top-ranked tracking libraries since the original detections aren't precise.
Results are reported on the
train
split since we don't have the ground truth for thetest
split.Test
results must be obtained by uploading the tracking output to MOT Challenge. We are going to obtain them in a near future and update the results then.Changes in this PR include:
We decided not to change the motmetrics test to include MOT20 or the new detections since the overall result isn't important, we are only checking if everything remained the same. We also decided not to add camera motion to the test since this implied using the entire dataset with images and it would be too heavy to download in CI.