Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to nv-flip for image comparison #3830

Merged
merged 10 commits into from
Jun 8, 2023

Conversation

cwfitzgerald
Copy link
Member

@cwfitzgerald cwfitzgerald commented Jun 4, 2023

Checklist

  • Run cargo clippy.
  • Run RUSTFLAGS=--cfg=web_sys_unstable_apis cargo clippy --target wasm32-unknown-unknown if applicable.
  • Add change to CHANGELOG.md. See simple instructions inside file.

Connections

Resolves #2761

Description

Integrates NVlab's ꟻLIP for image comparison, using their various weighted histogram methods. This feels soooo much more stable, thresholds aren't insanity, and new devices seem to consistently fall under threshold.

I also fixed up some other issues with the tests.

Testing

I ran the tests on all of my machines' cards and they all pass.

Copy link
Contributor

@nical nical left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. What color space is nvflip expecting the data to be in? It should be documented in the nv-flip crate and we should make sure the tests actually produce pixels in the right color space otherwise we are only giving the illusion of having a principled error metric.

wgpu/tests/common/image.rs Outdated Show resolved Hide resolved
wgpu/tests/common/image.rs Show resolved Hide resolved
@cwfitzgerald
Copy link
Member Author

Filed the color space info here: gfx-rs/nv-flip-rs#1

@cwfitzgerald cwfitzgerald requested a review from nical June 8, 2023 00:49
Copy link
Contributor

@nical nical left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Member

@teoxoy teoxoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@teoxoy
Copy link
Member

teoxoy commented Jun 8, 2023

@cwfitzgerald there seems to be an unexpected test pass 👀

@cwfitzgerald
Copy link
Member Author

Yeah, that was #3809 landing - will remove the failures.

@cwfitzgerald cwfitzgerald merged commit 1738202 into gfx-rs:trunk Jun 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Better Metric for Image Comparison Testing Failure
3 participants