Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cudacodec::Videoreader fix YUV color conversion #3828

Conversation

cudawarped
Copy link
Contributor

@cudawarped cudawarped commented Nov 22, 2024

Currently VideoReader color conversion:

  1. Forces quantization to 8 bits by using the NV12 surface format for all inputs regardless of the encoded bit depth or chroma subsampling.
  2. Does not respect the color space standard if specified in the video source.
  3. Only outputs BGR, BGRA, GRAY and NV12 frames.

Note: Previous attempts to fix this #3468 didn't address (2).

This PR uses the Nvidia Video Codec SDK ColorSpace.cu sample as a guide to:

  1. Use the color space standard if it exists and default back to a suitable format based on the video resolution if not.
  2. Decode YUV420 10/12 bit and YUV444 8/10/12 bit sources.
  3. Allow the user to choose the desired output bitdepth. e.g. Reqest 8 bit output from a 16 bit source.
  4. Add RGB/RGBA and planar outputs.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

  • I agree to contribute to the project under Apache 2 License.
  • To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
  • The PR is proposed to the proper branch
  • There is a reference to the original bug report and related work
  • There is accuracy test, performance test and test data in opencv_extra repository, if applicable
    Patch to opencv_extra has the same branch name.
  • The feature is well documented and sample code can be built with the project CMake

Comment on lines +154 to +159
if (format.colorSpaceStandard == ColorSpaceStandard::Unspecified) {
if (format.width > 1280 || format.height > 720)
format.colorSpaceStandard = ColorSpaceStandard::BT709;
else
format.colorSpaceStandard = ColorSpaceStandard::BT601;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it some heuristic? Do you have any reference for the decision?

Copy link
Contributor Author

@cudawarped cudawarped Dec 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, from my understanding 709 is for HD but I could't find a definitive source to say if there was an exact definition of what resolution this starts at. Although many sources mentioned 720p as the minimum resolution in the end I chose 720p as it replicated the color format behaviour from sources decoded with the VideoCapture class.

@asmorkalov asmorkalov merged commit 89529d7 into opencv:4.x Dec 30, 2024
11 checks passed
@asmorkalov asmorkalov mentioned this pull request Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants