Enhancement Proposal: Adding FFT3D as OpenCL Denoise Filter

[quamt]

Hi @rigaya ,

I hope you're doing well.

I appreciate your unwavering commitment to improving QSVEnc and responsiveness to user feedback.
I understand that you have many important tasks to attend to and that adding new features may only sometimes be possible.
Nevertheless, I am requesting that you kindly consider adding FFT3D as an OpenCL denoise filter.

Why FFT3D?
FFT3D provides a distinct noise reduction approach that can effectively complement the existing filters in QSVEnc.

Here is a summary of how it works and why it would be a valuable addition.

How It Works:
FFT3D utilizes fast Fourier transforms (FFT) to identify and isolate noise patterns in video material. This frequency-based approach filters noise more effectively, preserving finer details.

Unique Advantage:
Existing noise filters such as convolution3d, smooth, DCT, KNN, NLMeans, and PMD work efficiently for denoising videos.
However, these filters are limited when handling old and grainy video content. Even after making necessary adjustments, residual noise may be left behind. In such cases, using FFT3D can be highly beneficial as it can simultaneously handle noise across both temporal and spatial dimensions, resulting in more efficient and higher-quality denoising. This filter would add a tool to our denoise arsenal for challenging, older content.

Conclusion
This enhancement will be valuable for QSVEnc users, but I understand other tasks may take priority. If you can consider this request, it would be appreciated.

Thank you for your hard work and for providing such a versatile tool.

quamt

[MarcoRavich]

Well, even if the FFT3dGPU is a bit dated, @pinterf may provide useful info to implement it...

Here's the relative Doom9's info 3ad:
https://forum.doom9.org/showthread.php?t=89941

Last but not least, as already reported in my old degrain issue, @AhmetCanSolak's aydin could be a very useful resource to exploit.

Hope that helps/ispires !

[quamt]

Hi @forart,

Thank you for your input and for bringing up resources like pinterf's's work and the Doom9 thread.

However, there is a crucial difference between FFT3DGPU and what I am proposing for QSVEnc.

FFT3DGPU utilizes DirectCompute, fundamentally distinct from the OpenCL framework that QSVEnc uses for acceleration.
A different approach is required than the one used by FFT3DGPU to implement FFT3D as an OpenCL filter within QSVEnc.

The main difficulty is in modifying FFT-based denoising to integrate smoothly with QSVEnc's current GPU-accelerated pipeline using OpenCL. Although the instances you referred to are useful, they do not provide the precise technical details required for implementing OpenCL in QSVEnc.

Thanks again for your suggestions, and I look forward to further discussions on how we can enhance QSVEnc's capabilities.

Best,
quamt

[rigaya]

I agree with the usefulness of 3D noise reduction.

I was able to understand the implementation of the basic FFT algorithms necessary to implement FFT3D. I've made a 1D FFT test code seems which seem to work fine, so I believe that I can add noise reduction by FFT.

However, FFT3D looks like it is more than FFT denoise filter, it seems to have several extra options, which makes things complex.

I think adding a similar filter to ffmpeg fftdnoiz shall be realistic, which is more simple 3D noise reduction filter by FFT.

[quamt]

I agree with the usefulness of 3D noise reduction.

I was able to understand the implementation of the basic FFT algorithms necessary to implement FFT3D. I've made a 1D FFT test code seems which seem to work fine, so I believe that I can add noise reduction by FFT.

However, FFT3D looks like it is more than FFT denoise filter, it seems to have several extra options, which makes things complex.

I think adding a similar filter to ffmpeg fftdnoiz shall be realistic, which is more simple 3D noise reduction filter by FFT.

Thank you for considering adding the FFT-based denoise filter to QSVEnc.
I appreciate your efforts and understanding of the implementation challenges.
I look forward to testing this new feature once it's available.

[MarcoRavich]

Well, may @AkarinVS help with this ?

Seems that has "nice" experiences in VS/FFMPEG hardware-optimized filtering (and comes from Japan too).

[quamt]

Well, may @AkarinVS help with this ?

Seems that has "nice" experiences in VS/FFMPEG hardware-optimized filtering (and comes from Japan too).

Dear @forart,

I want you to know that I value your participation and willingness to offer suggestions on this platform.
However, in my culture, your approach may be seen as dismissive and disrespectful to the developers and other contributors.

While this is not my GitHub project, everyone must maintain a respectful and constructive environment. Simply suggesting other tools or solutions without directly addressing the issues raised can be seen as unhelpful and dismissive of the developers' efforts.

To elaborate on the points:

1. Issue 199:
You suggested looking at other tools without providing specific feedback on the user's issue. This can seem dismissive to the person who reported the problem, as it does not acknowledge or address the specifics of their situation.
Your comment, "Hope that helps," can be perceived as condescending, especially when it follows a suggestion to look elsewhere rather than providing a direct solution or constructive feedback.

2. Issue 193:
Like Issue 199, your response did not directly address the user's concern, which can be perceived as unhelpful within the context of QSVEnc.
Again, using the phrase "Hope that helps" after suggesting other tools can be seen as dismissive, avoiding engagement with the issue at hand.

3. Issue 163:
Although relevant to this issue, your introduction of another project (Xin26x) seemed to promote it without directly relating it to QSVEnc's goals and context. This suggestion can appear off-topic and not fully aligned with the discussion.

4. Issue 76:
Rigaya responded patiently and provided detailed explanations. However, persistently promoting your point without fully engaging with the feedback provided may be seen as not valuing the developers' input and efforts. Acknowledging and considering the responses rather than repeatedly pushing the same suggestions is important.

To improve the effectiveness of your contributions, I suggest the following:

- Address Specific Issues:
When responding to an issue, provide detailed and specific feedback relevant to the problem raised. This shows respect for the original poster's time and effort in describing their issue and helps maintain a constructive dialogue.

- Offer Constructive Advice:
Instead of directing users to look at other tools, provide actionable steps or insights to help them resolve their issues within the context of QSVEnc. The community appreciates this type of advice.

- Acknowledge Efforts:
Recognise and appreciate the detailed responses from developers and contributors. Acknowledging their efforts fosters a positive and collaborative environment.

- Maintain a Respectful Tone:
While "Hope that helps" is a polite sign-off, the context of your suggestions might overshadow this phrase, making it seem condescending. A more neutral and supporting closing can ensure your advice is well-received and understood in the spirit in which it was intended.

Staying focused on the topic and providing relevant and constructive feedback are essential. This ensures that discussions remain focused and beneficial for everyone involved.

My intention is not to offend or criticise you personally. I aim to highlight how we can all contribute more effectively to this collaborative environment. By ensuring that our feedback is specific, relevant, and respectful, we can support the developers and each other more positively and productively.

Aim to contribute in a manner that is specific, constructive, and respectful:

Specificity:
I'd like to point out that addressing the issue with detailed, relevant feedback. This shows a deep understanding of the problem and provides actionable steps or insights directly related to the context of QSVEnc.
• Example: In Issue 199, I detailed the problem and suggested specific ways to resolve it within the scope of QSVEnc.

Constructive Feedback:
Providing constructive responses offers solutions or alternatives that can be implemented within the QSVEnc framework. This approach respects the developers' efforts and contributes to the collective knowledge and improvement of the project.
• Example: In Issue 193, I suggested optimisations or features that could enhance the functionality of QSVEnc, rather than directing users to other tools.

Tone and Respect:
Responding respectfully and supportively. Acknowledge the work of the developers and contributors, fostering a positive and collaborative environment.
• Example: In various issues, I thank the developer for the efforts and provide feedback in a manner that encourages further discussion and improvement.

Regarding links to other tools, when I provide links to other tools or filters, I offer specific, actionable examples that can help the functionality of QSVEnc. For instance, when I mentioned the FFT3D filter, it was to illustrate a particular feature or capability that could be beneficial if implemented within QSVEnc. This differs from how it could be perceived as suggesting users look elsewhere; my goal is always to provide ideas that can be integrated into QSVEnc to make it even better.

By focusing on these principles, I strive to ensure that my contributions benefit and support the ongoing development of QSVEnc.

Thank you for understanding, and I hope we can all contribute positively to the development and improvement of QSVEnc.

[MarcoRavich]

Dear @forart,

I want you to know that I value your participation and willingness to offer suggestions on this platform. However, in my culture, your approach may be seen as dismissive and disrespectful to the developers and other contributors.

Thanks for verbose & detailed explaination.

I basically agree: for people (like me) that can't materially contribute with code "Issues" is not the right place to push suggestions.

So, from now on, i'll refrain from any contribution until the "Discussion" section will be available.

Happy coding.

[quamt]

Dear @forart,

Thank you for understanding and for reading my message.
I appreciate you being willing to take the feedback to heart.

I want to ensure you know your ideas and suggestions are valuable.
The goal wasn't to discourage you from participating but to help ensure that all feedback in the Issues section is helpful and constructive.

Your interest in improving QSVEnc is excellent.
While the Issues section is mainly for bugs and specific feature requests, your ideas can still be very helpful if they are exact and directly related to current issues.

If a Discussions section is added in the future, that would be a perfect place for more general ideas and suggestions.

Thanks again for your understanding and support.
We're all trying to make QSVEnc better together.

Happy coding!

[rigaya]

--vpp-fft3d has been added in QSVEnc 7.66.

Please check the link for details.

I think the performance of the filter is acceptable, as it run around 145fps on Arc A380 with default parameters with 1080p target after I did some optimization (first implementation only run around 45fps...), but please have a test.

[quamt]

@rigaya

I am sincerely grateful for implementing the --vpp-fft3d filter in QSVEnc 7.66.
I have tested it on some old footage, and it works fantastically!
The filter effectively removes grain and noise, and the performance is impressive.
Your optimizations have made a significant difference.

Thank you again for your hard work and dedication.
It is much appreciated!

[rigaya]

Thank you for testing, nice to hear hat it works fine!

[quamt]

Could you tell me which settings you use for testing 1080p?
When I leave it on standard and convert a 1080p to HEVC 10-Bit, CQP, I get around 260 fps.

[rigaya]

I left it all default for the test at Arc A380.

[quamt]

Ok. I did a re-test with the Big Buck Bunny
https://download.blender.org/demo/movies/BBB/

It is nearly 400 fps

If I calculated correctly, the A770 is approximately 167.8% faster.

[quamt]

Great speed and excellent quality :-)

[rigaya]

That's great! The strong performance of Arc A770 (compared to A380) really helps with filters like fft3d or nlmeans with somewhat high computational load.