Combination of --avsw, --pa [preanalysis] and --vpp-resize when using amf specific scalers causes video to not encode.

[DimkaTsv]

Full details here:
GPUOpen-LibrariesAndSDKs/AMF#456

Thought it was AMF issue during original testing, but after expanding on it discovered that problem lies mainly in --avsw interaction and --pa, when --vpp-resize [amf_bilinear / amf_bicubic / amf_fsr] scaler is being utilized.

How trace level log looks like:

[AMFPreAnalysisImpl] 2024-02-21 20:06:30.022     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.023     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.025     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.026     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.028     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.029     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.031     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.033     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.034     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.036     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.038     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.039     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.041     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.042     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()
[AMFPreAnalysisImpl] 2024-02-21 20:06:30.044     5500 [AMFPreAnalysisImpl]   Debug: AMFPreAnalysisImpl::QueryOutput()

---

Also... Side question @rigaya . Is there any particular reason that AV1 codec forces --avsw flag to decoder on RX 7000 GPU's? (and RX 6000 GPU's as well, i assume?). Because HW decoder is capable for decoding AV1 on these GPU's. Or was that done to increase general compatability with older GPU while using autoconfiguration of non-specified parameters?

[rigaya]

Thank you for the details.

This problem can now be avoided by a workaround in VCEEnc 8.23.

It seems like that it gets stuck(freezed) when calling QueryOutput of HQScaler, when PreAnalysis is running and the input frame is coming from avsw via host memory. I found out that by explicitly converting the frame in host memory to device memory, the problem can be avoided ( 342b394 ).

[rigaya]

Is there any particular reason that AV1 codec forces --avsw flag to decoder on RX 7000 GPU's?

Yes. Currently, I have problem that AV1 hw decoder crushes when sending input bitstream (when calling SubmitInput). As I'm not sure of the cause, I am have AV1 hw decoder to be disabled.

[DimkaTsv]

Can confirm fix, issue closed.