Advance upstreams to 25.02

[nv-kmcgill53]

No description provided.

[mc-nv]

Why do we need to install OpenVINO for ONNXRuntime?

[nv-kmcgill53]

Why do we need to install OpenVINO for ONNXRuntime?

Onnxruntime has a concept of execution providers (EPs) and there are two we have historically supported: OpenVINO Execution Provider and the TensorRT Execution Provider. Since we install the OpenVINO backend in our container, we thought it would be a good idea not to limit our users when using the OpenVINO EP with ONNXRuntime.

[mc-nv]

Why do we need to install OpenVINO for ONNXRuntime?

Onnxruntime has a concept of execution providers (EPs) and there are two we have historically supported: OpenVINO Execution Provider and the TensorRT Execution Provider. Since we install the OpenVINO backend in our container, we thought it would be a good idea not to limit our users when using the OpenVINO EP with ONNXRuntime.

My concern here is that we may making a double work.
OpenVINO contributes into ONNX Runtime same as to Triton Inference Server.
Scenario that ONNX Runtime uses for OpenVINO are related to headers it will obtain during the compilation.
https://github.com/microsoft/onnxruntime/blob/main/cmake/onnxruntime_providers_openvino.cmake#L6-L11

Unless we start pulling openvino library alongside with onnxruntime library it could be a redundant work.
Note I didn't have time to make a full review and ananlizes and not going to block this PR.

[nv-kmcgill53]

My concern here is that we may making a double work.
OpenVINO contributes into ONNX Runtime same as to Triton Inference Server.
Scenario that ONNX Runtime uses for OpenVINO are related to headers it will obtain during the compilation.
https://github.com/microsoft/onnxruntime/blob/main/cmake/onnxruntime_providers_openvino.cmake#L6-L11

This issue has been around for as long as I can remember. Maybe @tanmayv25 can give more context on this, but the way I understand this part of the code is: We ran into an issue where the versions of openvino needed to match between the ORT openvino EP and the openvino_backend itself, otherwise we would have conflicting installs. We also have the difference where we build openvino from source in the openvino backend, and we use the prebuilt openvino libs for the ORT openvino EP. IIRC this is because we invoked some flags on the openvino build to enable certain compiler optimizations we were taking advantage of. All of this I am recalling is from 2+ years ago, so the ecosystem may have changed a lot since I looked deep into this.

[nv-kmcgill53]

Note I didn't have time to make a full review and ananlizes and not going to block this PR.

Please change your status from "Requesting Changes" to "Approved."

[mc-nv]

Note I didn't have time to make a full review and ananlizes and not going to block this PR.

Please change your status from "Requesting Changes" to "Approved."

It's approved but I think we need a follow up ticket in JIRA to review it usage, looks like it could be a redundant step.

[nv-kmcgill53]

I think we need a follow up ticket in JIRA to review it usage

Created ticket TPRD-1214