Skip to content

Commit

Permalink
update README and remove beta status
Browse files Browse the repository at this point in the history
  • Loading branch information
dtrawins committed Dec 19, 2024
1 parent 0a3fb4c commit 8c7a65c
Showing 1 changed file with 10 additions and 7 deletions.
17 changes: 10 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,20 +30,18 @@

# OpenVINO Backend

**Note: OpenVINO backend is beta quality. As a result you may
encounter performance and functional issues that will be resolved in
future releases.**
**Note: OpenVINO backend is now considered of general availability. It is co-maintained by Nvidia and Intel.**

The Triton backend for the
[OpenVINO](https://docs.openvinotoolkit.org/latest/index.html). You
[OpenVINO](https://docs.openvino.ai/). You
can learn more about Triton backends in the [backend
repo](https://github.com/triton-inference-server/backend). Ask
questions or report problems in the main Triton [issues
page](https://github.com/triton-inference-server/server/issues). The backend
is designed to run models in Intermediate Representation (IR). See [here](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model.html) for instruction to convert a model to IR format. The backend is implemented using openVINO C++ API. Auto completion of the model config is not supported in the backend and complete `config.pbtxt` must be provided with the model.
is designed to run models in Intermediate Representation (IR), TensorFlow saved_model, TensorFlow Lite, ONNX and PaddlePaddle. PyTorch models can be used after converting to IR or ONNX. See [here](https://docs.openvino.ai/2024/openvino-workflow/model-preparation/convert-model-pytorch.html) for instruction. The backend is implemented using openVINO C++ API.

## Supported Devices
OpenVINO backend currently supports inference only on Intel CPU devices using [OpenVINO CPU plugin](https://docs.openvinotoolkit.org/latest/openvino_docs_IE_DG_supported_plugins_CPU.html). Note the CPU plugin does not support
OpenVINO backend in the public docker image version currently supports inference only on Intel CPU devices using [OpenVINO CPU plugin](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device.html). Note the CPU plugin does not support
iGPU.

## Build the OpenVINO Backend
Expand Down Expand Up @@ -232,7 +230,12 @@ string_value:"yes"
}
```

Check also the [Quick deploy guide](https://github.com/triton-inference-server/tutorials/tree/main/Quick_Deploy/OpenVINO).

Examples of the supported models and configs are included in the [functional tests](tests).

## Known Issues

* Models with the scalar on the input (shape without any dimension are not supported)
* Reshaping using [dimension ranges](https://docs.openvino.ai/2023.3/ovms_docs_dynamic_shape_dynamic_model.html) is not supported.
* Reshaping using [dimension ranges](https://docs.openvino.ai/2024/openvino-workflow/model-server/ovms_docs_dynamic_shape_dynamic_model.html) is not supported.
* Models without output names are not supported. Models must be saved with names assigned.

0 comments on commit 8c7a65c

Please sign in to comment.