Add note on lack of ensemble support (#6648)

triton-inference-server · Dec 5, 2023 · 4ac7f37 · 4ac7f37
1 parent b44ee7c
commit 4ac7f37
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/docs/user_guide/response_cache.md b/docs/user_guide/response_cache.md
@@ -237,3 +237,7 @@ the cache.
   response caching.
 - The response cache does not currently support
   [decoupled models](decoupled_models.md).
+- Top-level requests to ensemble models do not currently support response
+  caching. However, composing models within an ensemble may have their
+  responses cached if supported and enabled by that composing model.
+