Skip to content

Commit

Permalink
Update caching.mdx
Browse files Browse the repository at this point in the history
  • Loading branch information
kathayl authored Aug 14, 2024
1 parent bfad1d4 commit df06cdb
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions src/content/docs/ai-gateway/configuration/caching.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,8 @@ Enable and customize your gateway cache to serve requests directly from Cloudfla
:::note


Caching currently only works for identical requests. This is helpful for use cases when there are limited prompt options - for example, a support bot that asks "How can I help you?" and lets the user select an answer from a limited set of options works well with the current caching configuration.
Currently caching is supported only for text and image responses, and it applies only to identical requests. This is helpful for use cases when there are limited prompt options - for example, a support bot that asks "How can I help you?" and lets the user select an answer from a limited set of options works well with the current caching configuration.
We plan on adding semantic search for caching in the future to improve cache hit rates.

Additionally, caching is supported only for text and image responses, so other types of responses will not benefit from caching.
:::

## Default configuration
Expand Down

0 comments on commit df06cdb

Please sign in to comment.