Quantized KV Cache? #595
bkaruman
started this conversation in
New features / APIs
Quantized KV Cache?
#595
Replies: 2 comments
-
|
Any update on this ? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Out of interest. What is quantized KV Caching? Which model is it for? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi! Is there a plan to support quantized KV Cache?
Beta Was this translation helpful? Give feedback.
All reactions