any plan to use the KV-cache? #141
JasonGanggg
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
why not use the KV cache in current code? and any plan to use the KV cache in future? how much improvement do you think KV cache can make?
Beta Was this translation helpful? Give feedback.
All reactions