Support Mixtral quantization using HQT#67

Closed

dudilester wants to merge 24 commits intohabana_mainfrom dev/dlester/mixtral_hqt

+688-128

Commits on Jun 19, 2024

Move model to hpu when HQT is not used
nirda7
committed

Commits on Jun 21, 2024

fix CR comments
nirda7
committed

Commits on Jun 22, 2024

add model weights device load
nirda7
committed

Commits on Jun 26, 2024

Commits on Jun 27, 2024

pass optimizations flags only in Lazy mode
nirda7
committed

Commits on Jun 28, 2024

Merge remote-tracking branch 'origin/habana_next' into vllm-hqt-fork
madamczykhabana
committed
Filter-out warmup_mode before passing to model.forward
madamczykhabana
committed
Merge pull request #75 from HabanaAI/vllm-hqt-fork
madamczykhabana
authored

Commits on Jul 1, 2024

Profile single forward (#68 )
adobrzyniewicz-habana
authored
Skip logprobs processing for greedy

Krzysztof Laskowski
authored and
madamczykhabana
committed

Commits on Jul 2, 2024

Commits on Jul 4, 2024

Commits on Jul 5, 2024

Revert "Revert "Disable tensor cache set to True (#88 )" (#89 )" (#90 )
michalkuligowski
authored

Commits on Jul 7, 2024

Support Mixtral quantization using HQT
dudilester
committed