Skip to content
This repository has been archived by the owner on Aug 30, 2024. It is now read-only.

Enable Mixtral8x7B #138

Merged
merged 38 commits into from
Mar 4, 2024
Merged
Changes from 1 commit
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
8905a97
fix top-k and argsort error
intellinjun Feb 7, 2024
cbe21a9
enable mistral8X7b f32 gguf
intellinjun Feb 21, 2024
692e2d0
enable moe jblas
intellinjun Feb 22, 2024
f285b1f
add write and read n_experts parameter
intellinjun Feb 23, 2024
299a996
Merge branch 'main' into mixtral
intellinjun Feb 23, 2024
ce00092
Update __init__.py
intellinjun Feb 23, 2024
a698dac
enable q40
intellinjun Feb 23, 2024
b2fd598
Merge branch 'mixtral' of https://github.com/intel/neural-speed into …
intellinjun Feb 23, 2024
ffdd11a
fix format error
intellinjun Feb 23, 2024
2abcb14
enable mixtral8x7b from hf to bin
intellinjun Feb 23, 2024
9f56f45
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 23, 2024
4a01f65
fix llama load error
intellinjun Feb 23, 2024
38a368b
Merge branch 'mixtral' of https://github.com/intel/neural-speed into …
intellinjun Feb 23, 2024
28bc0db
fix format error
intellinjun Feb 26, 2024
932ee25
Update llama.cpp
intellinjun Feb 26, 2024
41606f6
fix convert and format error
intellinjun Feb 27, 2024
5454e1f
Merge branch 'mixtral' of https://github.com/intel/neural-speed into …
intellinjun Feb 27, 2024
00c4ff6
Merge branch 'main' into mixtral
intellinjun Feb 27, 2024
3857b5b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 27, 2024
e4260dd
Update convert_quantized_llama.py
intellinjun Feb 27, 2024
a88ac05
add extension test for mixtral
intellinjun Feb 27, 2024
13999fc
Merge branch 'mixtral' of https://github.com/intel/neural-speed into …
intellinjun Feb 27, 2024
1ea128e
update argsort
intellinjun Feb 28, 2024
e27880c
Update argsort.cpp
intellinjun Feb 28, 2024
e6f2fbf
Update ne_layers.c
intellinjun Feb 29, 2024
cad4114
fix format error
intellinjun Feb 29, 2024
f8cd5a2
Merge branch 'mixtral' of https://github.com/intel/neural-speed into …
intellinjun Feb 29, 2024
32a951d
fix format error
intellinjun Feb 29, 2024
db3b00d
Update ne_layers.c
intellinjun Feb 29, 2024
7f9fcf8
add mul_id_ffn_fusion
intellinjun Feb 29, 2024
24156e0
Merge branch 'mixtral' of https://github.com/intel/neural-speed into …
intellinjun Feb 29, 2024
7ef7a2e
fix compile error
intellinjun Feb 29, 2024
9e76221
fix compile error
intellinjun Feb 29, 2024
9d6c93c
amend function name
intellinjun Feb 29, 2024
5e474d0
Update convert_mixtral.py
intellinjun Feb 29, 2024
c32c991
Update CMakeLists.txt
intellinjun Feb 29, 2024
affe69c
fix mixtral_q40 multi thread error
intellinjun Mar 1, 2024
590d65b
add ffn silu support assert
intellinjun Mar 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update convert_mixtral.py
intellinjun authored Feb 29, 2024

Verified

This commit was signed with the committer’s verified signature.
bkueng Beat Küng
commit 5e474d0a530bbec0487eb0488801b5619325c2e0
3 changes: 0 additions & 3 deletions neural_speed/convert/convert_mixtral.py
Original file line number Diff line number Diff line change
@@ -1095,9 +1095,6 @@ def write_tensor_header(self, name: str, shape: Sequence[int], data_type: DataTy
def write_vocab(self, vocab: Vocab) -> None:
for text, score in vocab.all_tokens():
self.fout.write(struct.pack("i", len(text)))
print(len(text))
if(len(text)>100):
break
self.fout.write(text)
self.fout.write(struct.pack("f", score))