Skip to content

Commit

Permalink
Llama3 and Phi3 validation results update
Browse files Browse the repository at this point in the history
  • Loading branch information
yao531441 committed May 28, 2024
1 parent 7a3f115 commit 33d31b8
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions comps/llms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,15 @@ curl http://${your_ip}:9000/v1/chat/completions\
-d '{"query":"What is Deep Learning?","max_new_tokens":17,"top_k":10,"top_p":0.95,"typical_p":0.95,"temperature":0.01,"repetition_penalty":1.03,"streaming":true}' \
-H 'Content-Type: application/json'
```


## Validated Model

| Model | TGI-Gaudi | vLLM-CPU | Ray |
|---------------------------| --------- |----------| --- |
| Intel/neural-chat-7b-v3-3 ||||
| Llama-2-7b-chat-hf ||||
| Llama-2-70b-chat-hf || - | x |
| Meta-Llama-3-8B-Instruct ||||
| Meta-Llama-3-70B-Instruct || - | x |
| Phi-3 | x | Limit 4K ||

0 comments on commit 33d31b8

Please sign in to comment.