[DOC]Add modelscope example #1578
Conversation
Signed-off-by: intellinjun <[email protected]>
⚡ Required checks status: All passing 🟢Groups summary🟢 Format Scan Tests workflow
These checks are required after the changes to 🟢 Optimize Unit Test workflow
These checks are required after the changes to 🟢 NeuralChat Unit Test
These checks are required after the changes to 🟢 Engine Unit Test workflow
These checks are required after the changes to 🟢 Chat Bot Test workflow
These checks are required after the changes to Thank you for your contribution! 💜
|
Signed-off-by: intellinjun <[email protected]>
Signed-off-by: intellinjun <[email protected]>
examples/modelscope/README.md
Outdated
| @@ -0,0 +1,24 @@ | |||
| # ModelScope with ITREX | |||
|
|
|||
| Intel extension for transformers(ITREX) support almost all the LLMs in Pytorch format from ModelScope such as phi,Qwen,ChatGLM,Baichuan,gemma,etc. | |||
There was a problem hiding this comment.
phi,Qwen,ChatGLM,Baichuan,gemma,etc.
phi, Qwen, ChatGLM, Baichuan, gemma, etc.
Signed-off-by: intellinjun <[email protected]>
examples/modelscope/README.md
Outdated
|
|
||
| ITREX provides a script that demonstrates the use of modelscope. Run it with the following command: | ||
| ```bash | ||
| numactl -m 0 -C 0-55 python run_modelscope_example.py --model_path=qwen/Qwen-7B --prompt=你好 |
There was a problem hiding this comment.
numactl -l -C xx-xx? if user across sockets?
Add a note explaining why adding numactl is necessary (to improve performance and teach them how to bind core_id).
examples/modelscope/README.md
Outdated
|
|
||
| ITREX provides a script that demonstrates the use of modelscope. Run it with the following command: | ||
| ```bash | ||
| numactl -m 0 -C 0-55 python run_modelscope_example.py --model_path=qwen/Qwen-7B --prompt=你好 |
There was a problem hiding this comment.
numactl -m 0 -C 0-55 python run_modelscope_example.py --model_path=qwen/Qwen-7B --prompt=你好
change to
OMP_NUM_THREADS= numactl -m -C python run_modelscope_example.py
--model <MODEL_NAME_OR_PATH>
--prompt=你好
| ``` | ||
|
|
||
| ## Supported and Validated Models | ||
| We have validated the majority of existing models using modelscope==1.13.1: |
There was a problem hiding this comment.
please add the requirment.txt
examples/modelscope/README.md
Outdated
| @@ -0,0 +1,24 @@ | |||
| # ModelScope with ITREX | |||
|
|
|||
| Intel extension for transformers(ITREX) support almost all the LLMs in Pytorch format from ModelScope such as phi, Qwen, ChatGLM, Baichuan, gemma, etc. | |||
There was a problem hiding this comment.
Intel extension for transformers
Intel® Extension for Transformers
Signed-off-by: intellinjun <[email protected]>
Signed-off-by: intellinjun <[email protected]>
Signed-off-by: intellinjun <[email protected]>
Type of Change
feature or bug fix or documentation or others
API changed or not
not
Description
Add modelscope example