Add a whisper example using Intel Neural Compressor #412
Conversation
| @@ -4,6 +4,7 @@ This folder contains a sample use case of Olive to optimize a [Whisper](https:// | |||
| Performs optimization pipeline: | |||
| - CPU, FP32: *PyTorch Model -> Onnx Model -> Transformers Optimized Onnx Model -> Insert Beam Search Op -> Insert Pre/Post Processing Ops* | |||
| - CPU, INT8: *PyTorch Model -> Onnx Model -> Transformers Optimized Onnx Model -> Dynamic Quantized Onnx Model -> Insert Beam Search Op -> Insert Pre/Post Processing Ops* | |||
| - CPU, INT8: *PyTorch Model -> Onnx Model -> Transformers Optimized Onnx Model -> Intel® Neural Compressor Dynamic Quantized Onnx Model -> Insert Beam Search Op -> Insert Pre/Post Processing Ops* | |||
There was a problem hiding this comment.
It is better to tell user to install olive-ai[inc] if they run this optimization chain.
IIRC, whisper is in our CI pipeline, you may need to add neural-compressor into whisper requirements.txt.
|
/azp run |
|
No commit pushedDate could be found for PR 412 in repo microsoft/Olive |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
|
|
Signed-off-by: yuwenzho <[email protected]>
Signed-off-by: yuwenzho <[email protected]>
Signed-off-by: yuwenzho <[email protected]>
Signed-off-by: yuwenzho <[email protected]>
yuwenzho
force-pushed
the
yuwenzho/inc_whisper_example
branch
from
955395a to
f2fefec
Compare
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Oh, yes, we can skip the inc_int8 test on windows. Please add follow code snipped under import platform
if platform.system() == "Windows" and device_precision[1].startswith("inc_"):
pytest.skip("Skip test on Windows. neural-compressor import is hanging on Windows.") |
| @@ -29,7 +29,7 @@ def setup(): | |||
| sys.path.remove(example_dir) | |||
|
|
|||
|
|
|||
| @pytest.mark.parametrize("device_precision", [("cpu", "fp32"), ("cpu", "int8")]) | |||
| @pytest.mark.parametrize("device_precision", [("cpu", "fp32"), ("cpu", "int8"), ("cpu", "inc_int8")]) | |||
| def test_whisper(device_precision): | |||
| from olive.workflows import run as olive_run | |||
There was a problem hiding this comment.
To skip test on windows.
import platform
if platform.system() == "Windows" and device_precision[1].startswith("inc_"):
pytest.skip("Skip test on Windows. neural-compressor import is hanging on Windows.")Signed-off-by: yuwenzho <[email protected]>
Signed-off-by: yuwenzho <[email protected]>
|
/azp run |
|
No commit pushedDate could be found for PR 412 in repo microsoft/Olive |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
Describe your changes
Add a whisper example using Intel® Neural Compressor Quantization.
Checklist before requesting a review
pre-commit run --all-files(Optional) Issue link