HPU only release binary #302
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
wenhuach21
reviewed
Nov 5, 2024
test/test_auto_round_hpu_only.py
Outdated
| @@ -0,0 +1,3 @@ | |||
| def test_import(): | |||
| from auto_round import AutoRound | |||
| from auto_round.export.export_to_itrex.export import export_to_itrex | |||
There was a problem hiding this comment.
@WeiweiZhang1 the import is very confusing, better rename one of them
Signed-off-by: Sun, Xuehao <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
yiliu30
commented
Nov 7, 2024
wenhuach21
reviewed
Nov 7, 2024
auto_round/export/__init__.py
Outdated
| from .export_to_autoround.export import save_quantized_as_autoround | ||
| from .export_to_awq.export import save_quantized_as_autoawq | ||
| from auto_round.utils import LazyImport | ||
| save_quantized_as_autogptq = LazyImport("auto_round.export.export_to_autogptq.export.save_quantized_as_autogptq") |
There was a problem hiding this comment.
a little ugly if others see the code, how about first exporting them in auto_round.export, then we could just call auto_round.export.save_xxx
There was a problem hiding this comment.
Update by registering all export formats in __init__.py, and only invoke the save function when necessary.
wenhuach21
reviewed
Nov 7, 2024
auto_round/export/__init__.py
Outdated
| from auto_round.utils import LazyImport | ||
| save_quantized_as_autogptq = LazyImport("auto_round.export.export_to_autogptq.export.save_quantized_as_autogptq") | ||
| save_quantized_as_itrex = LazyImport("auto_round.export.export_to_itrex.export.save_quantized_as_itrex") | ||
| QuantConfig = LazyImport("auto_round.export.export_to_itrex.config.QuantConfig") |
There was a problem hiding this comment.
Better change the quantconfig to itrex_quantconfig
There was a problem hiding this comment.
Since it's specific to the export format, remove it from __init__.py and use it with the full path: from auto_round.export.export_to_itrex import QuantConfig.
wenhuach21
reviewed
Nov 7, 2024
auto_round/auto_quantizer.py
Outdated
| from auto_round.utils import LazyImport | ||
| qlinear_qbits = LazyImport("auto_round_extension.qbits.qlinear_qbits") | ||
| qlinear_qbits_gptq = LazyImport("auto_round_extension.qbits.qlinear_qbits_gptq") | ||
| qlinear_ipex_gptq = LazyImport("auto_round_extension.ipex.qlinear_ipex_gptq") |
There was a problem hiding this comment.
we have no unit test for generation due to several reasons. Could you help have a test to make sure the change in this file is ok?
There was a problem hiding this comment.
Update to import them only when needed.
There was a problem hiding this comment.
Local test results:
python main.py --format auto_round:gptq --task wikitext
python main.py --format auto_gptq --task wikitext
python main.py --format auto_round --task wikitext
| Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr|
|--------|------:|------|-----:|---------------|---|------:|---|------|
|wikitext| 2|none | 0|bits_per_byte |↓ | 0.9433|± |N/A |
| | |none | 0|byte_perplexity|↓ | 1.9229|± |N/A |
| | |none | 0|word_perplexity|↓ |32.9958|± |N/A |
| Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr|
|--------|------:|------|-----:|---------------|---|------:|---|------|
|wikitext| 2|none | 0|bits_per_byte |↓ | 0.9433|± |N/A |
| | |none | 0|byte_perplexity|↓ | 1.9229|± |N/A |
| | |none | 0|word_perplexity|↓ |32.9958|± |N/A |
| Tasks |Version|Filter|n-shot| Metric | | Value | |Stderr|
|--------|------:|------|-----:|---------------|---|------:|---|------|
|wikitext| 2|none | 0|bits_per_byte |↓ | 0.9433|± |N/A |
| | |none | 0|byte_perplexity|↓ | 1.9229|± |N/A |
| | |none | 0|word_perplexity|↓ |32.9958|± |N/A |Signed-off-by: Sun, Xuehao <[email protected]>
|
/azp run Unit-Test-HPU-AutoRound |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: Sun, Xuehao <[email protected]>
Signed-off-by: Sun, Xuehao <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
Signed-off-by: yiliu30 <[email protected]>
yiliu30
commented
Nov 11, 2024
wenhuach21
approved these changes
Nov 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The expected behavior is to build a binary for HPU-only, as we did for INC 3x intel/neural-compressor#1336.
Usage
pip install auto-round[hpu] # It will install auto-round with `requirements-hpu.txt`python setup.py install hpu # Within the gaudi docker, install hpu-only version by default python setup.py installcc @thuang6
Test scope:
To ensure that new PRs don't break this version, we need to define a set of smoke tests for it. Additionally, we need a runner that installs the HPU-only version and runs these tests.