Intel® Low Precision Optimization Toolkit (iLiT) v1.0 Alpha Release

Intel® Low Precision Optimization Tool (iLiT) is an open-sourced python library which is intended to deliver a unified low-precision inference solution cross multiple Intel optimized DL frameworks on both CPU and GPU. It supports automatic accuracy-driven tuning strategies, along with additional objectives like performance, model size, or memory footprint. It also provides the easy extension capability for new backends, tuning strategies, metrics and objectives.

Feature List:

• Unified low precision quantization interface cross multiple Intel optimized frameworks (TensorFlow, PyTorch, and MXNet)
• Built-in tuning strategies, including Basic, Bayesian, and MSE
• Built-in evaluation metrics, including TopK (image classification), F1 (NLP), and CocoMAP (object detection)
• Built-in tuning objectives, including Performance, ModelSize, and Footprint
• Extensible API design to add new strategy, framework backend, metric, and objective
• KL-divergence calibration for TensorFlow and MXNet
• Tuning process resume from certain checkpoint

Supported Models:

Model	Framework	Model	Framework	Model	Framework
ResNet50 V1	MXNet	BERT-Large RTE	PyTorch	ResNet18	PyTorch
MobileNet V1	MXNet	BERT-Large QNLI	PyTorch	ResNet50 V1	TensorFlow
MobileNet V2	MXNet	BERT-Large CoLA	PyTorch	ResNet50 V1.5	TensorFlow
SSD-ResNet50	MXNet	BERT-Base SST-2	PyTorch	ResNet101	TensorFlow
SqueezeNet V1	MXNet	BERT-Base RTE	PyTorch	Inception V1	TensorFlow
ResNet18	MXNet	BERT-Base STS-B	PyTorch	Inception V2	TensorFlow
Inception V3	MXNet	BERT-Base CoLA	PyTorch	Inception V3	TensorFlow
DLRM	PyTorch	BERT-Base MRPC	PyTorch	Inception V4	TensorFlow
BERT-Large MRPC	PyTorch	ResNet101	PyTorch	Inception ResNet V2	TensorFlow
BERT-Large SQUAD	PyTorch	ResNet50 V1.5	PyTorch	SSD ResNet50 V1	TensorFlow

Known Issues:

• Statistics collection for KL algorithm is slow in TensorFlow due to lack of tensor inspector APIs
• MSE tuning strategy is not supported in PyTorch

Validated Configurations:

• Python 3.6 & 3.7
• Centos 7
• TensorFlow 1.15, 2.0 and 2.1
• PyTorch 1.5
• MxNet 1.6

Distribution:

	Channel	Links	Install Command
Source	Github	https://github.com/intel/lp-opt-tool.git	$ git clone https://github.com/intel/lp-opt-tool.git
Binary	Pip	https://pypi.org/project/ilit	$ pip install ilit
Binary	Conda	https://anaconda.org/intel/ilit	$ conda config --add channels intel $ conda install ilit

Contact:

Please feel free to contact [email protected], if you get any questions.