From 3e1b9d4a19de6773147291d0b4a9080b1947026c Mon Sep 17 00:00:00 2001 From: "chen, suyue" Date: Fri, 15 Sep 2023 15:53:45 +0800 Subject: [PATCH] Update readme for v2.3 release (#1258) Signed-off-by: chensuyue Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- README.md | 18 +++++++++--------- docs/source/imgs/wechat_group.jpg | Bin 0 -> 29155 bytes docs/source/installation_guide.md | 23 ++++++++++++----------- docs/source/publication_list.md | 5 +++-- docs/source/quantization.md | 9 ++++++++- third-party-programs.txt | 17 +++++++++++------ 6 files changed, 43 insertions(+), 29 deletions(-) create mode 100644 docs/source/imgs/wechat_group.jpg diff --git a/README.md b/README.md index 30767fd0d95..1c194de539f 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ Intel® Neural Compressor

An open-source Python library supporting popular model compression techniques on all mainstream deep learning frameworks (TensorFlow, PyTorch, ONNX Runtime, and MXNet)

[![python](https://img.shields.io/badge/python-3.7%2B-blue)](https://github.com/intel/neural-compressor) -[![version](https://img.shields.io/badge/release-2.2-green)](https://github.com/intel/neural-compressor/releases) +[![version](https://img.shields.io/badge/release-2.3-green)](https://github.com/intel/neural-compressor/releases) [![license](https://img.shields.io/badge/license-Apache%202-blue)](https://github.com/intel/neural-compressor/blob/master/LICENSE) [![coverage](https://img.shields.io/badge/coverage-85%25-green)](https://github.com/intel/neural-compressor) [![Downloads](https://static.pepy.tech/personalized-badge/neural-compressor?period=total&units=international_system&left_color=grey&right_color=green&left_text=downloads)](https://pepy.tech/project/neural-compressor) @@ -21,9 +21,9 @@ In particular, the tool provides the key features, typical examples, and open co * Support a wide range of Intel hardware such as [Intel Xeon Scalable processor](https://www.intel.com/content/www/us/en/products/details/processors/xeon/scalable.html), [Intel Xeon CPU Max Series](https://www.intel.com/content/www/us/en/products/details/processors/xeon/max-series.html), [Intel Data Center GPU Flex Series](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/flex-series.html), and [Intel Data Center GPU Max Series](https://www.intel.com/content/www/us/en/products/details/discrete-gpus/data-center-gpu/max-series.html) with extensive testing; support AMD CPU, ARM CPU, and NVidia GPU through ONNX Runtime with limited testing -* Validate more than 10,000 models such as [Bloom-176B](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), [OPT-6.7B](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), [Stable Diffusion](/examples/pytorch/nlp/huggingface_models/text-to-image/quantization), [GPT-J](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/fx), [BERT-Large](/examples/pytorch/nlp/huggingface_models/text-classification/quantization/ptq_static/fx), and [ResNet50](/examples/pytorch/image_recognition/torchvision_models/quantization/ptq/cpu/fx) from popular model hubs such as [Hugging Face](https://huggingface.co/), [Torch Vision](https://pytorch.org/vision/stable/index.html), and [ONNX Model Zoo](https://github.com/onnx/models#models), by leveraging zero-code optimization solution [Neural Coder](/neural_coder#what-do-we-offer) and automatic [accuracy-driven](/docs/source/design.md#workflow) quantization strategies +* Validate popular LLMs such as LLama2, [LLama](examples/onnxrt/nlp/huggingface_model/text_generation/llama/quantization/ptq_static), [MPT](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/text-generation/quantization/README.md), [Falcon](https://github.com/intel/intel-extension-for-transformers/blob/main/examples/huggingface/pytorch/language-modeling/quantization/README.md), [GPT-J](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/fx), [Bloom](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), [OPT](/examples/pytorch/nlp/huggingface_models/language-modeling/quantization/ptq_static/ipex/smooth_quant), and more than 10,000 broad models such as [Stable Diffusion](/examples/pytorch/nlp/huggingface_models/text-to-image/quantization), [BERT-Large](/examples/pytorch/nlp/huggingface_models/text-classification/quantization/ptq_static/fx), and [ResNet50](/examples/pytorch/image_recognition/torchvision_models/quantization/ptq/cpu/fx) from popular model hubs such as [Hugging Face](https://huggingface.co/), [Torch Vision](https://pytorch.org/vision/stable/index.html), and [ONNX Model Zoo](https://github.com/onnx/models#models), by leveraging zero-code optimization solution [Neural Coder](/neural_coder#what-do-we-offer) and automatic [accuracy-driven](/docs/source/design.md#workflow) quantization strategies -* Collaborate with cloud marketplace such as [Google Cloud Platform](https://console.cloud.google.com/marketplace/product/bitnami-launchpad/inc-tensorflow-intel?project=verdant-sensor-286207), [Amazon Web Services](https://aws.amazon.com/marketplace/pp/prodview-yjyh2xmggbmga#pdp-support), and [Azure](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/bitnami.inc-tensorflow-intel), software platforms such as [Alibaba Cloud](https://www.intel.com/content/www/us/en/developer/articles/technical/quantize-ai-by-oneapi-analytics-on-alibaba-cloud.html) and [Tencent TACO](https://new.qq.com/rain/a/20221202A00B9S00), and open AI ecosystem such as [Hugging Face](https://huggingface.co/blog/intel), [PyTorch](https://pytorch.org/tutorials/recipes/intel_neural_compressor_for_pytorch.html), [ONNX](https://github.com/onnx/models#models), and [Lightning AI](https://github.com/Lightning-AI/lightning/blob/master/docs/source-pytorch/advanced/post_training_quantization.rst) +* Collaborate with cloud marketplace such as [Google Cloud Platform](https://console.cloud.google.com/marketplace/product/bitnami-launchpad/inc-tensorflow-intel?project=verdant-sensor-286207), [Amazon Web Services](https://aws.amazon.com/marketplace/pp/prodview-yjyh2xmggbmga#pdp-support), and [Azure](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/bitnami.inc-tensorflow-intel), software platforms such as [Alibaba Cloud](https://www.intel.com/content/www/us/en/developer/articles/technical/quantize-ai-by-oneapi-analytics-on-alibaba-cloud.html), [Tencent TACO](https://new.qq.com/rain/a/20221202A00B9S00) and [Microsoft Olive](https://github.com/microsoft/Olive), and open AI ecosystem such as [Hugging Face](https://huggingface.co/blog/intel), [PyTorch](https://pytorch.org/tutorials/recipes/intel_neural_compressor_for_pytorch.html), [ONNX](https://github.com/onnx/models#models), [ONNX Runtime](https://github.com/microsoft/onnxruntime), and [Lightning AI](https://github.com/Lightning-AI/lightning/blob/master/docs/source-pytorch/advanced/post_training_quantization.rst) ## Installation @@ -120,7 +120,7 @@ q_model = fit( SmoothQuant - Weight-Only Quantization + Weight-Only Quantization (INT8/INT4/FP4/NF4) @@ -139,10 +139,9 @@ q_model = fit( > More documentations can be found at [User Guide](./docs/source/user_guide.md). ## Selected Publications/Events +* arXiv: [Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs](https://arxiv.org/abs/2309.05516) (Sep 2023) * Post on Social Media: [ONNXCommunityMeetup2023: INT8 Quantization for Large Language Models with Intel Neural Compressor](https://www.youtube.com/watch?v=luYBWA1Q5pQ) (July 2023) * Blog by Intel: [Accelerate Llama 2 with Intel AI Hardware and Software Optimizations](https://www.intel.com/content/www/us/en/developer/articles/news/llama2.html) (July 2023) -* Blog on Medium: [Quantization Accuracy Loss Diagnosis with Neural Insights](https://medium.com/@NeuralCompressor/quantization-accuracy-loss-diagnosis-with-neural-insights-5d73f4ca2601) (Aug 2023) -* Blog on Medium: [Faster Stable Diffusion Inference with Intel Extension for Transformers](https://medium.com/intel-analytics-software/faster-stable-diffusion-inference-with-intel-extension-for-transformers-on-intel-platforms-7e0f563186b0) (July 2023) * NeurIPS'2022: [Fast Distilbert on CPUs](https://arxiv.org/abs/2211.07715) (Oct 2022) * NeurIPS'2022: [QuaLA-MiniLM: a Quantized Length Adaptive MiniLM](https://arxiv.org/abs/2210.17114) (Oct 2022) @@ -155,6 +154,7 @@ q_model = fit( * [Legal Information](./docs/source/legal_information.md) * [Security Policy](SECURITY.md) -## Research Collaborations - -Welcome to raise any interesting research ideas on model compression techniques and feel free to reach us ([inc.maintainers@intel.com](mailto:inc.maintainers@intel.com)). Look forward to our collaborations on Intel Neural Compressor! +## Communication +- [GitHub Issues](https://github.com/intel/neural-compressor/issues): mainly for bugs report, new feature request, question asking, etc. +- [Email](mailto:inc.maintainers@intel.com): welcome to raise any interesting research ideas on model compression techniques by email for collaborations. +- [WeChat group](/docs/source/imgs/wechat_group.jpg): scan the QA code to join the technical discussion. diff --git a/docs/source/imgs/wechat_group.jpg b/docs/source/imgs/wechat_group.jpg new file mode 100644 index 0000000000000000000000000000000000000000..ba94e508cc3649e3f24ae28b9f60f4d389e742c1 GIT binary patch literal 29155 zcmd6Q30xCb_x2zn;ucYGMN?N4s<;%f|YtF#`hwYytd>sdBMNm|=?+=s)+cs_6 zw(roXWBU%K9on|-*rlUsXESqi^LDJR-MW}{>ttqbhQ7qW5I)n=$i&FV#H@YW_GWtj zqq>2awl;Wb@YK+tFV@1;z|hn{brWM_n1K}D17bU3Ees3| zTeLJpqlTmX;eD*7sZpms!zZ-rJY%78-?e5Ve*Nua>wXh2CY#TEBMDQxzFPMG}~j*;w4L$Enl&6{f3Rd`1o$x zynV;cT>-oI>^*ciDERk~KaLzd6?Xc}+3<7cBQ9NziH(cDa`jqD>dmzDTet6IJbv;t z>sj{m7deGR#ji_B-@JWSR#{bDQ!A;H);FN*GQbSU%WAqZ#&wy(b+u^O(y*m5x-Nqj zKJaH~+R~`c@K&8B%rIWKwsYSRzqU4;_}j^g$tL|q&J>&fyzWt(F8#kL98ifajkvPr zUD&pNbY)E!rn#;htev3&+&n{5jDsm$PwlODm6R&3iVF9sutF2Icqg8+P=%S;%`cyi8IfITRn3?+T ze>DdBd3g~**bGNA`13n2ThD))FHTco;vIp~iK4<`Dr`qcd46>l8S6z-rUY+TI8>3p zT$28(zQC~Tnst>>$v&pS++0-HP?54om^Bp}Bn;-P_RbAeb`vQ({-(lObL9is;zNoA z<#=y>$!|0 zW-y+kPiP;-EjxpmaWe$*cIEd~SUdRY*5AXmSSvfTq*M5LDr~s-1w|;VnPn>MY^RDm zN#H84isKHmR9L_pJgc(`yDL;-SI!yM&fF`HWW30D`oBIVvUcXyHiJ%WZG#57?G1h( zMt>z=`5dNz@2Z`L)(z)QuuA6dRYnZb`KWNCe@)3e40ork z*{)-i%LR&b6(&g$S-r(h?Ik|aO@-ZG1*f)D=07wpnP;wW<5mgB8<{tpS79xOtFWUl zRM;99g$S(@rTbf9*{Qwo_`c!XDN*od3`Qf&5J~$4%AaM*w&CRpe#CSGUNA16TTy=s zuj{G8a@#1}&>=&8-5)XE=SRRDg~7wL)7O=CRkX-m``fb|(Kbd0 zr&f!~U=fyN!$Nd#I1hIW<452b%ki>6JS7em%^OLeBtU7nX+G~cCpWKDg>}gP1NIr* zsVrDicU@Ij!g5%aj5|ks^?tl84ljY*k{5$#bQP6dfT_6yU&)sENN{D#`lMyKD(rb~ z3Erh5q;{s({lIYA;Oe=KW(Kvq44(FqZ3X898I6vNqu;Zy5aSmI&LxrVz zg7L%U{dlWY*dH#rNlzq!msQvk-#9NBC&}lRTO4$wVWSnZ8=?gDEAcunegt?ZeVjpY ziMN#RqnN>wUsGYSy@Av{%6ETgTnqk%T9L!2C)NxN(F}e~y=S zCVYB1yXKYJrQgchFk?JBBhjBLYo}~rwd4Can@Q?WC{QD9-27hAW^HJs$ zH>xg^Z3Um9FLM1Cv0gJ#c8y_%ryDSq2h)KO%LSPg+9J|<6f7zLHB|HKj(jl1LcyKf zNElwUO_9+iTqDh3Wld#{5Ue%sZZ3}mTS}}1ipd2#nB~H_out)L+c0Csh^*e%x>85P z3*G>mfdm*jjkH1mux_D1>cx^y#EW9VqH&a_La=C-<@qaQPROENpEF7!$RC@qNB|Zs zol`r9Q}q-KS`L`$Q($_NEC1PVWbLce-bvoi+q+y900q^R#H*FLy=Ivub@+ynOU6W#lrYhd_}F`z4s^O6g+{Q@Tz9(6>ct9KY0Ger?3exo`PPXJ81<_zovZRI1tz;qXYW*DFZt--y600a#(Y&gd-;9zU$y93S( zfJay|AI60Z0szF=zC7)@M=RxTZ#8&X19{{l*zo8P_Aop$Zy;cyPry3n)hOK1ZUEqf zXBw)9K&^R3j&ZX^?%Dvm zMY9$ZX_APW085K8T{;&P44xK=Wn3F%$asyuiD5aziBm*oYlzsT?G1+>t)= zv?O9j3Y331s&{1lO#QbB<2Hi3L~Du?)d2e3%AtD!PD$JR+Mn>k#eg?ivavFb3Txq) zKeAkfeekLh?en>-y^T3^s7&=KL@+>@jL&>+1`SKPX^)b$MxPJT7*4Hm09dpALvhJe z9s3>TEmeb-W2@pJmS_jy90d~qC(VWGP<(Q;XZ`<^I9@Yrz}i zvT%y$;#mVDLPFF&91TPT+KGgyKp{{tYDGEII--!9uwG#I!Iaa&SBPHWTvRv*SO0&r zYZ+%Dpa+mPJtrwU3Z$n*g)DHdo&@pZlf!?6h@UR^Ma$|Qe0=_0yC)%)EAR; zO%xRJMwEcm-<8Dr5XgiAI&2))2M+L2#IOQ2!+H&T6yaDXaR^ui=1YOpI0e0bD#;Dt z=W1{1&j3FrI?4O+#DCx^KLAkL=O6GPe;EL!VIlmAm@SLE{lo^f^9Vfsw@4iUej4B= zeQyvTmaN_?Y$pViYctE)kO*SfM-YWf;L+CX4;$( zq+C@v8AUJ0Q&y2w1p@W}_U_FgkbMs4-Hs+;AMw_(p34ODG-R&>?9<=p6lgFA-~m`m zRXZyh{^E!);UA3^#}on;_N#;U5ee{Cqw0BS2_U2>@otxBAC&qSvM8G5}v*3m?-9b_$x$?-sXRjvkA*Y9+a+>BR7y&YdlcV}5 zG7)0JPf4xPT4IornvZ6vLY0g4%gNeSp!|u;@HDWhy7$ zPChC?J}_0;QSIwlYF{5l@%30kG(o+oDY(W*9RSD?C^(?Q5G~GxP(UxzZ;B4|;|_5U zK@)>?8`wHkHP`M z7%;vTLL27fnHe;kr?Q@i5Q0BrM3SkkkA8tFIF0EDms3_J^3~+(1Yhllr?4UT%_cHL z6)}o1DKgz_^hj^$m<%m^X}fXtsdM*!w;Mx*^l8-lR^up!j;zp~badNvdx(6IvA z9nGh+G!vjc|3Au6XtDJO=27@oH`+6I@E3?hWE??(g#vi^j8|b7<1>^)@@=%;-XM%Y&Cs7lLQVq%5=jrU zMmSa)jY^<V%0j?kECZC?L6`&fZPj5g?^T?kKRKi90F=phU!!YXcRtGUc%xSsW3mW{B!1)+uJO zrL#G8hRO#hQFlXNcTa}Pdin{A@~RlA0xhbf(b%(4{tre%LP=n;)KJod#B6wjw+2dj zOBsUzW&{!m6h98Dk&&7q;&qY`Kwy~~vyIJ!F)0EH5BE^vR22{#RGvkb{EIMh90==^V%5!yc zYj4>>91P)dyudEFM6q3w_{v*cULQ!=2|D7Ir1Q}#H^`W2z@rwmM-T#|Tc2EBikyk4;d#FT_MzIb*Gu8puOJJ<-Im6VKFo3R(sl z5(ngW(q4tVOnXiorRMf&q%6Yf%^-L%VT1}>C?GX7;LU-K83wjJgy2009S-mw>P!$Y zeOf#&;&!R~6k%~Hwo$m9``Ev#Xz2;o4MDD_r0Zb`=bG3tQM7+f`V;9fp=rOsDWkmF zj`zBOnVfkwH}lDUkj9MjmTZQ}oG8sL8Wp%G+E;~fd4?13pvw9p4*vbu=eQ1+3Q7)`$?kn$19y~Ap z(nMIEy?Y+^ALSrWY!8Sit@SkLv1TMd-iaVuyU>oE;x1GbR0jN@Qb{rircN$ z_DE4+PVT&GPeWzK(_$}OoR}fA5)!ScPg!zfIU#rNoRAB}{6YLX4 zgr{5$DmnwW*6w_L`Fw{v=kF&^of+>I!w*?rRUuvHohoWzUkCTakHV!#IfZvb9aPv` zekD)Q02$08&!;&33&WH>F5hD3J;udg*`(K=C|vI)*j&$4_63HtB0*`A#m~sUfCu19 zOp5|PusV765CKi`!Vg3)E%*3|`aQ^vU;1tR{XZMp)KwsU{atBs4V)&j1edPNw7`9) z09IQDFT4dD@K@!nx(xz(D_l7=u-0Y7ha|BI+r1s|_Lh%u12+Z8$5<9-E6mttDl9+J zTiG`hxEyDxPi~R%-oVG!JmtVdC;DJHj7uTM&ME2x?D{L2sNncK@*7mxq&i?RDzbzs ztTWG#2)|tH*x|q|^S;JYXZaS(uDK`;fs60zB)9cG$hmq3FBqx9DmUN}n?jb<1<1d% z2$kjg$UA#W(iPjvIi6meRBN04L$BLzIiZOBP}Ot5)q^gaJ9&vd=ZkO8T^QiX>w5Xbh(}~WzpR}rM8CN zV^&qqS494FHtOuPgL8TouPzMeo8;4T>&=qt*EiF%U)1pT%f1$lPF`k~7~R3|S>CEF zSy+`6=$p0K1pBw&(h6EV@5-eE+m1o~wq&g~eCQyTksrX6!F(Hp|nK4vBGj z5^MIw23xdB3T!1<&!CG1ebO&-`=Qky}UR--+R`zZGoR?nruvm-j78;}xv0 z@+Oz<+t@R*&cU@l|*XMWJTHhB=rH+op zFP}Wmyioph^;sv2!;76P4w+`(oxF-a{>lBQ8{J1`V_P7pRR~ATPhXeS_}vlp-@hI} zp5|Ohhl%r^zj-Vg&u8V`-y}{=?b`Wn|LQ5>DHZ+yn*C=`_x{JH2s5|6WS`-0m794> zW0JrWU+@hV@XMz?>tD-MvIDheU#bGf&1>lM!L;`jZ}VR z9B<_O{qpw*v#LVzO})pJvRyqKdhN8iKH*I6k9Ll>PiI`IHM-IF;`LOI-o26nU9w~D zo|=b}2UgQj5`E zJFjnXci*&hsfDY3&tjWSzNzyoZMMt3+D8>7FB%zxFLPXe+Woebo2%axab3Er-}gzC zf!kuehDTM5Kh!fY)=K`@^~k~BWSaQe&5>@2lm@2vwtf1SoAbi2mxNuN-Frp11(*?k z>9fvdcP1SU{e9G>y80T?k%mLNY~BxbA5*{S*hj^KX-7AAk1V=BsZfv{*xIG=c6q^5 z+dZW+$J85_5(;;>y?wqma5vBVIxAwiXOYXrsOXb@jdtgLH|gY8BQ6?l-~RY~j&y!; zkuX_c@rGCt{B#E^rM5l=d7c3X+xI2aXGTu>#0+x+s91YvERzm zHYtmF-oH=g%?!&FStZuJU>hGzvFSBCH_vd#)mMC7i&AK{sZE^$mt@n-m`u+~4@TwPKpXS@mN_||bU_G-R7dUi; zP>Q?ZV{i-3>lj~|GQB+@)2U#bMByS6u=6FEiVe2)7a6?pEtgj=%~CYcqdGtyOoA15 zJBqcz#WB<%5MYz4@}}@q6HlVJ5(VD01a>nOHZ;QZUy(Y1O`v`$gMvbg0bKOwkN7gR z1u-9>Owd5JFXw~59BH}&Tq=I-at4TpCfI-_Ym63KnSIYGtprx)Vs?W?u*7&fzq~@R zQHAx8zLBK(i@F8jPlvJ;9j3~n+jH-C4RbI&z#YZ)uIQKU!}f?j{oQLc)uNCfbuiAP}%6co`$jQsRB5Y3d7c{)JuJfC?=FAwUE$nTNPM?`IxO~GF&c1kKZ(Q-n#sn_(?URqy1bi1T9)N z@Xt*{mPZMOKk;Kp`xGQDyU~MxwP?kpSNWGeur@51ZrBmS_8+(ITx6Nz+}TN*Rd}pL zO3J%$#@=*YcmA9G9o7p%?YTC^bHW1)1Sv1P0*_eDbBT3Nsx}_o>Y8GbWRLX7>Gz|) zbGm z$#lOAyVVUbieE-}DOu9>-%0lsf4|XnTxNjZ*i}Q*u8r!I$_h47#_moUC7mTc!YzDu zYLRizq?|RZ1=Bnu#jnm3vAgp8Jm&GHb$QkI%H!hp;i+{=(|rxI&2LKaqE?Zo{4AgP z^?23p%8`n0zxh3T(B}2FDLps!vsgdxX_&|5m4E$trnlqidiOtF*FB1SkGnoNKkY#O zW8DU&&#m}r;K$HQWzV+xIg2ZH<|iNTUAb(h*U)e7dQDv6_2$wQ|FrI{*0tZXtN8u= z@Zzy;qu3IE@ppF5s$RUDa&GSJi_^}`jed2-KKsz7;MyaT)}_Y0ejRlB@am6uVq(ku z;=CNKDp$4{l0M}0U(t`fr|mp&>p0lg@Eb{dmefg^4CM|DnD7B-p^uc~Js2%I)WL{ih-T+`YoUNbOJhPL&w%ZFXsnN7h0gmsQFuZya?P`@bo;A= z(W|SIbcb0ljNufHA}SN%ezwkrDuy8vly`qGE6PQQ z667E?@#%Y=H6u`1!8oicS}(F-D=R+-d1%J;{2TRv%s77#J)H;p;WOH*VW=EUD>#Am zq+@C@C`D){^I9gLi^&K%Bz4^fZ`7-h2;(cWwg53xbNC(acbX2)i6igYFY~}*rCIdL zI^b-8V1wGk8c}k7Rjmq}Cko?2F<#uHa_Mb=_uCP?PaG*>DI$QmyWX?N+0S>8($xDy zp7W}tOvOn_=E&jRc?lVT>@D4r-#xn?FS+L>>K>v@o{qm=$9*N9n&ib6UoO0oVt3a| z@!D4qI`y*of(c)3|IdWiJ)%lylyYnAtT^o?%fe^#AMrYFkbNIjRB3U*Y^Ay1?HLD+ zCmgbhPON;AuC#ny`XshJfA-g<-41pDT^I!e$hrMt<0de3&-5Pj*r;}iM3 zqWdFffHllow7#hIsI(=P&*M679#j+US~Pf{(#$R;bF2e5)S*1_mycs^C0T{s+=mB@ zYRH61DKz}!ZoeN)6D%F&lhUH9Y_x#S} zoH5a^sbI^)cgUT^IW{c$)H&swB~CVWAw9|$$~iZ`UC-mp{ORRJ58po??JqPp8g%Y# zUQuZ5vz)qd*EWcKJzP&(9oql;^^&_r*Em0zIk(qY^`lMd|F;(z=x677yh^JU4oEIP{_A`V-U7xnZqLYJ{fM)-`hj zbM4LiV;!7)&K$aw5VtSmR%+?fcZwC)TuWzp7KDUM1A~e5TZ9U0ZdDHCVwhzFPL!mxllpztLw*e2zHfzZA5`BVTs8SDPV15!2+n2lw^$* zu(p~KsR*2!>(Np*lA>c;x{SyzBL72=us$_Oz)jON8lbhzNQ44%AThLTwOaoCmPpD{ zYIm9AK<*g0Ww?4Os{$+!k;?`GCLzU}HRMIv8{BXG<6RE$dcQx6?6;)+{{K z*QMWAH4W}E_C8*R4R>!3Eq5@x)hjDuPu~@`U6ybywoS0FoyD{b!{V)-J>LHm zIW)1$E{?~|QMth%kNRZ$_(+e1@8|bnnJ@pgLvWYCWoJ{*q{W{3)ojX|*9$HWdtVbh zX`>?=C3Fg(2Mz~5}`zuWeyF3$oXp9%HtI^>Po z>H+!^Mmmos!jHK&KUuY`bs)<70z_*}#Wzp!!y#)58Ppx~t08wg6H1#p+1UzA>XFQ~sY0$@okJ89Gxf-m??wRVse z&Z8F20D7aeB>-j@k)lYAdr}8jimXZkjo^F5cHGN(tmFdE97@7dzD+XE+7N$m;UYzt zB>nd;x2s2%IBo4#p5Fc^#TADx3Ze7G%5=f1%yEK-p{IU6khkCRkG!iXe=YlE+L%}4 ztSfIxxQf%ZnJ@67XQ4%k{XDF;qdc8#eMdj5e|Lt8Gg0(nA?~G_4?e8BH|^U)EBlF! z3sY*vv-U}U8mtU%sH*JR-fK-oh0~kB9c#NjJ|%a##~b-_M(LKlif;iy8H)3jJEFi< zV_oHdscFv#tFW8jrQn%+{NHl2ES%(W-k$n>@Im{`wY>021+3Iv?+T*2!bi>UsXlT@ z;-=yaonm-$0~c|avhTpoO1~c}CMeGpao!ZcSvj9ND;wim*tr-j7fG?O20(SDox%S* z45&K+^`&ds)s=k9Qkx?m2UU+t;~&RUIdVG{cG4Q+;Tw=KX~QWLC}H#ugHM4YZG-P* zSDFqJR2T6!vud4yvE+0u;xAbt@$rbQO+0WQCO#x&#Mi@eb5HvG_Pk(_VIu#ue6;x9 zaU&^y@@+h~?Z$ye7Fl%qarca$PF3CL?KCZ@qMdwVQzDE&rs7v_P?sj)W+JuQlCDh#AM7%HMzdz7YF8&I*jSbI5)q^Vfj0V-{q2~y~4 z9!4q|I-}^7P zGWC!oWBeeyovZCE=H3mA;;+V&fA(`M@($o2zU+d0O62m=Y8eky%}Iyy@4b)=Nr*H3 z(|gsjvo29!PH5UvFhhe+LM7HXFfqBxVnGfkhb?VivI$=R9XEVJFWP#Cg-n0mf+y`4zx%p#Wlv?7Hd% zpxB=&slpVN;O3?kEBQ-QSW;wt0{E4}%C1VtIhx@GWp9wS>QO#*WzmJN-_#2Cyt1k6 z73NX)$~xQjho#+?dv6?kcH+1;mpgif^shtGIDNt~<@p;o(uSDp)i!4|+1#eWAnkBXTPvhhuR|XrQ7a66hoF|P zGtm&I4T`fH<`5Fus*omhB|26y6ZlOH0?4F4f$V8C zYjZntshbbWiF!XPY)2DBqu^0b z8B#^qtio={1sf7QR@8Hsi#j?qyP^aQ66v(*DHG~5p^l_BeCiWgdZuold3m2{MU$^x z<#EE0HaN|x!Lj^9=AZ0PLZ`%ZmMt+M6=*5Z9j$j}DP~kiXXV#fC?6>wLUiM8eOG2I z+j2d>Ft?goh0NBe@MJ^-fX%mZzO+>YkuIUxK;1>HNEt;ptYncb&5%80lu7=ZEfolB zvsd5!4i(zj(8>-pU6gJE(5cM8k!UYQ_D&<~ya`$F!lu?~&|OT^In52?UQOj>464Pa zlN~Ws=0nH(vHy*dF0C{iDrIW6l6WgodZw$$pXlnSw~&ynO$tLF10uEAOrGmMtQ%H{ z`=gnWjB8~7B)Y1O8e0&x>dmaig?_OpP9Y?-UszbC-N6Y=9wPunT4sX826jwGVlo4Y zm{G2rXj@7&?`mvGu|v_=lEO}1k}L+@r5QcAL_$aUE7V!heplaMG?*k*>3au*njcN0 zPOt$4PH(p3VWv%Z`ln8H?q_6pG^q~IkU@VKB2^CoQ6XX+8XLj_w@Ls&q9 zR3LH=r88)jzJ|8LKi7aVNLRDi$WJ$L)gWLUssXabhpa>cXaFq)pl%91(nnaU1=)z= zHofo^QO?v@R7DcujqS@CUnABxy$DuUXH#%Y!J9$lpm}V=*cqT#P)pZQFEG^G^;12; zbTqPMB(Tt02j;{Kk;7!u5=KED zC=Y2-!d0N|r9`_4=w8~VA8Lj|z+}|-jHv5?zGqE^*Y$H~s632$^^J3_S09MK?-t9Z zBb!I#*&>mC^T{VpKuyLHH&Vl5Qr(KW8HeUwFyVxw7|(VmyJDc8F=UfhwLGe`S|0VP zsVheohe|dv;4U(8VDy?IO`_ERDc1Q1t;S(%(98iK3W!4JH~teTaIMA`ooh^~1g7Z#vg z$Wa&OITxu5^MDIu!~_WQ8|nEi3Rh+&jc!SszQ_?3(Le;J6QVOYG$hGvqV=>*I*1^N z#=Ff-D$Qc-jOBogm}dlN%tJO7Ks*?a#`b%J@gUmoseAe~S&!Ss1?IW}mq!dIsH60+ zK?tKKC7dWFrtU@yDY3?Gv?wK}=|=0;>|%COGe8)OfAGTs%o#DG?L>EdS=mojhaqr5 zE7XD+qdk8J%Z;9Zdds|jpD*JY1Z7+12*=V93Hqf5aMQ#)G=r=&Y$CWz2-KQGJN&ao z)mmgEDbWC}1=ChN(%t+^U3Ee*VN^Y+?)EfH^9{VRfulK4=gp_ewQdfnx@rs%2%ky7 z{iYz{Q@zb&wm{aF2q2o}Tbuk82Y4;U4;CQ8 zhUPvRYn)McU?ibIY8V>xl+CQddy5)YG~b&*{cCeX;{QU|Q3IYnFp2E9Mdjxhyrzyy zq7s+p*CC_UeMpT*kfLNWC$g;+3$lBjL|duG2Aq^;PGrkjN)(97J!xhHdLuzJpjT8u z>rGCeOp>>wl)96QqXwu#s+?X1N+5QEgrpMkn*WDZB19&HY)MC2jpk$fhG;D-P5(^^ zIyQ1A8n7t=6F0MuV&nHydSW3U1#$6zQ(8shlg1w!0hD9*w?Baa7zgPSsDKdA9>nfo zGl<_7YQ9d7K`X&eQ6Mx(VQFcB4V?AM-v2_{3J8}kZy*bD6x|+>-m&{LxsDs8)Dc2$ zN(`p%07(zPY2xU>zfV6j{=^R41It#J@K_*&4djIkjxEtUaXo2Y|4u$V6rJMK>?$lyp3Hc zn!&sGj@B?4{=S3Do&}+9RfAA4nb(v}1&a z*Trd)ydJ^rkqzRg-VkLOF(L6Y3xxlsp|BuiI3a$eX%I9LU!85E{vn$Q{!aay49E_m e|7S4L8jOb64K;zQzHlN!jG;{O2)&JUXa literal 0 HcmV?d00001 diff --git a/docs/source/installation_guide.md b/docs/source/installation_guide.md index 877b00388a8..22aa5afee00 100644 --- a/docs/source/installation_guide.md +++ b/docs/source/installation_guide.md @@ -145,21 +145,22 @@ The following prerequisites and requirements must be satisfied for a successful Version - 2.12.0
- 2.11.0
- 2.10.1
- 2.12.0
- 2.11.0
- 2.10.0
- 1.2.0
+ 2.13.0
+ 2.12.1
+ 2.11.1
+ 2.13.0
+ 2.12.0
+ 2.11.0
+ v2.13.0.0
+ 1.2.0
1.1.0 - 2.0.1+cpu
- 1.13.1+cpu
- 1.12.1+cpu
+ 2.0.1+cpu
+ 1.13.1+cpu
+ 1.12.1+cpu
2.0.1+cpu
1.13.1+cpu
1.12.1+cpu
- 1.15.0
+ 1.15.1
1.14.1
1.13.1
1.9.1
diff --git a/docs/source/publication_list.md b/docs/source/publication_list.md index 3ceaf62f1d6..574eb36dc4e 100644 --- a/docs/source/publication_list.md +++ b/docs/source/publication_list.md @@ -1,6 +1,7 @@ -Full Publications/Events (74) +Full Publications/Events (75) ========== -## 2023 (20) +## 2023 (21) +* arXiv: [Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs](https://arxiv.org/abs/2309.05516) (Sep 2023) * Blog on Medium: [Quantization Accuracy Loss Diagnosis with Neural Insights](https://medium.com/@NeuralCompressor/quantization-accuracy-loss-diagnosis-with-neural-insights-5d73f4ca2601) (Aug 2023) * Blog on Medium: [Faster Stable Diffusion Inference with Intel Extension for Transformers](https://medium.com/intel-analytics-software/faster-stable-diffusion-inference-with-intel-extension-for-transformers-on-intel-platforms-7e0f563186b0) (July 2023) * Post on Social Media: [ONNXCommunityMeetup2023: INT8 Quantization for Large Language Models with Intel Neural Compressor](https://www.youtube.com/watch?v=luYBWA1Q5pQ) (July 2023) diff --git a/docs/source/quantization.md b/docs/source/quantization.md index ef7ccfe8662..9e6a3796587 100644 --- a/docs/source/quantization.md +++ b/docs/source/quantization.md @@ -469,7 +469,7 @@ Intel(R) Neural Compressor support multi-framework: PyTorch, Tensorflow, ONNX Ru cpu - ONNX Runtime + ONNX Runtime CPUExecutionProvider MLAS "default" @@ -493,6 +493,12 @@ Intel(R) Neural Compressor support multi-framework: PyTorch, Tensorflow, ONNX Ru "onnxrt_dnnl_ep" cpu + + DmlExecutionProvider* + OneDNN + "onnxrt_dml_ep" + NA + Tensorflow Tensorflow @@ -518,6 +524,7 @@ Intel(R) Neural Compressor support multi-framework: PyTorch, Tensorflow, ONNX Ru

+> Note: DmlExecutionProvider support works as experimental, please expect exceptions. Examples of configure: ```python diff --git a/third-party-programs.txt b/third-party-programs.txt index f4c07f8f23d..6c2f7219ca2 100644 --- a/third-party-programs.txt +++ b/third-party-programs.txt @@ -402,6 +402,9 @@ terms are listed below. socket.io Copyright (c) 2014-2018 Automattic + sass + Copyright (c) 2016, Google Inc. + The MIT License (MIT) @@ -1840,13 +1843,16 @@ Code generated by the Protocol Buffer compiler is owned by the owner of the input file used when generating it. This code is not standalone and requires a support library to be linked with it. This support library is itself covered by the above license. + ------------------------------------------------------------- -7. Hardware-Aware Transformer software +8. Hardware-Aware Transformer software + Copyright (c) 2020, Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, + Ligeng Zhu, Chuang Gan and Song Han + All rights reserved. ------------- LICENSE For Hardware-Aware Transformer software --------------- -Copyright (c) 2020, Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, -Ligeng Zhu, Chuang Gan and Song Han -All rights reserved. + css-select + Copyright (c) Felix Böhm + All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: @@ -1893,7 +1899,6 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. - ------------------------------------------------------------------ The following third party programs have their own third party program files. These additional