Cherry-pick LLaMA/SDXL to rel-1.16.2#18202

Merged

tianleiwu merged 40 commits intorel-1.16.2from tlwu/rel-1.16.2_sdxl_llama

Nov 1, 2023

+28,481-4,055

Commits on Oct 31, 2023

Add LLaMA scripts (#17020 )

kunal-vaishnavi
authored and
tianleiwu
committed
Add Whisper scripts (#17043 )

kunal-vaishnavi
authored and
tianleiwu
committed
Fix attention fusion for UNet onnx model export when using LoRA weights (#17249 )

PatriceVignola
authored and
tianleiwu
committed
Handle dtype attribute in float16 conversion script (#17321 )
tianleiwu
committed
fix assert error in attention fusion script (#17375 )
tianleiwu
committed
Fix weight tensors in transformers optimizer not saved to external data (#17427 )
tianleiwu
committed
fix embed layer norm fusion with embedding sum output (#17460 )
tianleiwu
committed
Attention fusion for stable diffusion clip model (#17445 )
tianleiwu
committed
Improve performance of prune_graph in onnx_model.py (#17502 )
tianleiwu
committed
Update optimize_pipeline for SDXL (#17536 )
tianleiwu
committed
Refactoring of attention cuda kernel: move prepare qkv and concat_past_to_present (#17559 )
tianleiwu
committed
Reduce GPU memory for Whisper models converted to ONNX (#17378 )

authored and
tianleiwu
committed
Provide kwargs to remove_shared_initializers (#17539 )

jambayk
authored and
tianleiwu
committed
Refactor Attention cuda kernel (#17578 )
tianleiwu
committed
Bugfix: Add initializer to model in AttentionMask directly (#17719 )

jambayk
authored and
tianleiwu
committed
[ROCm] Update whisper benchmark script (#17391 )

PeixuanZuo
authored and
tianleiwu
committed
StableDiffusion XL with TensorRT EP (#17748 )
tianleiwu
committed
Add CUDA EP in StableDiffusion demo (#17788 )
tianleiwu
committed
[BeamSearch]optimize key cache reordering (#17771 )

authored and
tianleiwu
committed
[CUDA] GroupQueryAttention operator using FlashAttention (#17674 )

committed
[CUDA/ROCm] Update BiasSplitGelu for SD XL Refiner model (#17849 )
tianleiwu
committed
[CUDA/ROCm] Remove limitation of BiasAdd (#17848 )
tianleiwu
committed
Fix GroupNorm fusion: skip if num of channels not supported (#17869 )
tianleiwu
committed
[CUDA] Fix SkipLayerNorm strict mode when skip has broadcast (#17896 )
tianleiwu
committed
Fix build break with cuda 12.2 (#17922 )

yufenglee
authored and
tianleiwu
committed
Provide kwargs to remove_shared_initializers (#17539 )

yufenglee
authored and
tianleiwu
committed
[CUDA] Fix SkipLayerNorm vectorized kernel out-of-bounds read (#17943 )
tianleiwu
committed
Fix MHA shape inference (#18009 )

PatriceVignola
authored and
tianleiwu
committed
[CUDA] StableDiffusion XL demo with CUDA EP (#17997 )
tianleiwu
committed
Fix Packed MultiHead Attention (#17996 )

aciddelgado
authored and
tianleiwu
committed
Support large model export using multi-gpu (#17990 )

authored and
tianleiwu
committed
LLaMA Model Optimization (#18021 )

kunal-vaishnavi
authored and
tianleiwu
committed
Add MatMul FP4 and NF4 Support (#18066 )

jambayk
authored and
tianleiwu
committed
Add updates to LLaMA scripts (#18076 )

kunal-vaishnavi
authored and
tianleiwu
committed
optimize SLN with large dimension (#18138 )

yufenglee
authored and
tianleiwu
committed
Add one more MHA mask pattern (#18164 )

PatriceVignola
authored and
tianleiwu
committed
[CUDA] Update GroupNorm and Add SkipGroupNorm (#18091 )
tianleiwu
committed
fix windows build
tianleiwu
committed

Commits on Nov 1, 2023

Reduce LLaMA memory usage (#18181 )

kunal-vaishnavi
authored and
tianleiwu
committed
Split KV on MHA and Attention ops (#18007 )

aciddelgado
authored and
tianleiwu
committed