Cherry-pick LLaMA GQA mask to rel-1.16.2 (round 4)#18350

Merged

tianleiwu merged 2 commits intorel-1.16.2from tlwu/rel-1.16.2-llama-cherry-pick-round4

Nov 8, 2023

+1,306-463

Commits on Nov 8, 2023

GQA Flash Attention with Attention Mask (#18283 )

authored and
tianleiwu
committed
Add LLaMA GQA ragged batching (#18337 )

kunal-vaishnavi
authored and
tianleiwu
committed