Code

optimize gqa cpu#20598

Code

Merged

yufenglee merged 14 commits intomainmicrosoft/onnxruntime:mainfrom yufeng/gqa_cpu_optmicrosoft/onnxruntime:yufeng/gqa_cpu_optCopy head branch name to clipboardMay 8, 2024

+46-97

Commits

Commits on May 7, 2024

• only compute really length for score x value

  yufenglee

  committed
  52f6f0c

• remove unnessasary transpose in score x v

  yufenglee

  committed
  2166dfa

• use GemmEx for q x k

  yufenglee

  committed
  24d0c94

• compute softmax together with matmul

  yufenglee

  committed
  e677919

• remove mask

  yufenglee

  committed
  1754950

• only compute real length for q x k

  yufenglee

  committed
  b25c3ef

• clean up code

  yufenglee

  committed
  c577a70

• clean up softmax

  yufenglee

  committed
  38a9de4

• add support of local window

  yufenglee

  committed
  a52f048

• cleanup code

  yufenglee

  committed
  0c67354

• Merge branch 'main' into yufeng/gqa_cpu_opt

  yufenglee

  committed
  36df9b7

Commits on May 8, 2024

• fix merge error

  yufenglee

  committed
  3590796

• fix unit test

  yufenglee

  committed
  ef07314

• reduce the gqa_test in pipeline

  yufenglee

  committed
  1690826