Code

Add GQA support for ROCm#21032

Code

Merged

cloudhan merged 18 commits intomainmicrosoft/onnxruntime:mainfrom guangyunhan/rocm-gqamicrosoft/onnxruntime:guangyunhan/rocm-gqaCopy head branch name to clipboardJul 3, 2024

+810-42

Commits

Commits on Jun 4, 2024

• feat: init rocm gqa

  cloudhan

  committed
  c877adc

  Copy full SHA for c877adc

Commits on Jun 6, 2024

• feat: extend strided copy to support runtime tok idx

  cloudhan

  committed
  f845099

  Copy full SHA for f845099

Commits on Jun 7, 2024

• more case

  cloudhan

  committed
  0ea3335

  Copy full SHA for 0ea3335

Commits on Jun 11, 2024

• feat: local

  cloudhan

  committed
  99b2feb

  Copy full SHA for 99b2feb

Commits on Jun 12, 2024

• feat: rotary

  cloudhan

  committed
  816249c

  Copy full SHA for 816249c

Commits on Jun 13, 2024

• feat: allow rotary to read and write in a strided way, so that we don't need to explicit unpack the packed qkv tensor

  cloudhan

  committed
  6024dc9

  Copy full SHA for 6024dc9

• fix: rotary for packed qkv

  cloudhan

  committed
  48092ee

  Copy full SHA for 48092ee

• remove debug print

  cloudhan

  committed
  de2f30a

  Copy full SHA for de2f30a

Commits on Jun 21, 2024

• workaround: add flash_attn test to ci

  cloudhan

  committed
  6c4e612

  Copy full SHA for 6c4e612

• add gpu arch checking warning log

  cloudhan

  committed
  e9f6d13

  Copy full SHA for e9f6d13

• fix: build without ck tile

  cloudhan

  committed
  2b0c46e

  Copy full SHA for 2b0c46e

Commits on Jun 28, 2024

• test: update ci pytorch and triton version to fix tests which have failed with nan and inf from reference values

  cloudhan

  committed
  6091a69

  Copy full SHA for 6091a69

Commits on Jul 1, 2024

• format

  cloudhan

  committed
  8ca0634

  Copy full SHA for 8ca0634

• remove unused param is_input_bnsh_format from strided version LaunchRotaryEmbeddingKernel

  cloudhan

  committed
  e22dfb9

  Copy full SHA for e22dfb9

• make onnxruntime_USE_COMPOSABLE_KERNEL_CK_TILE depends on onnxruntime_USE_COMPOSABLE_KERNEL

  cloudhan

  committed
  789fee7

  Copy full SHA for 789fee7

• skip test_flash_attn_rocm on CUDA platform

  cloudhan

  committed
  b973217

  Copy full SHA for b973217

Commits on Jul 2, 2024

• fix lint and ci

  cloudhan

  committed
  c3c7089

  Copy full SHA for c3c7089

• fix typo

  cloudhan

  committed
  f4355d4

  Copy full SHA for f4355d4