Code

[CORE] add marlin inference kernel#310

Code

Merged

Qubitium merged 8 commits intoModelCloud:mainModelCloud/GPTQModel:mainfrom ZX-ModelCloud:zx_marlin_inference_kernelZX-ModelCloud/GPTQModel:zx_marlin_inference_kernelCopy head branch name to clipboardJul 30, 2024

+3,156-34

Commits

Commits on Jul 30, 2024

• add marlin_inference kernel

  ZX-ModelCloud

  committed
  483b1f6

  Copy full SHA for 483b1f6

• add MarlinInferenceQuantLinear

  ZX-ModelCloud

  committed
  2e09bb7

  Copy full SHA for 2e09bb7

• fix bias

  ZX-ModelCloud

  committed
  8bd96e4

  Copy full SHA for 8bd96e4

• add more marlin inference test

  ZX-ModelCloud

  committed
  32ea5e9

  Copy full SHA for 32ea5e9

• When desc_act=True and group_size == -1, change desc_act to False.

  ZX-ModelCloud

  committed
  c638f72

  Copy full SHA for c638f72

• cleanup

  ZX-ModelCloud

  committed
  a9d27c8

  Copy full SHA for a9d27c8

• cleanup

  ZX-ModelCloud

  committed
  2bdbe4f

  Copy full SHA for 2bdbe4f

• add license info

  ZX-ModelCloud

  committed
  4a571cc

  Copy full SHA for 4a571cc