Code

fused layernorm#1105

Code

Merged

Quentin-Anthony merged 5 commits intoEleutherAI:mainEleutherAI/gpt-neox:mainfrom yang:fused-layernormyang/gpt-neox:fused-layernormCopy head branch name to clipboardJan 26, 2024

+285-1

Commits

Commits on Jan 13, 2024

• Add simple util for CUDA timings

  yang

  committed
  24199ec

  Copy full SHA for 24199ec

• Add fused layernorm kernel from Megatron

  Show description for 4065730
  yang

  committed
  4065730

  Copy full SHA for 4065730

Commits on Jan 26, 2024

• change default fused layernorm to false

  jahatef

  committed
  6ab6331

  Copy full SHA for 6ab6331

• Update test_setup.yml

  jahatef

  authored
  8b223f9

  Copy full SHA for 8b223f9

• Update test_train_base.yml

  jahatef

  authored
  8abfbbd

  Copy full SHA for 8abfbbd