Code

add rwkv support#1198

Code

Merged

Quentin-Anthony merged 26 commits intomainEleutherAI/gpt-neox:mainfrom rwkvEleutherAI/gpt-neox:rwkvCopy head branch name to clipboardMay 6, 2024

+882-18

Commits

Commits on Mar 31, 2024

• add rwkv support

  jahatef

  committed
  2741e89

• Update init_functions.py

  jahatef

  authored
  282f800

• rwkv model files

  jahatef

  committed
  09ba65e

• Merge branch 'rwkv' of github.com:EleutherAI/gpt-neox into rwkv

  jahatef

  committed
  8892432

• configs

  jahatef

  committed
  04b8fdb

Commits on Apr 10, 2024

• kernels

  jahatef

  committed
  6e79cc2

Commits on Apr 16, 2024

• Cleanup

  jahatef

  committed
  cb49ff6

• Update 760M.yml

  jahatef

  authored
  96ea6f5

Commits on Apr 17, 2024

• remove preffn and mishglu

  jahatef

  committed
  54f2775

• merge

  jahatef

  committed
  3606bdd

Commits on Apr 19, 2024

• Merge branch 'main' into rwkv

  Quentin-Anthony

  authored
  8d60cef

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  276ffa9

• Add RWKV parallelism assertions

  Quentin-Anthony

  committed
  e20138c

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  428aad5

• pre-commit and config cleanup

  Quentin-Anthony

  committed
  1b0bbab

• Merge branch 'rwkv' of https://github.com/EleutherAI/gpt-neox into rwkv

  Quentin-Anthony

  committed
  7550d64

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  c0af563

Commits on May 3, 2024

• rwkv logging

  jahatef

  committed
  1eb5f51

• Merge branch 'rwkv' of https://github.com/EleutherAI/gpt-neox into rwkv

  jahatef

  committed
  330a802

Commits on May 4, 2024

• Merge branch 'main' into rwkv

  Quentin-Anthony

  authored
  1103663

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  a599ac7

• Add rwkv version dirname, make hdim 3.5x

  Quentin-Anthony

  committed
  682f7e5

• pre-commit

  Quentin-Anthony

  committed
  921c41a

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  8f60a43

Commits on May 5, 2024

• fix bug and set batch size to 32

  jahatef

  committed
  6fb840e

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  dd0138e