Code

Implement DoRA#1474

Code

Merged

BenjaminBossan merged 21 commits intohuggingface:mainhuggingface/peft:mainfrom BenjaminBossan:feat-doraBenjaminBossan/peft:feat-doraCopy head branch name to clipboardFeb 27, 2024

+352-30

Commits

Commits on Feb 16, 2024

• [WIP] Try to implement DoRA

  Show description for 6242f4a
  BenjaminBossan

  committed
  6242f4a

  Copy full SHA for 6242f4a

Commits on Feb 19, 2024

• Merge branch 'main' into feat-dora

  BenjaminBossan

  committed
  32ffc50

  Copy full SHA for 32ffc50

• Better var name

  BenjaminBossan

  committed
  53a5e25

  Copy full SHA for 53a5e25

• Update all signatures of update_layer

  Show description for 18fb476
  BenjaminBossan

  committed
  18fb476

  Copy full SHA for 18fb476

Commits on Feb 20, 2024

• Fix to how weight_norm is calculated

  BenjaminBossan

  committed
  e4a677f

  Copy full SHA for e4a677f

Commits on Feb 21, 2024

• Fix implementation

  BenjaminBossan

  committed
  c42bb0a

  Copy full SHA for c42bb0a

• Fix a few failing tests

  BenjaminBossan

  committed
  b915ea2

  Copy full SHA for b915ea2

Commits on Feb 22, 2024

• Some adjustments to correctly consider scaling

  BenjaminBossan

  committed
  8bd22d7

  Copy full SHA for 8bd22d7

• Make style

  BenjaminBossan

  committed
  ecf7160

  Copy full SHA for ecf7160

• Small fix of calculation of weight norm

  BenjaminBossan

  committed
  07f3e43

  Copy full SHA for 07f3e43

• Make style

  BenjaminBossan

  committed
  5b22170

  Copy full SHA for 5b22170

Commits on Feb 23, 2024

• Add documentation for DoRA

  BenjaminBossan

  committed
  f15dbae

  Copy full SHA for f15dbae

• Improve DoRA init test

  BenjaminBossan

  committed
  1e6d1d7

  Copy full SHA for 1e6d1d7

• Address reviewer feedback

  BenjaminBossan

  committed
  69a81a6

  Copy full SHA for 69a81a6

• Fix for DeepSpeed Zero3

  Show description for 4a90843
  BenjaminBossan

  committed
  4a90843

  Copy full SHA for 4a90843

• Fix context call

  BenjaminBossan

  committed
  951ae67

  Copy full SHA for 951ae67

Commits on Feb 26, 2024

• Address reviewer feedback

  BenjaminBossan

  committed
  9c4edc1

  Copy full SHA for 9c4edc1

• More efficient forward pass

  BenjaminBossan

  committed
  4bf7346

  Copy full SHA for 4bf7346

• Make style

  BenjaminBossan

  committed
  ebdda07

  Copy full SHA for ebdda07

• Update comment

  BenjaminBossan

  committed
  6194726

  Copy full SHA for 6194726

Commits on Feb 27, 2024

• Make deepspeed check work with older transformers

  BenjaminBossan

  committed
  51d4919

  Copy full SHA for 51d4919