Generate: Add new decoding strategy "DoLa" in `.generate()`#29619

Merged

gante merged 33 commits intohuggingface:mainfrom voidism:main

Jul 9, 2024

+530-5

Commits on Jul 9, 2024

add support for DoLa decoding

voidism
authored and
gante
committed
add docs; remove deprecated function

voidism
authored and
gante
committed
add test code for DoLa decoding

voidism
authored and
gante
committed
update docs and paper link

voidism
authored and
gante
committed
solved the issues that made tests failed on CircleCI

voidism
authored and
gante
committed
ruff reformatted

voidism
authored and
gante
committed
update DoLa decoding; test cases for llama/mistral/mixtral/gemma; docs

voidism
authored and
gante
committed
fix formatting; fix failed test cases

voidism
authored and
gante
committed
fix other test cases

voidism
authored and
gante
committed
ruff reformatted; mamba cache issue unsolved

voidism
authored and
gante
committed
remove keyword argument 'model_inputs' to match upstream changes

voidism
authored and
gante
committed
improve documentation

voidism
authored and
gante
committed
fixed suggestions from @gante

voidism
authored and
gante
committed
ruff reformated

voidism
authored and
gante
committed
moved config warning of dola generation from utils.py to configuration_utils.py

voidism
authored and
gante
committed
fixed suggestions from @amyeroberts

voidism
authored and
gante
committed
fixed format issue; removed print; added explanation

voidism
authored and
gante
committed
remove trailing whitespace

voidism
authored and
gante
committed
ruff reformat to pass test

voidism
authored and
gante
committed
fixed suggestions from @amyeroberts on Mar 28

voidism
authored and
gante
committed
fix failed CI tests

voidism
authored and
gante
committed
ruff reformatted; fixed missing argument generation_config

voidism
authored and
gante
committed
make dola_layers not optional

voidism
authored and
gante
committed
fix divergence w main
gante
committed
fix dola test on mamba
gante
committed
rwkv test (wont fix
gante
committed
slow tests running in fp16
gante
committed
make fixup
gante
committed
remove redundant fn
gante
committed
final rebase divergences
gante
committed
this one was missing
gante
committed
a few more nits
gante
committed
skip stateful models
gante
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate: Add new decoding strategy "DoLa" in `.generate()`#29619

Commits on Jul 9, 2024

add support for DoLa decoding

add docs; remove deprecated function

add test code for DoLa decoding

update docs and paper link

solved the issues that made tests failed on CircleCI

ruff reformatted

update DoLa decoding; test cases for llama/mistral/mixtral/gemma; docs

fix formatting; fix failed test cases

fix other test cases

ruff reformatted; mamba cache issue unsolved

remove keyword argument 'model_inputs' to match upstream changes

improve documentation

fixed suggestions from @gante

ruff reformated

moved config warning of dola generation from `utils.py` to `configuration_utils.py`

fixed suggestions from @amyeroberts

fixed format issue; removed print; added explanation

remove trailing whitespace

ruff reformat to pass test

fixed suggestions from @amyeroberts on Mar 28

fix failed CI tests

ruff reformatted; fixed missing argument generation_config

make `dola_layers` not optional

fix divergence w main

fix dola test on mamba

rwkv test (wont fix

slow tests running in fp16

make fixup

remove redundant fn

final rebase divergences

this one was missing

a few more nits

skip stateful models