Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add static cache #89

Merged
merged 69 commits into from
Aug 7, 2024
Merged

Add static cache #89

merged 69 commits into from
Aug 7, 2024

Commits on May 17, 2024

  1. add rope

    @sanchit-gandhi
    sanchit-gandhi committed May 17, 2024
    Configuration menu
    Copy the full SHA
    20dfbff View commit details
    Browse the repository at this point in the history

  2. don't include padding in rope

    @sanchit-gandhi
    sanchit-gandhi committed May 17, 2024
    Configuration menu
    Copy the full SHA
    fd16411 View commit details
    Browse the repository at this point in the history

  3. possibly use cross-attn for prompt

    @sanchit-gandhi
    sanchit-gandhi committed May 17, 2024
    Configuration menu
    Copy the full SHA
    1344de2 View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. fix rope

    @sanchit-gandhi
    sanchit-gandhi committed May 18, 2024
    Configuration menu
    Copy the full SHA
    e3fe843 View commit details
    Browse the repository at this point in the history

  2. fix cross-attn

    @sanchit-gandhi
    sanchit-gandhi committed May 18, 2024
    Configuration menu
    Copy the full SHA
    4f27ff7 View commit details
    Browse the repository at this point in the history

  3. fix self-attn

    @sanchit-gandhi
    sanchit-gandhi committed May 18, 2024
    Configuration menu
    Copy the full SHA
    c8887f2 View commit details
    Browse the repository at this point in the history

Commits on May 20, 2024

  1. fix dummy model

    @sanchit-gandhi
    sanchit-gandhi committed May 20, 2024
    Configuration menu
    Copy the full SHA
    ba0bfca View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. clean-up rope

    @sanchit-gandhi
    sanchit-gandhi committed May 21, 2024
    Configuration menu
    Copy the full SHA
    387de15 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. first gqa implementation

    @ylacombe
    ylacombe committed May 23, 2024
    Configuration menu
    Copy the full SHA
    e0500d0 View commit details
    Browse the repository at this point in the history

  2. fix wer eval

    @ylacombe
    ylacombe committed May 23, 2024
    Configuration menu
    Copy the full SHA
    3342e2e View commit details
    Browse the repository at this point in the history

  3. Merge branch 'huggingface:main' into gqa-experiments

    @ylacombe
    ylacombe authored May 23, 2024
    Configuration menu
    Copy the full SHA
    f094f7b View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. feat: add flash attention and spda

    @sang-nguyen-ts
    sang-nguyen-ts committed May 29, 2024
    Configuration menu
    Copy the full SHA
    b171d98 View commit details
    Browse the repository at this point in the history

  2. chore: add README for flash attention

    @sang-nguyen-ts
    sang-nguyen-ts committed May 29, 2024
    Configuration menu
    Copy the full SHA
    9e915aa View commit details
    Browse the repository at this point in the history

  3. chore: add benchmark script

    @sang-nguyen-ts
    sang-nguyen-ts committed May 29, 2024
    Configuration menu
    Copy the full SHA
    1eeca66 View commit details
    Browse the repository at this point in the history

  4. chore: add benchmark attention approach

    @sang-nguyen-ts
    sang-nguyen-ts committed May 29, 2024
    Configuration menu
    Copy the full SHA
    69521dd View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. multi node and fix wer and fix compile

    [email protected] authored and [email protected] committed May 30, 2024
    Configuration menu
    Copy the full SHA
    7539000 View commit details
    Browse the repository at this point in the history

  2. Update modeling_parler_tts.py

    @ylacombe
    ylacombe authored May 30, 2024
    Configuration menu
    Copy the full SHA
    76dbe87 View commit details
    Browse the repository at this point in the history

  3. Merge pull request huggingface#2 from sang-nguyen-ts/pef/flash-sdpa-a… 

    …ttention

Pef/flash sdpa attention
    @ylacombe
    ylacombe authored May 30, 2024
    Configuration menu
    Copy the full SHA
    9d67863 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'architecture-experiments' into cross-attn

    @ylacombe
    ylacombe authored May 30, 2024
    Configuration menu
    Copy the full SHA
    cd4fcc1 View commit details
    Browse the repository at this point in the history

  5. Merge pull request huggingface#1 from sanchit-gandhi/cross-attn 

    Cross attn
    @ylacombe
    ylacombe authored May 30, 2024
    Configuration menu
    Copy the full SHA
    934f08c View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. fix FA2, SDPA and add cross-attn MHA and attention type forcing

    @ylacombe
    ylacombe committed May 31, 2024
    Configuration menu
    Copy the full SHA
    fc79c06 View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. better cross_attention key values number of heads default + add train… 

    …ing arguments for attn implementation
    @ylacombe
    ylacombe committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    7808285 View commit details
    Browse the repository at this point in the history

  2. fix audio padding when torch compile or pad_to_max_length=True

    @ylacombe
    ylacombe committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    0ce0df2 View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. correct multi node

    [email protected] authored and [email protected] committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    8198fd9 View commit details
    Browse the repository at this point in the history

  2. make rope faster

    @ylacombe
    ylacombe committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    9b48d0a View commit details
    Browse the repository at this point in the history

  3. fix encoder sdpa

    @ylacombe
    ylacombe committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    54b56d9 View commit details
    Browse the repository at this point in the history

  4. fix training with cross attention + with FAZ

    @ylacombe
    ylacombe committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    1da55fc View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. use fp32 as default model dtype + fix generation when using FA2 with … 

    …autocast
    @ylacombe
    ylacombe committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    8f6047a View commit details
    Browse the repository at this point in the history

  2. remove redundant passes in generate + clean and fix attentions

    @ylacombe
    ylacombe committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    d056ca5 View commit details
    Browse the repository at this point in the history

  3. fix edge case in WER evaluation when longform generation

    [email protected] authored and [email protected] committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    7dfbbca View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2024

  1. better multi-node mapping and saving / add eval dataloader num workers

    [email protected] authored and [email protected] committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    15edf7c View commit details
    Browse the repository at this point in the history

  2. remove old benchmarks

    [email protected] authored and [email protected] committed Jun 7, 2024
    Configuration menu
    Copy the full SHA
    ef40654 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. faster audio encoding + checkpointing + fix generation step

    [email protected] authored and [email protected] committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    954d8c5 View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. Merge branch 'dev' into architecture-experiments

    @ylacombe
    ylacombe authored Jul 8, 2024
    Configuration menu
    Copy the full SHA
    25490f0 View commit details
    Browse the repository at this point in the history

  2. Merge pull request huggingface#82 from ylacombe/architecture-experiments 

    Architecture experiments on Dev
    @ylacombe
    ylacombe authored Jul 8, 2024
    Configuration menu
    Copy the full SHA
    011855e View commit details
    Browse the repository at this point in the history

  3. unpin trfms

    @eustlb
    eustlb committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    f9c36ac View commit details
    Browse the repository at this point in the history

  4. remove CFG

    @eustlb
    eustlb committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    e52a8f0 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. imports and constants 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ccae5a9 View commit details
    Browse the repository at this point in the history

  2. attention modifications to handle static cach 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    a9f75d5 View commit details
    Browse the repository at this point in the history

  3. decoder layer modification to handle static cache 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    89e50d5 View commit details
    Browse the repository at this point in the history

  4. ParlerTTSPreTrainedModel modifs to handle static cache 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    fb750fe View commit details
    Browse the repository at this point in the history

  5. ParlerTTSDecoder modifs to handle static cache 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    41fa4fd View commit details
    Browse the repository at this point in the history

  6. ParlerTTSModel + ParlerTTSForCausalLM modfis 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    5a484f8 View commit details
    Browse the repository at this point in the history

  7. ParlerTTSForConditionalGeneration modifs 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    c5da07e View commit details
    Browse the repository at this point in the history

  8. decoder_attention_mask for static cache 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    8c780ef View commit details
    Browse the repository at this point in the history

  9. create inputs_embeds early to have a good cache initialization 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    afa18a3 View commit details
    Browse the repository at this point in the history

  10. _get_cache method 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    45d0fbb View commit details
    Browse the repository at this point in the history

  11. init the cache 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    054b751 View commit details
    Browse the repository at this point in the history

  12. ensure good device 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    11b693f View commit details
    Browse the repository at this point in the history

  13. pin tfrms version 

    Co-Authored-By:  sang-nguyen-ts <[email protected]>
    @eustlb @sang-nguyen-ts
    eustlb and sang-nguyen-ts committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    6af19d6 View commit details
    Browse the repository at this point in the history

  14. fix attention_mask FA2

    @eustlb
    eustlb committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    682ca70 View commit details
    Browse the repository at this point in the history

  15. remove unnecessary method

    @eustlb
    eustlb committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    024a354 View commit details
    Browse the repository at this point in the history

  16. Update parler_tts/modeling_parler_tts.py 

    Co-authored-by: Sanchit Gandhi <[email protected]>
    @eustlb @sanchit-gandhi
    eustlb and sanchit-gandhi authored Jul 24, 2024
    Configuration menu
    Copy the full SHA
    a097aa4 View commit details
    Browse the repository at this point in the history

  17. Update parler_tts/modeling_parler_tts.py 

    Co-authored-by: Sanchit Gandhi <[email protected]>
    @eustlb @sanchit-gandhi
    eustlb and sanchit-gandhi authored Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ecd06c1 View commit details
    Browse the repository at this point in the history

  18. remove unnecessary imports

    @eustlb
    eustlb committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ad25e2b View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. replace the hardcoded cache_position with a more elegant approach

    @eustlb
    eustlb committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f29392b View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2024

  1. make style

    @eustlb
    eustlb committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    d08e4eb View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into add-static-cache

    @eustlb
    eustlb committed Jul 31, 2024
    Configuration menu
    Copy the full SHA
    15d8e6a View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2024

  1. unpin transformers

    @eustlb
    eustlb committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    0e46d0b View commit details
    Browse the repository at this point in the history

  2. pin transformers

    @eustlb
    eustlb committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    43764cd View commit details
    Browse the repository at this point in the history

  3. pin torch

    @eustlb
    eustlb committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    b5e25f0 View commit details
    Browse the repository at this point in the history

  4. refactor + unpin torch

    @eustlb
    eustlb committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    441bafd View commit details
    Browse the repository at this point in the history

  5. Update parler_tts/modeling_parler_tts.py 

    Co-authored-by: Yoach Lacombe <[email protected]>
    @eustlb @ylacombe
    eustlb and ylacombe authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    45ee62e View commit details
    Browse the repository at this point in the history

  6. update training script to match 11b209e

    @eustlb
    eustlb committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    a294fb5 View commit details
    Browse the repository at this point in the history

  7. Update parler_tts/modeling_parler_tts.py 

    Co-authored-by: Yoach Lacombe <[email protected]>
    @eustlb @ylacombe
    eustlb and ylacombe authored Aug 1, 2024
    Configuration menu
    Copy the full SHA
    824b183 View commit details
    Browse the repository at this point in the history

  8. ensure compatibility with trfms 4.43.3, changes taken from #31980 on … 

    …trfms
    @eustlb
    eustlb committed Aug 1, 2024
    Configuration menu
    Copy the full SHA
    66fdbde View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2024

  1. fix input_ids_length

    @eustlb
    eustlb committed Aug 2, 2024
    Configuration menu
    Copy the full SHA
    dc21c87 View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2024

  1. warning full attention mask creation

    @eustlb
    eustlb committed Aug 5, 2024
    Configuration menu
    Copy the full SHA
    650f276 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. changes for training compatibility

    @eustlb
    eustlb committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    41edc2a View commit details
    Browse the repository at this point in the history