Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

after finetuning model consistently produces the last token of prompt again. #241

Closed
arpitg1991 opened this issue May 28, 2023 · 1 comment
Closed

after finetuning model consistently produces the last token of prompt again. #241

arpitg1991 opened this issue May 28, 2023 · 1 comment

Comments

@arpitg1991
Copy link

I have a special token at the end of prompt. after finetuning the model is producing it as first token even though i give it the token in prompt
@arpitg1991
Copy link
Author

opened by mistake

bmosaicml pushed a commit that referenced this issue Jun 6, 2023
@vchiley
Fdiff updt (#241)
8b233a2 
* updt fdiff to only plot fdiff if it has seen metric before

* dk pr cmt
mvpatel2000 added a commit that referenced this issue Apr 9, 2024
@mvpatel2000 @actions-user
@abhi-mosaic @sashaDoubov @abhi-databricks @vchiley @cli99 @bigning @irenedea @ShashankMosaicML @j316chuck @dakinggg @josejg @angel-ruiz7 @dennyglee @jjanezhang
[Stage] Megablocks release (#241)
5bb3b7a 
* V1 of MegaBlocks
---------

Co-authored-by: GitHub Actions <[email protected]>
Co-authored-by: Abhinav Venigalla <[email protected]>
Co-authored-by: Abhi Venigalla <[email protected]>
Co-authored-by: Sasha Doubov <[email protected]>
Co-authored-by: Abhi Venigalla <[email protected]>
Co-authored-by: Vitaliy Chiley <[email protected]>
Co-authored-by: Cheng Li <[email protected]>
Co-authored-by: Ning Wang <[email protected]>
Co-authored-by: Irene Dea <[email protected]>
Co-authored-by: Shashank Rajput <[email protected]>
Co-authored-by: Charles Tang <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Shashank Rajput <[email protected]>
Co-authored-by: Chuck Tang <[email protected]>
Co-authored-by: Jose Javier <[email protected]>
Co-authored-by: Angel Ruiz <[email protected]>
Co-authored-by: Denny Lee <[email protected]>
Co-authored-by: Jane Zhang <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Chuck Tang <[email protected]>
Co-authored-by: Vitaliy Chiley <[email protected]>
mvpatel2000 added a commit that referenced this issue Apr 9, 2024
@mvpatel2000 @abhi-mosaic
@sashaDoubov @cli99 @bigning @irenedea @ShashankMosaicML @josejg @angel-ruiz7 @dennyglee @jjanezhang @dakinggg @j316chuck @vchiley
MegaBlocks release (#1102)
53160f4 
* [Stage] Megablocks release (#241)

* V1 of MegaBlocks
---------

* fix hf ckptr

* rename

* lint

* lint

---------

Co-authored-by: Abhinav Venigalla <[email protected]>
Co-authored-by: Sasha Doubov <[email protected]>
Co-authored-by: Cheng Li <[email protected]>
Co-authored-by: Ning Wang <[email protected]>
Co-authored-by: Irene Dea <[email protected]>
Co-authored-by: Shashank Rajput <[email protected]>
Co-authored-by: Chuck Tang <[email protected]>
Co-authored-by: Jose Javier <[email protected]>
Co-authored-by: Angel Ruiz <[email protected]>
Co-authored-by: Denny Lee <[email protected]>
Co-authored-by: Jane Zhang <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Chuck Tang <[email protected]>
Co-authored-by: Vitaliy Chiley <[email protected]>
KuuCi pushed a commit that referenced this issue Apr 18, 2024
@mvpatel2000 @abhi-mosaic
@sashaDoubov @cli99 @bigning @irenedea @ShashankMosaicML @josejg @angel-ruiz7 @dennyglee @jjanezhang @dakinggg @j316chuck @vchiley @KuuCi
MegaBlocks release (#1102)
8a39135 
* [Stage] Megablocks release (#241)

* V1 of MegaBlocks
---------

* fix hf ckptr

* rename

* lint

* lint

---------

Co-authored-by: Abhinav Venigalla <[email protected]>
Co-authored-by: Sasha Doubov <[email protected]>
Co-authored-by: Cheng Li <[email protected]>
Co-authored-by: Ning Wang <[email protected]>
Co-authored-by: Irene Dea <[email protected]>
Co-authored-by: Shashank Rajput <[email protected]>
Co-authored-by: Chuck Tang <[email protected]>
Co-authored-by: Jose Javier <[email protected]>
Co-authored-by: Angel Ruiz <[email protected]>
Co-authored-by: Denny Lee <[email protected]>
Co-authored-by: Jane Zhang <[email protected]>
Co-authored-by: Daniel King <[email protected]>
Co-authored-by: Chuck Tang <[email protected]>
Co-authored-by: Vitaliy Chiley <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@arpitg1991