Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix non-parallel residual for GPT-NeoX #45

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

eyusupov
Copy link

@eyusupov eyusupov commented May 6, 2023

This fixes generation with RedPajama model that uses non-parallel case.

Before (after commenting out printf("use_parallel_residual == 0\n"); so that the output is handled correctly by python script):

Please enter your prompt (type 'exit' to quit): Hello
....
Hello"}).(?erfatilyettiqeleventio 1.ICAokley Sunoosersyoulippidaato0Pantonicbelelwilliottervegomeraczeasinschanzesimshdeu00fosterKristjanusmsie "their 20IDCAubojyaeras and othersD3 New Year Bailistvio1029anreds, as they’lyudnidaaerridthr Gogigafy

After:

Please enter your prompt (type 'exit' to quit): def hello_world:
...
<|BEGIN> def hello_world:
    @hello = "Hello World!"

Q:

How do I create a custom type in C++? (Not the STL one)?

My goal is to make my own class that inherits from std::list. The problem i have with this approach though, if someone asks me for example what its position number should be when we move it by 1 or 2 then I can't say 'this will work as an answer' because there are no examples of how
---------------------

def hello_world:
    @hello = "Hello World!"

Q:

How do I create a custom type in C++? (Not the STL one)?

My goal is to make my own class that inherits from std::list. The problem i have with this approach though, if someone asks me for example what its position number should be when we move it by 1 or 2 then I can't say 'this will work as an answer' because there are no examples of how

See Transofmers source code for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant