Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Message and StepRun Deltas missing from eventMessage callback object #331

Closed
StephenHodgson opened this issue Jun 16, 2024 Discussed in #327 · 0 comments · Fixed by #332 or RageAgainstThePixel/com.openai.unity#254
Assignees
Labels
bug Something isn't working

Comments

@StephenHodgson
Copy link
Member

Discussed in #327

Originally posted by @rkuiper1 June 16, 2024
Today I was lucky enough to have a decent internet connection and I took a look at your streaming implementation.
I focused on ThreadsEndpoint.cs and in particular the StreamRunAsync method.
There I found that all the delta's are neatly received and processed, but not passed to the streamEventHandler, which explains the empty content we see in our logfiles.
This tiny tweak made the difference:

                        case "thread.message.incomplete":
                            var partialMessage = sseResponse.Deserialize<MessageResponse>(ssEvent, client);
                            if (message == null)
                            {
                                message = partialMessage;
                            }
                            else
                            {
                                message.AppendFrom(partialMessage);
                            }

481                         streamEventHandler?.Invoke(partialMessage);
                            return;

Line 481 originally contains: streamEventHandler?.Invoke(message); where message does not contain the actual delta.
Changing it to streamEventHandler?.Invoke(partialMessage); will pass the message event that does contain the delta.

Next, I can print the delta's in my test project like this:

                    case MessageResponse messageResponse:
                        if (messageResponse.Delta != null)
                        {
                            Console.Write($"{messageResponse.Delta.Content.FirstOrDefault()?.Text}");
                        }
                        break;

Running the test project will show the familiar chat-gpt delta streaming I was trying to accomplish.
I know this is a quick hack and I can't oversee all consequences of this change, but the way it works would be good enough for me.

@StephenHodgson StephenHodgson added the bug Something isn't working label Jun 16, 2024
@StephenHodgson StephenHodgson self-assigned this Jun 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
1 participant