[azopenai] Azure OpenAI first entry is empty when streaming chat completions or completions

[richardpark-msft]

Using 2023-06-01-preview, the first entry that comes back from the service is empty. Skipping it allows things to work but this isn't needed for OpenAI, just Azure OpenAI.

This has to be fixed before release.

[richardpark-msft]

I have more information, and I believe we can close this.

The "bug" here was actually an interesting side-effect of content-filtering. in Azure OpenAI events can carry prompt annotations, which give you information on when content has been filtered and why (ie, violence, etc..). When this happens it's possible you get an "empty" event, but in reality it's an event that's just carrying metadata, not content.

I think we can close this - the example and our tests have been adjusted to handle this.

[jacoblee93]

So future models will have this behavior?

[richardpark-msft]

Currently (ie, as of my next PR).

We're discussing how to represent this because, without prior knowledge, it's not clear that an empty Choices array is expected.

I'm curious on your opinion on this, if you don't mind.

[jacoblee93]

It's not related to the Go SDK directly but it's a change in behavior from core OpenAI and breaks handling for LangChain's wrappers:

langchain-ai/langchainjs#1914

It's possible to account for - just didn't want to do so before knowing if it was here to stay or if it was just part of preview. I'd personally prefer to keep it the same but I lack context as to why the change was made.

[m-hamaro]

Will the response format change again from 06-01-preview to 07-01?
What is the current discussion?

[richardpark-msft]

Reopening and adding @trrwilson to aid in tracking.

[richardpark-msft]

Closing until there's an update to share.