Error while copying content to a stream

[sebastianbk]

First of all, thank you so much for creating this library! You've done an outstanding job and made it so much easier to get started with Gemini as a .NET developer. Amazing work!

I've done some extensive testing and I continuously experience the error. It's quite erratic and I can't really tell why the error is being thrown. It seems random to me.

Is there anything I can do to avoid this?

Thanks!

Error while copying content to a stream.
System.Net.Http.HttpRequestException: Error while copying content to a stream.
      ---> System.Net.Http.HttpIOException: The response ended prematurely while waiting for the next frame from the server. (ResponseEnded)
      at System.Net.Http.Http2Connection.ThrowRequestAborted(Exception innerException)
      at System.Net.Http.Http2Connection.Http2Stream.TryReadFromBuffer(Span`1 buffer, Boolean partOfSyncRead)
      at System.Net.Http.Http2Connection.Http2Stream.CopyToAsync(HttpResponseMessage responseMessage, Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
      at System.Net.Http.HttpConnectionResponseContent.<SerializeToStreamAsync>g__Impl|6_0(Stream stream, CancellationToken cancellationToken)
      at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
      --- End of inner exception stack trace ---
      at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
      at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
      at Mscc.GenerativeAI.GenerativeModel.GenerateContent(GenerateContentRequest request)
      at Mscc.GenerativeAI.GenerativeModel.GenerateContent(String prompt, GenerationConfig generationConfig, List`1 safetySettings, List`1 tools, ToolConfig toolConfig)

[jochenkirstaetter]

Hello @sebastianbk

Yes, that's an issue with .NET itself and how the streaming content is received from the Google server. There is quite some discussion in the .NET source code about that. The root cause is that there seems to be too much "chatter" from the HttpClient and therefore the server closes the connection. I actually commented the references in the source code:

// Ref: dotnet/runtime#97128 - HttpIOException
// grpc/grpc-dotnet#2361 (comment)

Currently, a potential workaround might be to switch to the "SSE" mode as Google calls it. It's based on the Server-Sent Events concept although it uses the response format only.

See test case

generative-ai/tests/Mscc.GenerativeAI/GoogleAi_GeminiPro_Should.cs

Line 449 in 96a4602

public async void GenerateContent_Stream_WithRequest_ServerSentEvents()

on how to use this format.

Hope this helps!

Cheers, JoKi

[jochenkirstaetter]

@sebastianbk

After I upgraded to the latest .NET 8 SDK and runtime, I observed that the issue seems to be less likely. However, the .NET team targets primarily .NET 9 with additional HTTP response exceptions that would provide the details.

Cheers, JoKi