GenerateContent method hangs

[Ar4ics]

I dont know if this issue is just mine or not, but for me the method GenerateContentStream is working fine (just 3-5 seconds waiting for completion of the request), while GenerateContent sometimes hangs and throws
Connection handshake was canceled due to the configured timeout of 00:00:10 seconds elapsing. (generativelanguage.googleapis.com:443)

These 2 methods Im using:

    private static async Task<string> GetResponse(GenerativeModel model, Mscc.GenerativeAI.GenerationConfig generationConfig, string prompt, CancellationToken cancellationToken)
    {
        var sb = new StringBuilder();
        await foreach (var streamedResponse in model.GenerateContentStream(prompt, generationConfig).WithCancellation(cancellationToken))
        {
            sb.Append(streamedResponse.Text ?? string.Empty);
        }
        var response = sb.ToString();
        return response;
    }
    
    private static async Task<string> GetResponse1(GenerativeModel model, Mscc.GenerativeAI.GenerationConfig generationConfig, string prompt)
    {
        var response = await model.GenerateContent(prompt, generationConfig);
        return response.Text ?? string.Empty;
    }

Im using model Gemini 2.0 Flash Exp, other settings are default.
I wonder, is there a significant difference in the implementation of Stream and non-Stream methods?

[jochenkirstaetter]

Hi @Ar4ics

That's an issue with HTTP/3 or QUIC support in .NET.
I'm also facing this issue sometimes.

Right now, I'd suggest to disable HTTP/3 support in your application. This can be done in the project file or in the AppContext.

<ItemGroup>
    <RuntimeHostConfigurationOption Include="System.Net.SocketsHttpHandler.Http3Support" Value="false" />
</ItemGroup>

or

AppContext.SetSwitch("System.Net.SocketsHttpHandler.Http3Support", false);

This has been discussed here #34

See also here: https://learn.microsoft.com/en-us/dotnet/core/extensions/httpclient-http3

[Ar4ics]

@jochenkirstaetter understoood, thanks.

[jochenkirstaetter]

Hi @Ar4ics

I wonder, is there a significant difference in the implementation of Stream and non-Stream methods?

Slightly only. ;-)
The non-streaming method works straight sending a PostAsync with StringContent and receives the complete response from the Gemini API whereas the streaming method uses SendAsync with StreamContent and yields the response into an IAsyncEnumerable return type.

You might check whether model.UseServerSentEventsFormat could be of assistance.

Cheers, JoKi

[Ar4ics]

@jochenkirstaetter also for example I am passing cancellation token .WithCancellation(token) for streaming method, but for non-streaming it is currently not possible?

[jochenkirstaetter]

Hi @Ar4ics
Yes, there's currently no CancellationToken in the non-streaming methods. I'm trying to get the picture / use case when and how this would be relevant?

Once the request is on the wire waiting for the response... How would you issue a cancellation?

You got some sample? Maybe we take this to another issue or as a discussion.

Cheers, JoKi

[Ar4ics]

@jochenkirstaetter actually its not big deal for me.
For example prevoiusly I had this code to call Gemini Api:

        var response = await restClient.PostAsync<GptResponse>(restRequest, cancellationToken);

As you can see, RestSharp's RestClient type supports cancellation.

[jochenkirstaetter]

Thanks, I'll have a look.
Maybe I missed something.

[jochenkirstaetter]

Hi @Ar4ics

This had been improved in Release v2.1.0

Hope this helps. JoKi