[Feature Request]: Implement Service Provider Google Support Custom End Point Url

[H0llyW00dzZ]

Problem Description

In the current version, the service provider Google does not support custom endpoints, unlike the service provider OpenAI, which provides a custom endpoint URL.

Solution Description

Implementing support for custom endpoints in the service provider Google could greatly enhance performance. As an example, I've been operating this repository on Kubernetes, utilizing containers such as one-api and litellm across 13 pods. This configuration has shown to be more stable compared to deployments on Vercel or other platforms.

Alternatives Considered

No response

Additional Context

No response

[nextchat-manager]

Please follow the issue template to update title and description of your issue.

[H0llyW00dzZ]

Additional Note

Note

This implement could significantly enhance performance not only for web-based applications, but also for desktop versions.

[fred-bf]

OneAPI doesnt support Google Gemini API definition, you can diectly config the enpoint in custom endpoint(GPT like). As for model providers, the progress will be tracked in issue #4030

[H0llyW00dzZ]

OneAPI doesnt support Google Gemini API definition, you can diectly config the enpoint in custom endpoint(GPT like). As for model providers, the progress will be tracked in issue #4030

it's supported

I've been testing it running on kubernetes with better configuration

(base) root@H0llyW00dzZ:/# kubectl top pods
NAME                       CPU(cores)   MEMORY(bytes)
mysql-78mv3xmb37-hpmz5     5m           402Mi
one-api-j3laxb3o30-bz0gu   0m           46Mi

To be honest, I like the one-api RESTful APIs because they are written in Go, and they also utilize Gin.

[fred-bf]

@H0llyW00dzZ Cool! The base url of gemini api should be able to configured via GOOGLE_URL, have you encountered any issue while requesting with the variable?

[H0llyW00dzZ]

@H0llyW00dzZ Cool! The base url of gemini api should be able to configured via GOOGLE_URL, have you encountered any issue while requesting with the variable?

no, but env variable GOOGLE_URL can't be used in desktop version

for example in this one:

also desktop version it much stable when using custom end point url, plus smoothly unlike web-based applications

[H0llyW00dzZ]

The reason it can't be used in the desktop version is because it needs to be recompiled again into binaries.

[H0llyW00dzZ]

also another reason, rust is so slow when compiling it

[fred-bf]

GOOGLE_URL

@H0llyW00dzZ Cool! The base url of gemini api should be able to configured via GOOGLE_URL, have you encountered any issue while requesting with the variable?

no, but env variable GOOGLE_URL can't be used in desktop version

for example in this one:

also desktop version it much stable when using custom end point url, plus smoothly unlike web-based applications

They are same thing eventually, you can try switching the endpoint to one-api service

[H0llyW00dzZ]

GOOGLE_URL

@H0llyW00dzZ Cool! The base url of gemini api should be able to configured via GOOGLE_URL, have you encountered any issue while requesting with the variable?

no, but env variable GOOGLE_URL can't be used in desktop version
for example in this one:

also desktop version it much stable when using custom end point url, plus smoothly unlike web-based applications

They are same thing eventually, you can try switching the endpoint to one-api service

No, it doesn't work that way. When using the gemini-pro model, it utilizes Google as a service provider and it's mandatory to use this service.

https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/blob/main/app/constant.ts#L14C1-L14C77

[fred-bf]

@H0llyW00dzZ When requesting gemini api server, the client will check whether the googleUrl is configured first. If not set, will fallback to the url you just mentioned.

https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/blob/08fa22749aea8f497811f684bd9c7ef68d698666/app/api/google/%5B...path%5D/route.ts#L20

[fred-bf]

Oh, for native clients, bypassing through a proxy service is still necessary. It might need to using a proxy service like /api/cors api

[fred-bf]

@H0llyW00dzZ have you checked the developer console in development build, it much likely the requests being block by CORS right now.

[H0llyW00dzZ]

@H0llyW00dzZ have you checked the developer console in development build, it much likely the requests being block by CORS right now.

no response

I am using one-api on k8s

[H0llyW00dzZ]

when using service provider openai + model open ai it work, but gemini it doesn't work

[H0llyW00dzZ]

in one-api it work while testing

[H0llyW00dzZ]

also I am pretty sure because of this

https://github.com/ChatGPTNextWeb/ChatGPT-Next-Web/blob/08fa22749aea8f497811f684bd9c7ef68d698666/app/client/platforms/google.ts#L121C1-L129C8