Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow the use a fully custom base address #233

Closed
wants to merge 1 commit into from

Conversation

gpt4thewin
Copy link

I've added an extra constructor to specify a custom base address.
Useful for :

  • Reverse proxies
  • OpenAI compatible APIs (like Mistral)

@StephenHodgson
Copy link
Member

StephenHodgson commented Feb 4, 2024

Use built in OpenAI-DotNet-Proxy

@gpt4thewin
Copy link
Author

  1. "Reverse proxies": It's not the same thing
  2. "OpenAI compatible APIs (like Mistral)": What about that ?

@StephenHodgson
Copy link
Member

StephenHodgson commented Feb 4, 2024

Let's open a discussion about it first.

@gpt4thewin
Copy link
Author

gpt4thewin commented Feb 4, 2024

Let me develop:

Case 1

Say I want to use my local proxy (made with nginx, for caching). It does not receive connections with ssl/https. Why should it ?

I will do that:

var baseUrl = "localhost:81";
var apiKey = new OpenAIAuthentication(myApiKey);
var settings = new OpenAIClientSettings(domain: baseUrl);

But it will (obviously) raise this: One or more errors occurred. (The SSL connection could not be established, see inner exception.)

Case 2
... Forget about it. Specifying domain "api.mistral.ai" is enough

@ADefWebserver
Copy link
Collaborator

People want to use local models like this?
https://lmstudio.ai/

@StephenHodgson
Copy link
Member

updated apiKey prefix checking in #234

@gpt4thewin
Copy link
Author

As far as I understand this change, this still forces you to use https. Correct ?

@StephenHodgson
Copy link
Member

As far as I understand this change, this still forces you to use https. Correct ?

Why wouldn't you use HTTPS?

@gpt4thewin
Copy link
Author

gpt4thewin commented Feb 6, 2024

Say I want to use my local proxy (made with nginx, for caching). It does not receive connections with ssl/https. Why should it ?

@StephenHodgson
Copy link
Member

It's just good practice. Even for the local loopback

@StephenHodgson
Copy link
Member

I also designed this library to work specifically with open AI and open AI for azure. I will not be compromising security for this edge case when a local LLM offering from open AI does not exist.

@RageAgainstThePixel RageAgainstThePixel locked as resolved and limited conversation to collaborators Feb 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants