Update docs for ai.proto #10358
Update docs for ai.proto #10358
Conversation
...erence/api/github.com/solo-io/gloo/projects/gloo/api/v1/enterprise/options/ai/ai.proto.sk.md
Outdated
Show resolved
Hide resolved
…oo/api/v1/enterprise/options/ai/ai.proto.sk.md Co-authored-by: Nadine Spies <[email protected]>
|
Issues linked to changelog: |
|
Visit the preview URL for this PR (updated for commit d43ffee): https://gloo-edge--pr10358-rlg-ai-proto-fjg0h6cb.web.app (expires Tue, 03 Dec 2024 19:26:56 GMT) 🔥 via Firebase Hosting GitHub Action 🌎 Sign: 77c2b86e287749579b7ff9cadb81e099042ef677 |
...erence/api/github.com/solo-io/gloo/projects/gloo/api/v1/enterprise/options/ai/ai.proto.sk.md
Outdated
Show resolved
Hide resolved
| message SingleAuthToken { | ||
|
|
||
| // Passthrough the existing token. This token can either |
There was a problem hiding this comment.
repeating this from the comments inside auth_token_source sounds a little odd to me. The is the configuration when Passthrough is selected as the auth_token_source. It just happens to be empty now. In the future, for example, we could put the header name that contains the token here so it doesn't need to be in the Authorization header. So, maybe something along this line.
There was a problem hiding this comment.
@andy-fong To make sure I'm understanding - do you prefer that I leave this field description blank here? I dont want to promise anything that we could or might put here in the future, so if anything i think i could just remove this comment. Is that what you mean?
There was a problem hiding this comment.
Maybe something like this:
// Confirguration for Passthrough the existing token. Currently, the present of an empty object will indicate Passthrough is selected
| // Auth Token to use for the Azure OpenAI API | ||
| // This token will be placed into the `api-key` header | ||
| // The authorization token that the AI gateway uses to access the Azure OpenAI API. | ||
| // This token is automatically sent in the `api-key` header of the request. | ||
| oneof auth_token_source { |
There was a problem hiding this comment.
So, this is not a documentation issue but I just notice this is odd. Inside SingleAuthToken there is already a oneof auth_token_source. Here unlike the OpenAI (which I think is correct) has the oneof outside and then the oneof inside? @EItanya @npolshakova is this right?
Same for the others. If this is correct, should we change the OpenAI to match this so it's consistent?
| // Auth Token to use for the Gemini API | ||
| // This token will be placed into the `key` header | ||
| // The authorization token that the AI gateway uses to access the Gemini API. | ||
| // This token is automatically sent in the `key` header of the request. |
There was a problem hiding this comment.
The example from Gemini doc always put the key in the key qs param (like in the example above) and not the header. In our code, we do both but I cannot find any reference that Gemini api will accept the key in the key header but it's hard to find anything in the Gemini api doc. @npolshakova @EItanya can you confirm this is correct?
There was a problem hiding this comment.
Yep, for Gemini (not Vertex-AI) you need to pass the API token in query param. For vertex ai, you need the authorization token in the header.
| message OpenAI { | ||
| oneof auth_token_source { | ||
| // The authorization token that the AI gateway uses to access the OpenAI API. | ||
| // This token is automatically sent in the `Authorization` header of the | ||
| // request and prefixed with `Bearer`. | ||
| SingleAuthToken auth_token = 1; | ||
| // re-use the token from the backend | ||
| // google.protobuf.Empty inherit_backend_token = 3; | ||
| } |
There was a problem hiding this comment.
@EItanya do we need a way to specify the model here?
| uint32 grpc_port = 3; | ||
| // Whether or not to use a secure connection, true by default | ||
| // Whether to use a secure connection. Defaults to `true`. |
There was a problem hiding this comment.
This is a bit confusing and a bit against the convention where the default is always 0, "" or false. In the code, I also do not find where we set this to true by default (maybe I missed it). Maybe the original comment means secure connection by default? @EItanya can you confirm?
Also, it looks like this setting applies to both the http and grpc connections.
| message Webhook { | ||
| // Host to send the traffic to. | ||
| string host = 1; | ||
| // Port to send the traffic to | ||
| uint32 port = 2; | ||
| // Describes how to match a given string in HTTP headers. Match is case-sensitive. |
There was a problem hiding this comment.
Not a document issue. case-sensitive will never work because HTTP/1 can be camel case but HTTP/2 is always lower case. Also, envoy lower case the header before sending to ext_proc, so the user really need to enter all lower case here for it to work. So, case-sensitive match is incorrect. HTTP header name should be case-insensitive. Should we open an issue to fix this, @EItanya
Co-authored-by: Andy Fong <[email protected]>
Updates the comments in the AI gateway proto to clean up the api ref doc.
BOT NOTES:
resolves https://github.com/solo-io/docs/issues/624