[Inference API] Add API to get configuration of inference services #114862
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ymao1
force-pushed
the
ml-api-get-services
branch
4 times, most recently
from
c6b0074 to
774709a
Compare
ymao1
changed the title
ymao1
commented
Oct 23, 2024
| import static org.elasticsearch.xpack.inference.rest.Paths.TASK_TYPE; | ||
| import static org.elasticsearch.xpack.inference.rest.Paths.TASK_TYPE_INFERENCE_SERVICES_PATH; | ||
|
|
||
| @ServerlessScope(Scope.INTERNAL) |
There was a problem hiding this comment.
I set this to internal but I'm not sure whether it should be public or not.
There was a problem hiding this comment.
As long in will be possible to call it on Kibana side with esClient.transport.request. I don't see any reasons why it should be public for now. Also public APIs usually requires an extensive documentation and maintenance complexity.
ymao1
force-pushed
the
ml-api-get-services
branch
from
774709a to
a0a37ad
Compare
ymao1
force-pushed
the
ml-api-get-services
branch
from
a0a37ad to
82fd150
Compare
ymao1
added
>enhancement
:ml
|
Hi @ymao1, I've created a changelog YAML for you. |
|
Pinging @elastic/ml-core (Team:ML) |
prwhelan
reviewed
Oct 24, 2024
.../main/java/org/elasticsearch/xpack/inference/action/TransportGetInferenceServicesAction.java
Show resolved
Hide resolved
...org/elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchService.java
Outdated
Show resolved
Hide resolved
...org/elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchService.java
Outdated
Show resolved
Hide resolved
| "text_embedding", | ||
| "completion" | ||
| ], | ||
| "configuration": { |
There was a problem hiding this comment.
I think we may need to split up configuration into service_settings and task_settings? @YulNaumenko does that sound right?
For example, https://www.elastic.co/guide/en/elasticsearch/reference/current/infer-service-amazon-bedrock.html
Could be:
PUT _inference/completion/amazon_bedrock_completion
{
"service": "amazonbedrock",
"service_settings": {
"access_key": "<aws_access_key>",
"secret_key": "<aws_secret_key>",
"region": "us-east-1",
"provider": "amazontitan",
"model": "amazon.titan-text-premier-v1:0"
},
"task_settings": {
"temperature": 0.5,
"top_p": 0.5
}
}
Do we need to communicate back to the kibana connector? Or am I misunderstanding the configuration object?
There was a problem hiding this comment.
I believe this is a separate issue: https://github.com/elastic/ml-team/issues/1308. Is that correct @YulNaumenko?
There was a problem hiding this comment.
I think we can make it within the single request as @prwhelan suggested and that will help to avoid calling two separate APIs. I will need to change the Connector code to consume the new schema.
For example we can extend it by adding to each provider the taskTypes prop similar to what I requested as a separate API:
taskTypes: [
{
"task_type": string,
"configuration": {
category?: string;
default_value: string | number | boolean | null;
depends_on: Dependency[];
display: DisplayType;
label: string;
options?: SelectOption[];
order?: number | null;
placeholder?: string;
required: boolean;
sensitive: boolean;
tooltip: string | null;
type: FieldType;
ui_restrictions: string[];
validations: Validation[];
value: string | number | boolean | null;
}
}
]
|
dropping in to say that ya'll are the coolest, thanks @prwhelan and @YulNaumenko |
all credit goes to @ymao1 |
prwhelan
approved these changes
Oct 24, 2024
YulNaumenko
approved these changes
Oct 28, 2024
prwhelan
approved these changes
Oct 30, 2024
server/src/main/java/org/elasticsearch/inference/EmptySettingsConfiguration.java
Outdated
Show resolved
Hide resolved
💔 Backport failed
You can use sqren/backport to manually backport by running |
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
ymao1
added a commit
to ymao1/elasticsearch
that referenced
this pull request
Oct 30, 2024
…lastic#114862) * Adding API to get list of service configurations * Update docs/changelog/114862.yaml * Fixing some configurations * PR feedback -> Stream.of * PR feedback -> singleton * Renaming ServiceConfiguration to SettingsConfiguration. Adding TaskSettingsConfiguration * Adding task type settings configuration to response * PR feedback (cherry picked from commit 4ecdfbb) # Conflicts: # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchService.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceService.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/amazonbedrock/AmazonBedrockServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/googlevertexai/GoogleVertexAiServiceTests.java
ymao1
added a commit
that referenced
this pull request
Oct 30, 2024
…114862) (#115979) * Adding API to get list of service configurations * Update docs/changelog/114862.yaml * Fixing some configurations * PR feedback -> Stream.of * PR feedback -> singleton * Renaming ServiceConfiguration to SettingsConfiguration. Adding TaskSettingsConfiguration * Adding task type settings configuration to response * PR feedback (cherry picked from commit 4ecdfbb) # Conflicts: # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchService.java # x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/elastic/ElasticInferenceService.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/amazonbedrock/AmazonBedrockServiceTests.java # x-pack/plugin/inference/src/test/java/org/elasticsearch/xpack/inference/services/googlevertexai/GoogleVertexAiServiceTests.java
jfreden
pushed a commit
to jfreden/elasticsearch
that referenced
this pull request
Nov 4, 2024
…lastic#114862) * Adding API to get list of service configurations * Update docs/changelog/114862.yaml * Fixing some configurations * PR feedback -> Stream.of * PR feedback -> singleton * Renaming ServiceConfiguration to SettingsConfiguration. Adding TaskSettingsConfiguration * Adding task type settings configuration to response * PR feedback
YulNaumenko
added a commit
to elastic/kibana
that referenced
this pull request
Nov 12, 2024
…s ES API instead of hardcoded values. (#199047) @ymao1 [introduced](elastic/elasticsearch#114862) new ES API which allows to fetch available services providers list with the settings and task types: `GET _inference/_services` This PR is changing hardcoded providers list `x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts` to use new ES API.
tkajtoch
pushed a commit
to tkajtoch/kibana
that referenced
this pull request
Nov 12, 2024
…s ES API instead of hardcoded values. (elastic#199047) @ymao1 [introduced](elastic/elasticsearch#114862) new ES API which allows to fetch available services providers list with the settings and task types: `GET _inference/_services` This PR is changing hardcoded providers list `x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts` to use new ES API.
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Nov 18, 2024
…s ES API instead of hardcoded values. (elastic#199047) @ymao1 [introduced](elastic/elasticsearch#114862) new ES API which allows to fetch available services providers list with the settings and task types: `GET _inference/_services` This PR is changing hardcoded providers list `x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts` to use new ES API.
YulNaumenko
added a commit
to YulNaumenko/kibana
that referenced
this pull request
Dec 18, 2024
…s ES API instead of hardcoded values. (elastic#199047) @ymao1 [introduced](elastic/elasticsearch#114862) new ES API which allows to fetch available services providers list with the settings and task types: `GET _inference/_services` This PR is changing hardcoded providers list `x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts` to use new ES API. (cherry picked from commit abf6a1d)
YulNaumenko
added a commit
to YulNaumenko/kibana
that referenced
this pull request
Dec 18, 2024
…s ES API instead of hardcoded values. (elastic#199047) @ymao1 [introduced](elastic/elasticsearch#114862) new ES API which allows to fetch available services providers list with the settings and task types: `GET _inference/_services` This PR is changing hardcoded providers list `x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts` to use new ES API. (cherry picked from commit abf6a1d)
YulNaumenko
added a commit
to elastic/kibana
that referenced
this pull request
Dec 18, 2024
…ervices ES API instead of hardcoded values. (#199047) (#204680) # Backport This will backport the following commits from `main` to `8.x`: - [[Inference Connector] Modified getProvider to use _inference/_services ES API instead of hardcoded values. (#199047)](#199047) <!--- Backport version: 9.6.1 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Yuliia Naumenko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-12T01:50:29Z","message":"[Inference Connector] Modified getProvider to use _inference/_services ES API instead of hardcoded values. (#199047)\n\n@ymao1\r\n[introduced](elastic/elasticsearch#114862) new\r\nES API which allows to fetch available services providers list with the\r\nsettings and task types:\r\n`GET _inference/_services` \r\nThis PR is changing hardcoded providers list\r\n`x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts`\r\nto use new ES API.","sha":"abf6a1d4f6e83f108ed11a3e6279d73db514b352","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:skip","v9.0.0","v8.17.0"],"title":"[Inference Connector] Modified getProvider to use _inference/_services ES API instead of hardcoded values.","number":199047,"url":"https://github.com/elastic/kibana/pull/199047","mergeCommit":{"message":"[Inference Connector] Modified getProvider to use _inference/_services ES API instead of hardcoded values. (#199047)\n\n@ymao1\r\n[introduced](elastic/elasticsearch#114862) new\r\nES API which allows to fetch available services providers list with the\r\nsettings and task types:\r\n`GET _inference/_services` \r\nThis PR is changing hardcoded providers list\r\n`x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts`\r\nto use new ES API.","sha":"abf6a1d4f6e83f108ed11a3e6279d73db514b352"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199047","number":199047,"mergeCommit":{"message":"[Inference Connector] Modified getProvider to use _inference/_services ES API instead of hardcoded values. (#199047)\n\n@ymao1\r\n[introduced](elastic/elasticsearch#114862) new\r\nES API which allows to fetch available services providers list with the\r\nsettings and task types:\r\n`GET _inference/_services` \r\nThis PR is changing hardcoded providers list\r\n`x-pack/plugins/stack_connectors/public/connector_types/inference/providers/get_providers.ts`\r\nto use new ES API.","sha":"abf6a1d4f6e83f108ed11a3e6279d73db514b352"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"},{"url":"https://github.com/elastic/kibana/pull/204679","number":204679,"branch":"8.17","state":"OPEN"}]}] BACKPORT-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves https://github.com/elastic/ml-team/issues/1308, https://github.com/elastic/ml-team/issues/1309
Summary
Adds API to return a list of configurations by inference service. As recommended in the issue, the response format follows the search connectors protocol
GET _inference/_services- returns configurations for all available servicesGet _inference/_services/{taskType}- returns configuration for all services that support the given task typeExample response
GET _inference/_services