Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ML connector edits #4636

Merged
merged 22 commits into from
Aug 10, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions _ml-commons-plugin/extensibility/blueprints.md
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
layout: default
title: Building blueprints
has_children: false
nav_order: 65
parent: ML extensibility
---

# Building blueprints

All connectors consist a JSON blueprint created by ML developers to allow administrators and data scientists to make connections between OpenSearch and an AI service or model serving technology.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

For an example of a blueprint that connects to Amazon SageMaker, see the following example:
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

```json
POST /_plugins/_ml/connectors/_create
{
"name": "<YOUR MODEL NAME>",
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
"description": "<YOUR MODEL DESCRIPTION>",
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
"version": "<YOUR MODEL VERSION>",
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
"protocol": "aws_sigv4",
"credential": {
"access_key": "<PLEASE ADD YOUR AWS ACCESS KEY HERE>",
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
"secret_key": "<PLEASE ADD YOUR AWS SECRET KEY HERE>",
"session_token": "<PLEASE ADD YOUR AWS SECURITY TOKEN HERE>"
},
"parameters": {
"region": "<PLEASE ADD YOUR AWS REGION TOKEN HERE>",
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
"service_name": "sagemaker"
},
"actions": [
{
"action_type": "predict",
"method": "POST",
"headers": {
"content-type": "application/json"
},
"url": "<PLEASE ADD YOUR Sagemaker MODEL ENDPOINT URL>",
"request_body": "<PLEASE ADD YOUR REQUEST BODY. Example: ${parameters.inputs}>"
}
]
}
```


## Example blueprints

You can find blueprints for each connector in the [ML Commons repo](https://github.com/opensearch-project/ml-commons/tree/2.x/docs/remote_inference_blueprints).
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

## Configuration options

The following configuration options are **required** in order to build a connector blueprint. These settings can be used for both external and local connectors.

| Field | Data type | Description |
| :--- | :--- | :--- |
| `name` | String | The name of the connector. |
| `description` | String | A description of the connector. |
| `version` | Integer | The version of the connector. |
| `protocol` | String | The protocol for the connection. For AWS services such as Amazon SageMaker and Amazon Bedrock, use `aws_sigv4`. For all other services, use `http`. |
| `parameters` | JSON object | The default connector parameters, including `endpoint` and `model`. Any parameters indicated in this field can be overridden by parameters made in a predict request. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| `credential` | `Map<string, string>` | Defines any credential variables required to connect to your chosen endpoint. ML Commons uses **AES/GCM/NoPadding** symmetric encryption to encrypt your credentials. When the connection to the cluster first starts, OpenSearch creates a random 32 byte encryption key which persists in OpenSearch's system index. Therefore, you do not need to manually set the encryption key. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| `actions` | JSON array | Define what actions can run within the connector. If you're an administrator making a connection, add the [blueprint]({{site.url}}{{site.baseurl}}/ml-commons-plugin/extensibility/blueprints/) for your desired connection. |
| `backend_roles` | JSON array | A list of OpenSearch backend roles. For more information about setting up backend roles, see [Assigning backend roles to users]({{site.url}}{{site.baseurl}}/ml-commons-plugin/model-access-control#assigning-backend-roles-to-users). |
| `access_mode` | String | Sets the access mode for the model, either `public`, `restricted`, or `private`. Default is `private`. For more information about `access_mode`, see [Model groups]({{site.url}}{{site.baseurl}}/ml-commons-plugin/model-access-control#model-groups). |
| `add_all_backend_roles` | Boolean | When set to `true`, adds all `backend_roles` to the access list, which only a user with admin permissions can adjust. When set to `false`, non-admins can add `backend_roles`. |

The `action` parameter supports the following options.

| Field | Data type | Description |
| :--- | :--- | :--- |
| `action_type` | String | Required. Sets the ML Commons API operation to use upon connection. As of OpenSearch 2.9, only `predict` is supported. |
| `method` | String | Required. Defines the HTTP method for the API call. Supports `POST` and `GET`. |
| `url` | String | Required. Sets the connection endpoint at which the action takes place. This must match the regex expression for the connection used when [adding trusted endpoints]({{site.url}}{{site.baseurl}}/ml-commons-plugin/extensibility/index#adding-trusted-endpoints). |
| `headers` | JSON object | Sets the headers used inside the request or response body. Default is `ContentType: application/json`. If your third-part ML tool requires access control, define any `credential` parameters need in the `headers` parameter. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| `request_body` | String | Required. Sets the parameters contained inside the request body of the action. The parameters must include `\"inputText\`, which sets how users of the connector should construct the request payload for the `action_type`. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

## Next step

To see how sysadmins and data scientists use blueprints for connectors, see [Creating connectors for third-party ML platforms]({{site.url}}{{site.baseurl}}/ml-commons-plugin/extensibility/connectors/).
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Loading