From df091fc84372a73475c6cded9de6014312ea867a Mon Sep 17 00:00:00 2001 From: Xun Zhang Date: Wed, 20 Mar 2024 14:52:54 -0700 Subject: [PATCH 1/5] update documentation for automatic remote model deployment Signed-off-by: Xun Zhang --- _ml-commons-plugin/api/model-apis/deploy-model.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/_ml-commons-plugin/api/model-apis/deploy-model.md b/_ml-commons-plugin/api/model-apis/deploy-model.md index 52cf3f232e..b43d880a6b 100644 --- a/_ml-commons-plugin/api/model-apis/deploy-model.md +++ b/_ml-commons-plugin/api/model-apis/deploy-model.md @@ -8,7 +8,17 @@ nav_order: 20 # Deploy a model -The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. +The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, this API can be bypassed and the model can be deployed automatically when it's predicted the first time through the Predict API. To enable the automatic remote model deployment, please enable the following cluster setting: + +```json +PUT _cluster/settings +{ + "persistent": { + "plugins.ml_commons.model_auto_deploy.enable": "true" + } +} +``` +{% include copy-curl.html %} For information about user access for this API, see [Model access control considerations]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/model-apis/index/#model-access-control-considerations). From 1a504f01f5c525dbe71c38b2f7ab3a73bfe58bed Mon Sep 17 00:00:00 2001 From: Xun Zhang Date: Wed, 20 Mar 2024 16:45:52 -0700 Subject: [PATCH 2/5] add automatic deploy doc in connecting to externall hosted models Signed-off-by: Xun Zhang --- _ml-commons-plugin/api/model-apis/deploy-model.md | 4 ++-- _ml-commons-plugin/remote-models/index.md | 13 ++++++++++++- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/_ml-commons-plugin/api/model-apis/deploy-model.md b/_ml-commons-plugin/api/model-apis/deploy-model.md index b43d880a6b..3233b30ab3 100644 --- a/_ml-commons-plugin/api/model-apis/deploy-model.md +++ b/_ml-commons-plugin/api/model-apis/deploy-model.md @@ -8,13 +8,13 @@ nav_order: 20 # Deploy a model -The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, this API can be bypassed and the model can be deployed automatically when it's predicted the first time through the Predict API. To enable the automatic remote model deployment, please enable the following cluster setting: +The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, by default the model is deployed automatically when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index). ```json PUT _cluster/settings { "persistent": { - "plugins.ml_commons.model_auto_deploy.enable": "true" + "plugins.ml_commons.model_auto_deploy.enable": "false" } } ``` diff --git a/_ml-commons-plugin/remote-models/index.md b/_ml-commons-plugin/remote-models/index.md index 0b9c6d03ed..1b592208dd 100644 --- a/_ml-commons-plugin/remote-models/index.md +++ b/_ml-commons-plugin/remote-models/index.md @@ -205,7 +205,18 @@ Take note of the returned `model_id` because you’ll need it to deploy the mode ## Step 4: Deploy the model -To deploy the registered model, provide its model ID from step 3 in the following request: +From 2.13, we support automatically deploy remote model by default so this step can be skipped. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false +```json +PUT _cluster/settings +{ + "persistent": { + "plugins.ml_commons.model_auto_deploy.enable" : "false" + } +} +``` +{% include copy-curl.html %} + +To deploy the registered model manually, provide its model ID from step 3 in the following request: ```bash POST /_plugins/_ml/models/cleMb4kBJ1eYAeTMFFg4/_deploy From 1fc0f2e77a02f21c1185daf9e24648cb3a1ada99 Mon Sep 17 00:00:00 2001 From: Xun Zhang Date: Thu, 21 Mar 2024 14:14:49 -0700 Subject: [PATCH 3/5] add remind to undeploy model Signed-off-by: Xun Zhang --- _ml-commons-plugin/api/model-apis/deploy-model.md | 2 +- _ml-commons-plugin/remote-models/index.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/_ml-commons-plugin/api/model-apis/deploy-model.md b/_ml-commons-plugin/api/model-apis/deploy-model.md index 3233b30ab3..ecef44ed84 100644 --- a/_ml-commons-plugin/api/model-apis/deploy-model.md +++ b/_ml-commons-plugin/api/model-apis/deploy-model.md @@ -8,7 +8,7 @@ nav_order: 20 # Deploy a model -The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, by default the model is deployed automatically when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index). +The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, from 2.13 the model is deployed automatically by default when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index). ```json PUT _cluster/settings diff --git a/_ml-commons-plugin/remote-models/index.md b/_ml-commons-plugin/remote-models/index.md index 1b592208dd..02e545eefc 100644 --- a/_ml-commons-plugin/remote-models/index.md +++ b/_ml-commons-plugin/remote-models/index.md @@ -205,7 +205,7 @@ Take note of the returned `model_id` because you’ll need it to deploy the mode ## Step 4: Deploy the model -From 2.13, we support automatically deploy remote model by default so this step can be skipped. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false +From 2.13, we support automatically deploy remote model by default so this step can be skipped. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false. To undeploy the model, please use the undeploy API to undeploy it. ```json PUT _cluster/settings { From 876f41087600a293cd421164b932c2affb114df7 Mon Sep 17 00:00:00 2001 From: Xun Zhang Date: Fri, 22 Mar 2024 15:30:32 -0700 Subject: [PATCH 4/5] address comments Signed-off-by: Xun Zhang --- _ml-commons-plugin/api/model-apis/deploy-model.md | 4 +++- _ml-commons-plugin/remote-models/index.md | 4 ++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/_ml-commons-plugin/api/model-apis/deploy-model.md b/_ml-commons-plugin/api/model-apis/deploy-model.md index ecef44ed84..b6cbd45218 100644 --- a/_ml-commons-plugin/api/model-apis/deploy-model.md +++ b/_ml-commons-plugin/api/model-apis/deploy-model.md @@ -8,7 +8,9 @@ nav_order: 20 # Deploy a model -The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. For remote models, from 2.13 the model is deployed automatically by default when it's predicted the first time through the Predict API. You can disable the remote model auto deployment by setting plugins.ml_commons.model_auto_deploy.enable to false. To learn more about remote models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index). +The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. + +Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`: ```json PUT _cluster/settings diff --git a/_ml-commons-plugin/remote-models/index.md b/_ml-commons-plugin/remote-models/index.md index 02e545eefc..657d7254be 100644 --- a/_ml-commons-plugin/remote-models/index.md +++ b/_ml-commons-plugin/remote-models/index.md @@ -205,7 +205,7 @@ Take note of the returned `model_id` because you’ll need it to deploy the mode ## Step 4: Deploy the model -From 2.13, we support automatically deploy remote model by default so this step can be skipped. You can disable it by setting plugins.ml_commons.model_auto_deploy.enable as false. To undeploy the model, please use the undeploy API to undeploy it. +Starting with OpenSearch version 2.13, externally hosted models are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`: ```json PUT _cluster/settings { @@ -216,7 +216,7 @@ PUT _cluster/settings ``` {% include copy-curl.html %} -To deploy the registered model manually, provide its model ID from step 3 in the following request: +To undeploy the model, use the [Undeploy API]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/model-apis/undeploy-model/). ```bash POST /_plugins/_ml/models/cleMb4kBJ1eYAeTMFFg4/_deploy From e17868186aed01e4a12ba8053ec452faa7628b5c Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Mon, 25 Mar 2024 09:08:06 -0400 Subject: [PATCH 5/5] Update _ml-commons-plugin/api/model-apis/deploy-model.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _ml-commons-plugin/api/model-apis/deploy-model.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_ml-commons-plugin/api/model-apis/deploy-model.md b/_ml-commons-plugin/api/model-apis/deploy-model.md index b6cbd45218..2c6991ba22 100644 --- a/_ml-commons-plugin/api/model-apis/deploy-model.md +++ b/_ml-commons-plugin/api/model-apis/deploy-model.md @@ -8,7 +8,7 @@ nav_order: 20 # Deploy a model -The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache into memory. This operation requires the `model_id`. +The deploy model operation reads the model's chunks from the model index and then creates an instance of the model to cache in memory. This operation requires the `model_id`. Starting with OpenSearch version 2.13, [externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index) are deployed automatically by default when you send a Predict API request for the first time. To disable automatic deployment for an externally hosted model, set `plugins.ml_commons.model_auto_deploy.enable` to `false`: