Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] add new delete trained model aliases API #69195

Merged
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
[role="xpack"]
[testenv="platinum"]
[[delete-trained-models-aliases]]
= Delete Trained Models Aliases API
[subs="attributes"]
++++
<titleabbrev>Delete Trained Models Aliases</titleabbrev>
++++

Deletes a trained models alias.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it "model's" possessive or singular "Deletes a trained model alias"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definitely


beta::[]

[[ml-delete-trained-models-aliases-request]]
== {api-request-title}

`DELETE _ml/trained_models/<model_id>/model_aliases/<model_alias>`


[[ml-delete-trained-models-aliases-prereq]]
== {api-prereq-title}

If the {es} {security-features} are enabled, you must have the following
built-in roles and privileges:

* `machine_learning_admin`

For more information, see <<built-in-roles>>, <<security-privileges>>, and
{ml-docs-setup-privileges}.

[[ml-delete-trained-models-aliases-desc]]
== {api-description-title}

This API deletes an existing model alias that refers to trained models.

If the model alias is missing or refers to a model other than the one identified by
the `model_id`, this API will return an error.

[[ml-delete-trained-models-aliases-path-params]]
== {api-path-parms-title}

`model_id`::
(Required, string)
The trained model ID to which the model alias refers.

`model_alias`::
(Required, string)
The model alias to delete.

[[ml-delete-trained-models-aliases-example]]
== {api-examples-title}

[[ml-delete-trained-models-aliases-example-delete]]
=== Deleting a model alias

The following example shows how to delete a model alias for a trained model ID.

[source,console]
--------------------------------------------------
DELETE _ml/trained_models/flight-delay-prediction-1574775339910/model_aliases/flight_delay_model
--------------------------------------------------
// TEST[skip:setup kibana sample data]
1 change: 1 addition & 0 deletions docs/reference/ml/df-analytics/apis/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ include::update-dfanalytics.asciidoc[leveloffset=+2]
//DELETE
include::delete-dfanalytics.asciidoc[leveloffset=+2]
include::delete-trained-models.asciidoc[leveloffset=+2]
include::delete-trained-models-aliases.asciidoc[leveloffset=+2]
//EVALUATE
include::evaluate-dfanalytics.asciidoc[leveloffset=+2]
//ESTIMATE_MEMORY_USAGE
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ You can use the following APIs to perform {infer} operations.
* <<get-trained-models-stats>>
* <<delete-trained-models>>
* <<put-trained-models-aliases>>
* <<delete-trained-models-aliases>>

You can deploy a trained model to make predictions in an ingest pipeline or in
an aggregation. Refer to the following documentation to learn more.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

package org.elasticsearch.xpack.core.ml.action;

import org.elasticsearch.action.ActionRequestValidationException;
import org.elasticsearch.action.ActionType;
import org.elasticsearch.action.support.master.AcknowledgedRequest;
import org.elasticsearch.action.support.master.AcknowledgedResponse;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.xpack.core.ml.inference.TrainedModelConfig;
import org.elasticsearch.xpack.core.ml.utils.ExceptionsHelper;

import java.io.IOException;
import java.util.Objects;


public class DeleteTrainedModelAliasAction extends ActionType<AcknowledgedResponse> {

public static final DeleteTrainedModelAliasAction INSTANCE = new DeleteTrainedModelAliasAction();
public static final String NAME = "cluster:admin/xpack/ml/inference/model_aliases/delete";

private DeleteTrainedModelAliasAction() {
super(NAME, AcknowledgedResponse::readFrom);
}

public static class Request extends AcknowledgedRequest<Request> {

public static final String MODEL_ALIAS = "model_alias";

private final String modelAlias;
private final String modelId;

public Request(String modelAlias, String modelId) {
this.modelAlias = ExceptionsHelper.requireNonNull(modelAlias, MODEL_ALIAS);
this.modelId = ExceptionsHelper.requireNonNull(modelId, TrainedModelConfig.MODEL_ID);
}

public Request(StreamInput in) throws IOException {
super(in);
this.modelAlias = in.readString();
this.modelId = in.readString();
}

public String getModelAlias() {
return modelAlias;
}

public String getModelId() {
return modelId;
}

@Override
public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
out.writeString(modelAlias);
out.writeString(modelId);
}

@Override
public ActionRequestValidationException validate() {
return null;
}

@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Request request = (Request) o;
return Objects.equals(modelAlias, request.modelAlias)
&& Objects.equals(modelId, request.modelId);
}

@Override
public int hashCode() {
return Objects.hash(modelAlias, modelId);
}

}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
package org.elasticsearch.xpack.core.ml.action;

import org.elasticsearch.common.io.stream.Writeable;
import org.elasticsearch.test.AbstractWireSerializingTestCase;
import org.elasticsearch.xpack.core.ml.action.DeleteTrainedModelAliasAction.Request;


public class DeleteTrainedModelAliasActionRequestTests extends AbstractWireSerializingTestCase<Request> {

@Override
protected Request createTestInstance() {
return new Request(randomAlphaOfLength(10), randomAlphaOfLength(10));
}

@Override
protected Writeable.Reader<Request> instanceReader() {
return Request::new;
}

public void testCtor() {
expectThrows(Exception.class, () -> new Request(null, randomAlphaOfLength(10)));
expectThrows(Exception.class, () -> new Request(randomAlphaOfLength(10), null));
}

}
2 changes: 2 additions & 0 deletions x-pack/plugin/ml/qa/ml-with-security/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,8 @@ tasks.named("yamlRestTest").configure {
'ml/inference_crud/Test update model alias with bad alias',
'ml/inference_crud/Test update model alias where alias exists but old model id is different inference type',
'ml/inference_crud/Test update model alias where alias exists but reassign is false',
'ml/inference_crud/Test delete model alias with missing alias',
'ml/inference_crud/Test delete model alias where alias points to different model',
'ml/inference_processor/Test create processor with missing mandatory fields',
'ml/inference_stats_crud/Test get stats given missing trained model',
'ml/inference_stats_crud/Test get stats given expression without matches and allow_no_match is false',
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,27 @@ public void testDeleteModelWhileAliasReferencedByPipeline() throws Exception {
waitForStats();
}

public void testDeleteModelAliasWhileAliasReferencedByPipeline() throws Exception {
putRegressionModel();
putModelAlias("regression_to_delete", MODEL_ID);
createdPipelines.add("first_pipeline");
putPipeline("regression_to_delete", "first_pipeline");
Exception ex = expectThrows(Exception.class,
() -> client().performRequest(
new Request(
"DELETE",
"_ml/trained_models/" + MODEL_ID + "/model_aliases/regression_to_delete"
)
));
assertThat(
ex.getMessage(),
containsString("Cannot delete model_alias [regression_to_delete] as it is still referenced by ingest processors")
);
infer("first_pipeline");
deletePipeline("first_pipeline");
waitForStats();
}

public void testDeleteModelWhileReferencedByPipeline() throws Exception {
putRegressionModel();
createdPipelines.add("first_pipeline");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@
import org.elasticsearch.xpack.core.ml.action.DeleteJobAction;
import org.elasticsearch.xpack.core.ml.action.DeleteModelSnapshotAction;
import org.elasticsearch.xpack.core.ml.action.DeleteTrainedModelAction;
import org.elasticsearch.xpack.core.ml.action.DeleteTrainedModelAliasAction;
import org.elasticsearch.xpack.core.ml.action.EstimateModelMemoryAction;
import org.elasticsearch.xpack.core.ml.action.EvaluateDataFrameAction;
import org.elasticsearch.xpack.core.ml.action.ExplainDataFrameAnalyticsAction;
Expand Down Expand Up @@ -170,6 +171,7 @@
import org.elasticsearch.xpack.ml.action.TransportDeleteJobAction;
import org.elasticsearch.xpack.ml.action.TransportDeleteModelSnapshotAction;
import org.elasticsearch.xpack.ml.action.TransportDeleteTrainedModelAction;
import org.elasticsearch.xpack.ml.action.TransportDeleteTrainedModelAliasAction;
import org.elasticsearch.xpack.ml.action.TransportEstimateModelMemoryAction;
import org.elasticsearch.xpack.ml.action.TransportEvaluateDataFrameAction;
import org.elasticsearch.xpack.ml.action.TransportExplainDataFrameAnalyticsAction;
Expand Down Expand Up @@ -317,6 +319,7 @@
import org.elasticsearch.xpack.ml.rest.filter.RestPutFilterAction;
import org.elasticsearch.xpack.ml.rest.filter.RestUpdateFilterAction;
import org.elasticsearch.xpack.ml.rest.inference.RestDeleteTrainedModelAction;
import org.elasticsearch.xpack.ml.rest.inference.RestDeleteTrainedModelAliasAction;
import org.elasticsearch.xpack.ml.rest.inference.RestGetTrainedModelsAction;
import org.elasticsearch.xpack.ml.rest.inference.RestGetTrainedModelsStatsAction;
import org.elasticsearch.xpack.ml.rest.inference.RestPutTrainedModelAction;
Expand Down Expand Up @@ -939,6 +942,7 @@ public List<RestHandler> getRestHandlers(Settings settings, RestController restC
new RestPutTrainedModelAction(),
new RestUpgradeJobModelSnapshotAction(),
new RestPutTrainedModelAliasAction(),
new RestDeleteTrainedModelAliasAction(),
// CAT Handlers
new RestCatJobsAction(),
new RestCatTrainedModelsAction(),
Expand Down Expand Up @@ -1023,6 +1027,7 @@ public List<RestHandler> getRestHandlers(Settings settings, RestController restC
new ActionHandler<>(PutTrainedModelAction.INSTANCE, TransportPutTrainedModelAction.class),
new ActionHandler<>(UpgradeJobModelSnapshotAction.INSTANCE, TransportUpgradeJobModelSnapshotAction.class),
new ActionHandler<>(PutTrainedModelAliasAction.INSTANCE, TransportPutTrainedModelAliasAction.class),
new ActionHandler<>(DeleteTrainedModelAliasAction.INSTANCE, TransportDeleteTrainedModelAliasAction.class),
usageAction,
infoAction);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ protected void masterOperation(Task task,
ActionListener<AcknowledgedResponse> listener) {
String id = request.getId();
IngestMetadata currentIngestMetadata = state.metadata().custom(IngestMetadata.TYPE);
Set<String> referencedModels = getReferencedModelKeys(currentIngestMetadata);
Set<String> referencedModels = getReferencedModelKeys(currentIngestMetadata, ingestService);

if (referencedModels.contains(id)) {
listener.onFailure(new ElasticsearchStatusException("Cannot delete model [{}] as it is still referenced by ingest processors",
Expand Down Expand Up @@ -142,7 +142,7 @@ public ClusterState execute(final ClusterState currentState) {
});
}

private Set<String> getReferencedModelKeys(IngestMetadata ingestMetadata) {
static Set<String> getReferencedModelKeys(IngestMetadata ingestMetadata, IngestService ingestService) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is used also in the transport delete trained model alias class to gather all the referenced models to make sure a model alias is not used by a pipeline

Set<String> allReferencedModelKeys = new HashSet<>();
if (ingestMetadata == null) {
return allReferencedModelKeys;
Expand Down
Loading