feat: add monitor_window to ModelDeploymentMonitoringScheduleConfig p…

…roto in aiplatform v1/v1beta1 model_deployment_monitoring_job.proto

PiperOrigin-RevId: 453226330
Source-Link: googleapis/googleapis@239db5e
Source-Link: https://github.com/googleapis/googleapis-gen/commit/4cf0c542505025be51be9e67120f0413415e7d67
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNGNmMGM1NDI1MDUwMjViZTUxYmU5ZTY3MTIwZjA0MTM0MTVlN2Q2NyJ9
See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
feat: add env to ContainerSpec in aiplatform v1beta1 custom_job.proto
PiperOrigin-RevId: 453732998
Source-Link: googleapis/googleapis@a95a55f
Source-Link: https://github.com/googleapis/googleapis-gen/commit/e2ea25266cc6608d2180843dde9051b4ba37cd9b
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZTJlYTI1MjY2Y2M2NjA4ZDIxODA4NDNkZGU5MDUxYjRiYTM3Y2Q5YiJ9
feat: add successful_forecast_point_count to CompletionStats in aiplatform v1 completion_stats.proto
feat: add neighbors to Explanation in aiplatform v1 explanation.proto
feat: add examples_override to ExplanationSpecOverride in aiplatform v1 explanation.proto
feat: add version_id, version_aliases, version_create_time, version_update_time, and version_description to aiplatform v1 model.proto
feat: add ModelVersion CRUD methods in aiplatform v1 model_service.proto
feat: add model_id and parent_model to TrainingPipeline in aiplatform v1 training_pipeline.proto
PiperOrigin-RevId: 453734478
Source-Link: googleapis/googleapis@84720fb
Source-Link: https://github.com/googleapis/googleapis-gen/commit/93ace7716e168e3c2cba123bce51a5362a39429c
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiOTNhY2U3NzE2ZTE2OGUzYzJjYmExMjNiY2U1MWE1MzYyYTM5NDI5YyJ9
See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
feat: add default_skew_threshold to TrainingPredictionSkewDetectionConfig in aiplatform v1beta1, v1 model_monitoring.proto
feat: add default_drift_threshold to PredictionDriftDetectionConfig in aiplatform v1beta1, v1 model_monitoring.proto
PiperOrigin-RevId: 455164942
Source-Link: googleapis/googleapis@db1cd5e
Source-Link: https://github.com/googleapis/googleapis-gen/commit/47772101d683e45375539ffcdee797f78fe608cb
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNDc3NzIxMDFkNjgzZTQ1Mzc1NTM5ZmZjZGVlNzk3Zjc4ZmU2MDhjYiJ9
feat: add model_version_id to UploadModelResponse in aiplatform v1 model_service.proto
PiperOrigin-RevId: 455486702
Source-Link: googleapis/googleapis@f2e5538
Source-Link: https://github.com/googleapis/googleapis-gen/commit/15200223b157cb9e76c7923bc387dd6e9c0a83bd
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTUyMDAyMjNiMTU3Y2I5ZTc2Yzc5MjNiYzM4N2RkNmU5YzBhODNiZCJ9
feat: add model_monitoring_config to BatchPredictionJob in aiplatform v1beta1 batch_prediction_job.proto
PiperOrigin-RevId: 456339444
Source-Link: googleapis/googleapis@3b9c451
Source-Link: https://github.com/googleapis/googleapis-gen/commit/95a293beab5db62d3c734b083550e0ab446a9ad3
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiOTVhMjkzYmVhYjVkYjYyZDNjNzM0YjA4MzU1MGUwYWI0NDZhOWFkMyJ9
feat: add BatchImportModelEvaluationSlices API in aiplatform v1beta1 model_service.proto
PiperOrigin-RevId: 456559576
Source-Link: googleapis/googleapis@f5cd509
Source-Link: https://github.com/googleapis/googleapis-gen/commit/b3549d7a62116d5dbf49e22fb3d4c759d5da8313
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjM1NDlkN2E2MjExNmQ1ZGJmNDllMjJmYjNkNGM3NTlkNWRhODMxMyJ9
feat: add model_version_id to BatchPredictionJob in aiplatform v1 batch_prediction_job.proto
feat: add model_version_id to DeployedModel in aiplatform v1 endpoint.proto
feat: add model_version_id to PredictResponse in aiplatform v1 prediction_service.proto
PiperOrigin-RevId: 456571762
Source-Link: googleapis/googleapis@19bd392
Source-Link: https://github.com/googleapis/googleapis-gen/commit/d4dfc802a88fe77bc04191f4b1d9b59481e692fd
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDRkZmM4MDJhODhmZTc3YmMwNDE5MWY0YjFkOWI1OTQ4MWU2OTJmZCJ9
See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md
feat: add BatchImportModelEvaluationSlices API in aiplatform v1 model_service.proto
PiperOrigin-RevId: 456604066
Source-Link: googleapis/googleapis@d57f437
Source-Link: https://github.com/googleapis/googleapis-gen/commit/d9f3b867c60688ee2ed5dbf572ad88913d1fce45
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDlmM2I4NjdjNjA2ODhlZTJlZDVkYmY1NzJhZDg4OTEzZDFmY2U0NSJ9
feat: add ListSavedQueries rpc to aiplatform v1beta1 dataset_service.proto
feat: add saved_query.proto to aiplatform v1beta1
feat: add saved_query_id to InputDataConfig in aiplatform v1beta1 training_pipeline.proto
PiperOrigin-RevId: 456872207
Source-Link: googleapis/googleapis@aeb384a
Source-Link: https://github.com/googleapis/googleapis-gen/commit/c5c8894abbfcd1f6b9d38d9e9fd70a18f15cf0cc
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYzVjODg5NGFiYmZjZDFmNmI5ZDM4ZDllOWZkNzBhMThmMTVjZjBjYyJ9
feat: add ListSavedQueries rpc to aiplatform v1 dataset_service.proto
feat: add saved_query.proto to aiplatform v1
feat: add saved_query_id to InputDataConfig in aiplatform v1 training_pipeline.proto
PiperOrigin-RevId: 456872211
Source-Link: googleapis/googleapis@88f3973
Source-Link: https://github.com/googleapis/googleapis-gen/commit/daf8d1cb20fadabdd8914581da1ceeef048a34f7
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZGFmOGQxY2IyMGZhZGFiZGQ4OTE0NTgxZGExY2VlZWYwNDhhMzRmNyJ9
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Benjamin E. Coe <[email protected]>
Co-authored-by: Yu-Han Liu <[email protected]>

protos/google/cloud/aiplatform/v1/batch_prediction_job.proto

@@ -159,10 +159,16 @@ message BatchPredictionJob {
   // Starting this job has no impact on any existing deployments of the Model
   // and their resources.
   // Exactly one of model and unmanaged_container_model must be set.
+  //
+  // The model resource name may contain version id or version alias to specify
+  // the version, if no version is specified, the default version will be used.
   string model = 3 [(google.api.resource_reference) = {
                       type: "aiplatform.googleapis.com/Model"
                     }];
 
+  // Output only. The version ID of the Model that produces the predictions via this job.
+  string model_version_id = 30 [(google.api.field_behavior) = OUTPUT_ONLY];
+
   // Contains model information necessary to perform batch prediction without
   // requiring uploading to model registry.
   // Exactly one of model and unmanaged_container_model must be set.

protos/google/cloud/aiplatform/v1/completion_stats.proto

@@ -41,4 +41,8 @@ message CompletionStats {
   // Set to -1 if the number is unknown (for example, the operation failed
   // before the total entity number could be collected).
   int64 incomplete_count = 3 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. The number of the successful forecast points that are generated by the
+  // forecasting model. This is ONLY used by the forecasting batch prediction.
+  int64 successful_forecast_point_count = 5 [(google.api.field_behavior) = OUTPUT_ONLY];
 }

protos/google/cloud/aiplatform/v1/dataset_service.proto

@@ -25,6 +25,7 @@ import "google/cloud/aiplatform/v1/annotation_spec.proto";
 import "google/cloud/aiplatform/v1/data_item.proto";
 import "google/cloud/aiplatform/v1/dataset.proto";
 import "google/cloud/aiplatform/v1/operation.proto";
+import "google/cloud/aiplatform/v1/saved_query.proto";
 import "google/longrunning/operations.proto";
 import "google/protobuf/field_mask.proto";
 
@@ -126,6 +127,14 @@ service DatasetService {
     option (google.api.method_signature) = "parent";
   }
 
+  // Lists SavedQueries in a Dataset.
+  rpc ListSavedQueries(ListSavedQueriesRequest) returns (ListSavedQueriesResponse) {
+    option (google.api.http) = {
+      get: "/v1/{parent=projects/*/locations/*/datasets/*}/savedQueries"
+    };
+    option (google.api.method_signature) = "parent";
+  }
+
   // Gets an AnnotationSpec.
   rpc GetAnnotationSpec(GetAnnotationSpecRequest) returns (AnnotationSpec) {
     option (google.api.http) = {
@@ -357,6 +366,44 @@ message ListDataItemsResponse {
   string next_page_token = 2;
 }
 
+// Request message for [DatasetService.ListSavedQueries][google.cloud.aiplatform.v1.DatasetService.ListSavedQueries].
+message ListSavedQueriesRequest {
+  // Required. The resource name of the Dataset to list SavedQueries from.
+  // Format:
+  // `projects/{project}/locations/{location}/datasets/{dataset}`
+  string parent = 1 [
+    (google.api.field_behavior) = REQUIRED,
+    (google.api.resource_reference) = {
+      type: "aiplatform.googleapis.com/Dataset"
+    }
+  ];
+
+  // The standard list filter.
+  string filter = 2;
+
+  // The standard list page size.
+  int32 page_size = 3;
+
+  // The standard list page token.
+  string page_token = 4;
+
+  // Mask specifying which fields to read.
+  google.protobuf.FieldMask read_mask = 5;
+
+  // A comma-separated list of fields to order by, sorted in ascending order.
+  // Use "desc" after a field name for descending.
+  string order_by = 6;
+}
+
+// Response message for [DatasetService.ListSavedQueries][google.cloud.aiplatform.v1.DatasetService.ListSavedQueries].
+message ListSavedQueriesResponse {
+  // A list of SavedQueries that match the specified filter in the request.
+  repeated SavedQuery saved_queries = 1;
+
+  // The standard List next-page token.
+  string next_page_token = 2;
+}
+
 // Request message for [DatasetService.GetAnnotationSpec][google.cloud.aiplatform.v1.DatasetService.GetAnnotationSpec].
 message GetAnnotationSpecRequest {
   // Required. The name of the AnnotationSpec resource.

protos/google/cloud/aiplatform/v1/endpoint.proto

@@ -156,13 +156,19 @@ message DeployedModel {
 
   // Required. The resource name of the Model that this is the deployment of. Note that
   // the Model may be in a different location than the DeployedModel's Endpoint.
+  //
+  // The resource name may contain version id or version alias to specify the
+  // version, if no version is specified, the default version will be deployed.
   string model = 2 [
     (google.api.field_behavior) = REQUIRED,
     (google.api.resource_reference) = {
       type: "aiplatform.googleapis.com/Model"
     }
   ];
 
+  // Output only. The version ID of the model that is deployed.
+  string model_version_id = 18 [(google.api.field_behavior) = OUTPUT_ONLY];
+
   // The display name of the DeployedModel. If not provided upon creation,
   // the Model's display_name is used.
   string display_name = 3;

protos/google/cloud/aiplatform/v1/explanation.proto

@@ -46,6 +46,12 @@ message Explanation {
   // the attributions are stored by [Attribution.output_index][google.cloud.aiplatform.v1.Attribution.output_index] in the same
   // order as they appear in the output_indices.
   repeated Attribution attributions = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. List of the nearest neighbors for example-based explanations.
+  //
+  // For models deployed with the examples explanations feature enabled, the
+  // attributions field is empty and instead the neighbors field is populated.
+  repeated Neighbor neighbors = 2 [(google.api.field_behavior) = OUTPUT_ONLY];
 }
 
 // Aggregated explanation metrics for a Model over a set of instances.
@@ -162,6 +168,15 @@ message Attribution {
   string output_name = 7 [(google.api.field_behavior) = OUTPUT_ONLY];
 }
 
+// Neighbors for example-based explanations.
+message Neighbor {
+  // Output only. The neighbor id.
+  string neighbor_id = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. The neighbor distance.
+  double neighbor_distance = 2 [(google.api.field_behavior) = OUTPUT_ONLY];
+}
+
 // Specification of Model explanation.
 message ExplanationSpec {
   // Required. Parameters that configure explaining of the Model's predictions.
@@ -375,6 +390,9 @@ message ExplanationSpecOverride {
 
   // The metadata to be overridden. If not specified, no metadata is overridden.
   ExplanationMetadataOverride metadata = 2;
+
+  // The example-based explanations parameter overrides.
+  ExamplesOverride examples_override = 3;
 }
 
 // The [ExplanationMetadata][google.cloud.aiplatform.v1.ExplanationMetadata] entries that can be overridden at
@@ -399,3 +417,45 @@ message ExplanationMetadataOverride {
   // overridden.
   map<string, InputMetadataOverride> inputs = 1 [(google.api.field_behavior) = REQUIRED];
 }
+
+// Overrides for example-based explanations.
+message ExamplesOverride {
+  // Data format enum.
+  enum DataFormat {
+    // Unspecified format. Must not be used.
+    DATA_FORMAT_UNSPECIFIED = 0;
+
+    // Provided data is a set of model inputs.
+    INSTANCES = 1;
+
+    // Provided data is a set of embeddings.
+    EMBEDDINGS = 2;
+  }
+
+  // The number of neighbors to return.
+  int32 neighbor_count = 1;
+
+  // The number of neighbors to return that have the same crowding tag.
+  int32 crowding_count = 2;
+
+  // Restrict the resulting nearest neighbors to respect these constraints.
+  repeated ExamplesRestrictionsNamespace restrictions = 3;
+
+  // If true, return the embeddings instead of neighbors.
+  bool return_embeddings = 4;
+
+  // The format of the data being provided with each call.
+  DataFormat data_format = 5;
+}
+
+// Restrictions namespace for example-based explanations overrides.
+message ExamplesRestrictionsNamespace {
+  // The namespace name.
+  string namespace_name = 1;
+
+  // The list of allowed tags.
+  repeated string allow = 2;
+
+  // The list of deny tags.
+  repeated string deny = 3;
+}

protos/google/cloud/aiplatform/v1/featurestore_online_service.proto

@@ -171,7 +171,6 @@ message StreamingReadFeatureValuesRequest {
 }
 
 // Value for a feature.
-// (-- NEXT ID: 15 --)
 message FeatureValue {
   // Metadata of feature value.
   message Metadata {

protos/google/cloud/aiplatform/v1/featurestore_service.proto

@@ -418,6 +418,7 @@ message UpdateFeaturestoreRequest {
   //
   //   * `labels`
   //   * `online_serving_config.fixed_node_count`
+  //   * `online_serving_config.scaling`
   google.protobuf.FieldMask update_mask = 2;
 }
 

protos/google/cloud/aiplatform/v1/model.proto

@@ -103,6 +103,32 @@ message Model {
   // The resource name of the Model.
   string name = 1;
 
+  // Output only. Immutable. The version ID of the model.
+  // A new version is committed when a new model version is uploaded or
+  // trained under an existing model id. It is an auto-incrementing decimal
+  // number in string representation.
+  string version_id = 28 [
+    (google.api.field_behavior) = IMMUTABLE,
+    (google.api.field_behavior) = OUTPUT_ONLY
+  ];
+
+  // User provided version aliases so that a model version can be referenced via
+  // alias (i.e.
+  // projects/{project}/locations/{location}/models/{model_id}@{version_alias}
+  // instead of auto-generated version id (i.e.
+  // projects/{project}/locations/{location}/models/{model_id}@{version_id}).
+  // The format is [a-z][a-zA-Z0-9-]{0,126}[a-z0-9] to distinguish from
+  // version_id. A default version alias will be created for the first version
+  // of the model, and there must be exactly one default version alias for a
+  // model.
+  repeated string version_aliases = 29;
+
+  // Output only. Timestamp when this version was created.
+  google.protobuf.Timestamp version_create_time = 31 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. Timestamp when this version was most recently updated.
+  google.protobuf.Timestamp version_update_time = 32 [(google.api.field_behavior) = OUTPUT_ONLY];
+
   // Required. The display name of the Model.
   // The name can be up to 128 characters long and can be consist of any UTF-8
   // characters.
@@ -111,6 +137,9 @@ message Model {
   // The description of the Model.
   string description = 3;
 
+  // The description of this version.
+  string version_description = 30;
+
   // The schemata that describe formats of the Model's predictions and
   // explanations as given and returned via
   // [PredictionService.Predict][google.cloud.aiplatform.v1.PredictionService.Predict] and [PredictionService.Explain][google.cloud.aiplatform.v1.PredictionService.Explain].

protos/google/cloud/aiplatform/v1/model_deployment_monitoring_job.proto

@@ -266,6 +266,16 @@ message ModelDeploymentMonitoringScheduleConfig {
   // Required. The model monitoring job scheduling interval. It will be rounded up to next
   // full hour. This defines how often the monitoring jobs are triggered.
   google.protobuf.Duration monitor_interval = 1 [(google.api.field_behavior) = REQUIRED];
+
+  // The time window of the prediction data being included in each prediction
+  // dataset. This window specifies how long the data should be collected from
+  // historical model results for each run. If not set,
+  // [ModelDeploymentMonitoringScheduleConfig.monitor_interval][google.cloud.aiplatform.v1.ModelDeploymentMonitoringScheduleConfig.monitor_interval] will be used.
+  // e.g. If currently the cutoff time is 2022-01-08 14:30:00 and the
+  // monitor_window is set to be 3600, then data from 2022-01-08 13:30:00
+  // to 2022-01-08 14:30:00 will be retrieved and aggregated to calculate the
+  // monitoring statistics.
+  google.protobuf.Duration monitor_window = 2;
 }
 
 // Statistics and anomalies generated by Model Monitoring.

protos/google/cloud/aiplatform/v1/model_monitoring.proto

@@ -82,6 +82,11 @@ message ModelMonitoringObjectiveConfig {
     // against attribution score distance between the training and prediction
     // feature.
     map<string, ThresholdConfig> attribution_score_skew_thresholds = 2;
+
+    // Skew anomaly detection threshold used by all features.
+    // When the per-feature thresholds are not set, this field can be used to
+    // specify a threshold for all features.
+    ThresholdConfig default_skew_threshold = 6;
   }
 
   // The config for Prediction data drift detection.
@@ -95,6 +100,11 @@ message ModelMonitoringObjectiveConfig {
     // Key is the feature name and value is the threshold. The threshold here is
     // against attribution score distance between different time windows.
     map<string, ThresholdConfig> attribution_score_drift_thresholds = 2;
+
+    // Drift anomaly detection threshold used by all features.
+    // When the per-feature thresholds are not set, this field can be used to
+    // specify a threshold for all features.
+    ThresholdConfig default_drift_threshold = 5;
   }
 
   // The config for integrating with Vertex Explainable AI. Only applicable if