Skip to content

Commit

Permalink
feat: add encryption_spec to index.proto and index_endpoint.proto (#4596
Browse files Browse the repository at this point in the history
)

* feat: add UpdatePersistentResourceRequest and add resource_pool_images and head_node_resource_pool_id to RaySpec

PiperOrigin-RevId: 561051086

Source-Link: googleapis/googleapis@da9f0be

Source-Link: googleapis/googleapis-gen@f7b6581
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImY3YjY1ODFmZTM5ZTczZWRhNDA3MTkxNGNlODU3NjhmMjlhMjUxMTEifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* feat: add encryption_spec to index.proto and index_endpoint.proto
feat: add TrialContext to study.proto
feat: add contexts to SuggestTrialsRequest in vizier_service.proto

PiperOrigin-RevId: 563249077

Source-Link: googleapis/googleapis@0e828f8

Source-Link: googleapis/googleapis-gen@e3d5760
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImUzZDU3NjA0ZTk1OWQ0ZDY1YzFjZTBjNjE5NmE2YjY2OTMxNjJjOTYifQ==

* feat: add encryption_spec to index.proto and index_endpoint.proto
feat: add TrialContext to study.proto
feat: add contexts to SuggestTrialsRequest in vizier_service.proto

PiperOrigin-RevId: 563257278

Source-Link: googleapis/googleapis@7534629

Source-Link: googleapis/googleapis-gen@63658af
Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjYzNjU4YWY2YzE2NjRhY2ZhYmRiNGY1NTdjNzQ5NzEwMTg5NTQ5MmEifQ==

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Denis DelGrosso <[email protected]>
  • Loading branch information
3 people authored Sep 11, 2023
1 parent 2ba7b48 commit 32d823a
Show file tree
Hide file tree
Showing 42 changed files with 3,946 additions and 4 deletions.
1 change: 1 addition & 0 deletions packages/google-cloud-aiplatform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -482,6 +482,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/google-cloud-node/
| Persistent_resource_service.delete_persistent_resource | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.delete_persistent_resource.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.delete_persistent_resource.js,packages/google-cloud-aiplatform/samples/README.md) |
| Persistent_resource_service.get_persistent_resource | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.get_persistent_resource.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.get_persistent_resource.js,packages/google-cloud-aiplatform/samples/README.md) |
| Persistent_resource_service.list_persistent_resources | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.list_persistent_resources.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.list_persistent_resources.js,packages/google-cloud-aiplatform/samples/README.md) |
| Persistent_resource_service.update_persistent_resource | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.update_persistent_resource.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/persistent_resource_service.update_persistent_resource.js,packages/google-cloud-aiplatform/samples/README.md) |
| Pipeline_service.cancel_pipeline_job | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/pipeline_service.cancel_pipeline_job.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/pipeline_service.cancel_pipeline_job.js,packages/google-cloud-aiplatform/samples/README.md) |
| Pipeline_service.cancel_training_pipeline | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/pipeline_service.cancel_training_pipeline.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/pipeline_service.cancel_training_pipeline.js,packages/google-cloud-aiplatform/samples/README.md) |
| Pipeline_service.create_pipeline_job | [source code](https://github.com/googleapis/google-cloud-node/blob/main/packages/google-cloud-aiplatform/samples/generated/v1beta1/pipeline_service.create_pipeline_job.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/google-cloud-node&page=editor&open_in_editor=packages/google-cloud-aiplatform/samples/generated/v1beta1/pipeline_service.create_pipeline_job.js,packages/google-cloud-aiplatform/samples/README.md) |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,7 @@ message ListEndpointsRequest {
// * A key including a space must be quoted. `labels."a key"`.
//
// Some examples:
//
// * `endpoint=1`
// * `displayName="myDisplayName"`
// * `labels.myKey="myValue"`
Expand All @@ -243,6 +244,7 @@ message ListEndpointsRequest {
// A comma-separated list of fields to order by, sorted in ascending order.
// Use "desc" after a field name for descending.
// Supported fields:
//
// * `display_name`
// * `create_time`
// * `update_time`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package google.cloud.aiplatform.v1;
import "google/api/field_behavior.proto";
import "google/api/resource.proto";
import "google/cloud/aiplatform/v1/deployed_index_ref.proto";
import "google/cloud/aiplatform/v1/encryption_spec.proto";
import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";

Expand Down Expand Up @@ -120,6 +121,10 @@ message Index {
// BATCH_UPDATE will be used by default.
IndexUpdateMethod index_update_method = 16
[(google.api.field_behavior) = IMMUTABLE];

// Immutable. Customer-managed encryption key spec for an Index. If set, this
// Index and all sub-resources of this Index will be secured by this key.
EncryptionSpec encryption_spec = 17 [(google.api.field_behavior) = IMMUTABLE];
}

// A datapoint of Index.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package google.cloud.aiplatform.v1;

import "google/api/field_behavior.proto";
import "google/api/resource.proto";
import "google/cloud/aiplatform/v1/encryption_spec.proto";
import "google/cloud/aiplatform/v1/machine_resources.proto";
import "google/cloud/aiplatform/v1/service_networking.proto";
import "google/protobuf/timestamp.proto";
Expand Down Expand Up @@ -122,6 +123,11 @@ message IndexEndpoint {
// index endpoint.
string public_endpoint_domain_name = 14
[(google.api.field_behavior) = OUTPUT_ONLY];

// Immutable. Customer-managed encryption key spec for an IndexEndpoint. If
// set, this IndexEndpoint and all sub-resources of this IndexEndpoint will be
// secured by this key.
EncryptionSpec encryption_spec = 15 [(google.api.field_behavior) = IMMUTABLE];
}

// A deployment of an Index. IndexEndpoints contain one or more DeployedIndexes.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,24 @@ message Trial {
[(google.api.field_behavior) = OUTPUT_ONLY];
}

// Next ID: 3
message TrialContext {
// A human-readable field which can store a description of this context.
// This will become part of the resulting Trial's description field.
string description = 1;

// If/when a Trial is generated or selected from this Context,
// its Parameters will match any parameters specified here.
// (I.e. if this context specifies parameter name:'a' int_value:3,
// then a resulting Trial will have int_value:3 for its parameter named
// 'a'.) Note that we first attempt to match existing REQUESTED Trials with
// contexts, and if there are no matches, we generate suggestions in the
// subspace defined by the parameters specified here.
// NOTE: a Context without any Parameters matches the entire feasible search
// space.
repeated Trial.Parameter parameters = 2;
}

// Represents specification of a Study.
message StudySpec {
// Represents a metric to optimize.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,44 @@ message SuggestTrialsRequest {
// the service will return the identical suggested Trial if the Trial is
// pending, and provide a new Trial if the last suggested Trial was completed.
string client_id = 3 [(google.api.field_behavior) = REQUIRED];

// Optional. This allows you to specify the "context" for a Trial; a context
// is a slice (a subspace) of the search space.
//
// Typical uses for contexts:
// 1) You are using Vizier to tune a server for best performance, but there's
// a strong weekly cycle. The context specifies the day-of-week.
// This allows Tuesday to generalize from Wednesday without assuming that
// everything is identical.
// 2) Imagine you're optimizing some medical treatment for people.
// As they walk in the door, you know certain facts about them
// (e.g. sex, weight, height, blood-pressure). Put that information in the
// context, and Vizier will adapt its suggestions to the patient.
// 3) You want to do a fair A/B test efficiently. Specify the "A" and "B"
// conditions as contexts, and Vizier will generalize between "A" and "B"
// conditions. If they are similar, this will allow Vizier to converge
// to the optimum faster than if "A" and "B" were separate Studies.
// NOTE: You can also enter contexts as REQUESTED Trials, e.g. via the
// CreateTrial() RPC; that's the asynchronous option where you don't need a
// close association between contexts and suggestions.
//
// NOTE: All the Parameters you set in a context MUST be defined in the
// Study.
// NOTE: You must supply 0 or $suggestion_count contexts.
// If you don't supply any contexts, Vizier will make suggestions
// from the full search space specified in the StudySpec; if you supply
// a full set of context, each suggestion will match the corresponding
// context.
// NOTE: A Context with no features set matches anything, and allows
// suggestions from the full search space.
// NOTE: Contexts MUST lie within the search space specified in the
// StudySpec. It's an error if they don't.
// NOTE: Contexts preferentially match ACTIVE then REQUESTED trials before
// new suggestions are generated.
// NOTE: Generation of suggestions involves a match between a Context and
// (optionally) a REQUESTED trial; if that match is not fully specified, a
// suggestion will be geneated in the merged subspace.
repeated TrialContext contexts = 4 [(google.api.field_behavior) = OPTIONAL];
}

// Response message for
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package google.cloud.aiplatform.v1beta1;
import "google/api/field_behavior.proto";
import "google/api/resource.proto";
import "google/cloud/aiplatform/v1beta1/deployed_index_ref.proto";
import "google/cloud/aiplatform/v1beta1/encryption_spec.proto";
import "google/protobuf/struct.proto";
import "google/protobuf/timestamp.proto";

Expand Down Expand Up @@ -120,6 +121,10 @@ message Index {
// BATCH_UPDATE will be used by default.
IndexUpdateMethod index_update_method = 16
[(google.api.field_behavior) = IMMUTABLE];

// Immutable. Customer-managed encryption key spec for an Index. If set, this
// Index and all sub-resources of this Index will be secured by this key.
EncryptionSpec encryption_spec = 17 [(google.api.field_behavior) = IMMUTABLE];
}

// A datapoint of Index.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package google.cloud.aiplatform.v1beta1;

import "google/api/field_behavior.proto";
import "google/api/resource.proto";
import "google/cloud/aiplatform/v1beta1/encryption_spec.proto";
import "google/cloud/aiplatform/v1beta1/machine_resources.proto";
import "google/cloud/aiplatform/v1beta1/service_networking.proto";
import "google/protobuf/timestamp.proto";
Expand Down Expand Up @@ -122,6 +123,11 @@ message IndexEndpoint {
// index endpoint.
string public_endpoint_domain_name = 14
[(google.api.field_behavior) = OUTPUT_ONLY];

// Immutable. Customer-managed encryption key spec for an IndexEndpoint. If
// set, this IndexEndpoint and all sub-resources of this IndexEndpoint will be
// secured by this key.
EncryptionSpec encryption_spec = 15 [(google.api.field_behavior) = IMMUTABLE];
}

// A deployment of an Index. IndexEndpoints contain one or more DeployedIndexes.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,25 @@ message RaySpec {
// you need all the resource pools to have the same Ray image, Otherwise, use
// the {@code resource_pool_images} field.
string image_uri = 1 [(google.api.field_behavior) = OPTIONAL];

// Optional. Required if image_uri is not set. A map of resource_pool_id to
// prebuild Ray image if user need to use different images for different
// head/worker pools. This map needs to cover all the resource pool ids.
// Example:
// {
// "ray_head_node_pool": "head image"
// "ray_worker_node_pool1": "worker image"
// "ray_worker_node_pool2": "another worker image"
// }
map<string, string> resource_pool_images = 6
[(google.api.field_behavior) = OPTIONAL];

// Optional. This will be used to indicate which resource pool will serve as
// the Ray head node(the first node within that pool). Will use the machine
// from the first workerpool as the head node by default if this field is not
// set.
string head_node_resource_pool_id = 7
[(google.api.field_behavior) = OPTIONAL];
}

// Persistent Cluster runtime information as output
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import "google/cloud/aiplatform/v1beta1/operation.proto";
import "google/cloud/aiplatform/v1beta1/persistent_resource.proto";
import "google/longrunning/operations.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/field_mask.proto";

option csharp_namespace = "Google.Cloud.AIPlatform.V1Beta1";
option go_package = "cloud.google.com/go/aiplatform/apiv1beta1/aiplatformpb;aiplatformpb";
Expand Down Expand Up @@ -84,6 +85,20 @@ service PersistentResourceService {
metadata_type: "DeleteOperationMetadata"
};
}

// Updates a PersistentResource.
rpc UpdatePersistentResource(UpdatePersistentResourceRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
patch: "/v1beta1/{persistent_resource.name=projects/*/locations/*/persistentResources/*}"
body: "persistent_resource"
};
option (google.api.method_signature) = "persistent_resource,update_mask";
option (google.longrunning.operation_info) = {
response_type: "PersistentResource"
metadata_type: "UpdatePersistentResourceOperationMetadata"
};
}
}

// Request message for
Expand Down Expand Up @@ -116,6 +131,12 @@ message CreatePersistentResourceOperationMetadata {
GenericOperationMetadata generic_metadata = 1;
}

// Details of operations that perform update PersistentResource.
message UpdatePersistentResourceOperationMetadata {
// Operation metadata for PersistentResource.
GenericOperationMetadata generic_metadata = 1;
}

// Request message for
// [PersistentResourceService.GetPersistentResource][google.cloud.aiplatform.v1beta1.PersistentResourceService.GetPersistentResource].
message GetPersistentResourceRequest {
Expand Down Expand Up @@ -176,3 +197,19 @@ message DeletePersistentResourceRequest {
}
];
}

// Request message for UpdatePersistentResource method.
message UpdatePersistentResourceRequest {
// Required. The PersistentResource to update.
//
// The PersistentResource's `name` field is used to identify the
// PersistentResource to update. Format:
// `projects/{project}/locations/{location}/persistentResources/{persistent_resource}`
PersistentResource persistent_resource = 1
[(google.api.field_behavior) = REQUIRED];

// Required. Specify the fields to be overwritten in the PersistentResource by
// the update method.
google.protobuf.FieldMask update_mask = 2
[(google.api.field_behavior) = REQUIRED];
}
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,24 @@ message Trial {
[(google.api.field_behavior) = OUTPUT_ONLY];
}

// Next ID: 3
message TrialContext {
// A human-readable field which can store a description of this context.
// This will become part of the resulting Trial's description field.
string description = 1;

// If/when a Trial is generated or selected from this Context,
// its Parameters will match any parameters specified here.
// (I.e. if this context specifies parameter name:'a' int_value:3,
// then a resulting Trial will have int_value:3 for its parameter named
// 'a'.) Note that we first attempt to match existing REQUESTED Trials with
// contexts, and if there are no matches, we generate suggestions in the
// subspace defined by the parameters specified here.
// NOTE: a Context without any Parameters matches the entire feasible search
// space.
repeated Trial.Parameter parameters = 2;
}

// Represents specification of a Study.
message StudySpec {
// Represents a metric to optimize.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -306,6 +306,44 @@ message SuggestTrialsRequest {
// the service will return the identical suggested Trial if the Trial is
// pending, and provide a new Trial if the last suggested Trial was completed.
string client_id = 3 [(google.api.field_behavior) = REQUIRED];

// Optional. This allows you to specify the "context" for a Trial; a context
// is a slice (a subspace) of the search space.
//
// Typical uses for contexts:
// 1) You are using Vizier to tune a server for best performance, but there's
// a strong weekly cycle. The context specifies the day-of-week.
// This allows Tuesday to generalize from Wednesday without assuming that
// everything is identical.
// 2) Imagine you're optimizing some medical treatment for people.
// As they walk in the door, you know certain facts about them
// (e.g. sex, weight, height, blood-pressure). Put that information in the
// context, and Vizier will adapt its suggestions to the patient.
// 3) You want to do a fair A/B test efficiently. Specify the "A" and "B"
// conditions as contexts, and Vizier will generalize between "A" and "B"
// conditions. If they are similar, this will allow Vizier to converge
// to the optimum faster than if "A" and "B" were separate Studies.
// NOTE: You can also enter contexts as REQUESTED Trials, e.g. via the
// CreateTrial() RPC; that's the asynchronous option where you don't need a
// close association between contexts and suggestions.
//
// NOTE: All the Parameters you set in a context MUST be defined in the
// Study.
// NOTE: You must supply 0 or $suggestion_count contexts.
// If you don't supply any contexts, Vizier will make suggestions
// from the full search space specified in the StudySpec; if you supply
// a full set of context, each suggestion will match the corresponding
// context.
// NOTE: A Context with no features set matches anything, and allows
// suggestions from the full search space.
// NOTE: Contexts MUST lie within the search space specified in the
// StudySpec. It's an error if they don't.
// NOTE: Contexts preferentially match ACTIVE then REQUESTED trials before
// new suggestions are generated.
// NOTE: Generation of suggestions involves a match between a Context and
// (optionally) a REQUESTED trial; if that match is not fully specified, a
// suggestion will be geneated in the merged subspace.
repeated TrialContext contexts = 4 [(google.api.field_behavior) = OPTIONAL];
}

// Response message for
Expand Down
Loading

0 comments on commit 32d823a

Please sign in to comment.