feat: Support for Model Versioning (#1438)

* Initial changes for first ModelRegistry design proposal * More changes for design doc * ModelRegistry class implementation * Added method docs * Changes from book doc * training_jobs versioning changes * More models.py changes for versioning * More version arg plumbing * Tests, implementation changes, and assorted tweaks to make GAPIC stuff work * Batch predict versioning, variable name cleanup * Reset training_jobs changes to limit scope * Prediction test fixes * Blackend and lint changes * Training jobs versioning support * Blackend and lint changes * Added TODO for async support * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Override for _construct_sdk_resource_from_gapic * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Fixed _construct_sdk_resource_from_gapic for Model class and gave documentation * Removed errant futuremanager init * Start of new versioning system test * Pass model version on upload cls init * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Fully-fleshed system test for model versioning * Improvements based on system testing * Nox fixes * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Nox fixes * Initial commit for vertexpreviews, with compat changes * Use compat type for DeploymentResourcesType * PR Feedback * Test compat fixes * More v1->v1beta1 shifts * Plumbing model changes through more tests * chore: release 1.13.1 Release-As: 1.13.1 * Nox run * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Training jobs test fixes * Reverted v1beta1 changes * Blacken and lint changes * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * PR feedback changes * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Revert "🦉 Updates from OwlBot post-processor" This reverts commit 0bff90b. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Revert "🦉 Updates from OwlBot post-processor" This reverts commit c57f305. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Revert "🦉 Updates from OwlBot post-processor" This reverts commit ee5ff17. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * fix: Prevent owlbot from re-adding 3.6 dependencies * Test fixes * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * nox blacken * Test fixes for 3.6 compat * Test fix for 3.6 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Quitting the fight against nox * Test fixes for python 3.7 * System test cleanup * retrigger checks * Update google/cloud/aiplatform/models.py Missing '.' Co-authored-by: sasha-gitg <[email protected]> * Update google/cloud/aiplatform/models.py Another missing '.' Co-authored-by: sasha-gitg <[email protected]> * Update google/cloud/aiplatform/models.py Missing return type Co-authored-by: sasha-gitg <[email protected]> * Update google/cloud/aiplatform/models.py Co-authored-by: sasha-gitg <[email protected]> * PR feedback changes * Pass location, project, creds to Model Registry * Credential fix when getting model from registry * Copyright update * Fixed issue with Model update trying to update a version, rather than the base model * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Revert "fix: Prevent owlbot from re-adding 3.6 dependencies" This reverts commit 4322fe2. * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Nox blacken * Revert "🦉 Updates from OwlBot post-processor" This reverts commit fe5a27b. * Fighting with owlbot * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Sam Goodman <[email protected]> Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: sasha-gitg <[email protected]>
googleapis · Jun 29, 2022 · d890685 · d890685
1 parent 425a32f
commit d890685
Show file tree

Hide file tree

Showing 17 changed files with 2,395 additions and 110 deletions.
diff --git a/google/cloud/aiplatform/base.py b/google/cloud/aiplatform/base.py
@@ -21,6 +21,7 @@
 import functools
 import inspect
 import logging
+import re
 import sys
 import threading
 import time
@@ -454,6 +455,24 @@ def _format_resource_name_method(self) -> str:
     # to use custom resource id validators per resource
     _resource_id_validator: Optional[Callable[[str], None]] = None
 
+    @staticmethod
+    def _revisioned_resource_id_validator(
+        resource_id: str,
+    ) -> None:
+        """Some revisioned resource names can have '@' in them
+        to separate the resource ID from the revision ID.
+        Thus, they need their own resource id validator.
+        See https://google.aip.dev/162
+
+        Args:
+            resource_id(str): A resource ID for a resource type that accepts revision syntax.
+                See https://google.aip.dev/162.
+        Raises:
+            ValueError: If a `resource_id` doesn't conform to appropriate revision syntax.
+        """
+        if not re.compile(r"^[\w-]+@?[\w-]+$").match(resource_id):
+            raise ValueError(f"Resource {resource_id} is not a valid resource ID.")
+
     def __init__(
         self,
         project: Optional[str] = None,

diff --git a/google/cloud/aiplatform/jobs.py b/google/cloud/aiplatform/jobs.py
@@ -392,6 +392,8 @@ def create(
                 Required. A fully-qualified model resource name or model ID.
                 Example: "projects/123/locations/us-central1/models/456" or
                 "456" when project and location are initialized or passed.
+                May optionally contain a version ID or alias in
+                {model_name}@{version} form.
 
                 Or an instance of aiplatform.Model.
             instances_format (str):
@@ -564,6 +566,7 @@ def create(
                 format_resource_name_method=aiplatform.Model._format_resource_name,
                 project=project,
                 location=location,
+                resource_id_validator=super()._revisioned_resource_id_validator,
             )
 
         # Raise error if both or neither source URIs are provided
@@ -713,7 +716,9 @@ def _create(
                 Required. BatchPredictionJob without _gca_resource populated.
             model_or_model_name (Union[str, aiplatform.Model]):
                 Required. Required. A fully-qualified model resource name or
-                an instance of aiplatform.Model.
+                an instance of aiplatform.Model. If a resource name, it may
+                optionally contain a version ID or alias in
+                {model_name}@{version} form.
             gca_batch_prediction_job (gca_bp_job.BatchPredictionJob):
                 Required. a batch prediction job proto for creating a batch prediction job on Vertex AI.
             generate_explanation (bool):
@@ -731,7 +736,6 @@ def _create(
                 provided instances_format or predictions_format are not supported
                 by Vertex AI.
         """
-        # select v1beta1 if explain else use default v1
 
         parent = initializer.global_config.common_location_path(
             project=empty_batch_prediction_job.project,
@@ -741,7 +745,7 @@ def _create(
         model_resource_name = (
             model_or_model_name
             if isinstance(model_or_model_name, str)
-            else model_or_model_name.resource_name
+            else model_or_model_name.versioned_resource_name
         )
 
         gca_batch_prediction_job.model = model_resource_name