Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Improved metadata artifact and execution creation using python / SDK #1430

Merged
merged 58 commits into from
Jun 28, 2022

Conversation

SinaChavoshi
Copy link
Contributor

@SinaChavoshi SinaChavoshi commented Jun 14, 2022

This is a proposal to add support for schema types via the python sdk.

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. api: vertex-ai Issues related to the googleapis/python-aiplatform API. labels Jun 14, 2022
@SinaChavoshi SinaChavoshi changed the title Improved metadata schema Authoring using python / SDK feat: Improved metadata schema Authoring using python / SDK Jun 14, 2022
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: xl Pull request size is extra large. labels Jun 14, 2022
@SinaChavoshi SinaChavoshi force-pushed the minimum_schema_set_from_ga branch from d40e612 to 49e34cc Compare June 14, 2022 18:14
@product-auto-label product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Jun 14, 2022
google/cloud/aiplatform/metadata/artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/types/base.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/types/google_types.py Outdated Show resolved Hide resolved
@SinaChavoshi SinaChavoshi requested a review from a team as a code owner June 16, 2022 08:43
google/cloud/aiplatform/metadata/artifact.py Show resolved Hide resolved
google/cloud/aiplatform/metadata/artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/system_schema.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/system_schema.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/system_schema.py Outdated Show resolved Hide resolved
tests/unit/aiplatform/test_metadata_schema_types.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/metadata.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_execution.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_artifact.py Outdated Show resolved Hide resolved
google/cloud/aiplatform/metadata/schema/base_execution.py Outdated Show resolved Hide resolved
@SinaChavoshi SinaChavoshi force-pushed the minimum_schema_set_from_ga branch from 41a2cf6 to bee8e93 Compare June 23, 2022 16:58
@SinaChavoshi SinaChavoshi merged commit 6c4374f into googleapis:main Jun 28, 2022
@SinaChavoshi SinaChavoshi deleted the minimum_schema_set_from_ga branch June 28, 2022 20:14
sararob pushed a commit to sararob/python-aiplatform that referenced this pull request Jun 29, 2022
…/ SDK (googleapis#1430)

* rebase to master after ga merge

* add support for artifact create

* add unit tests for create from artifact parameters

* update formatting

* fix lint issues

* Add integration tests

* add support for execution types

* correct execution type in create

* add execution support and unit tests

* add support for state in artifacts

* add support for start_execution

* add support for metadata_store_id in start_execution

* lint and docs update based on review feedback

* Add e2e integraton tests and lint update

* Update google/cloud/aiplatform/metadata/artifact.py

Co-authored-by: sasha-gitg <[email protected]>

* remove the duplicate test_experiments_copy.py

* refactor based on code review feedback

* regroup tests to match module names

* fix e2e integration tests

* remove call to     _temp_prefix = tmpvrtxsdk-e2e from E2E test

* Update google/cloud/aiplatform/metadata/schema/base_execution.py

Co-authored-by: sasha-gitg <[email protected]>

* remove artifact and schema referencing the create result to self

* remove kwargs

* fix typing for container spec

* remove resouceName from system types

* metrics should default to None

* change from using resouce_name to resource_id

* fix e2e tests

* change google and system to sub folders of schema

* use create_from_base_execution_schema instead of overloading create

* update api docs

* update docstring formatting

* Update google/cloud/aiplatform/metadata/metadata.py

Co-authored-by: sasha-gitg <[email protected]>

* add return types and move args to constructor

* using forward reference for parameter and return types to resolve circular import error

* change base classes to abstract classes

* Add tests for system.artifact type

* use resouce name instead of id and populate metadata with resourceNanme accoridngly

* remove start_execution from this pr and move to a separate PR

* change all args to keyword args

* always make a copy of metadata instead of pass by reference

* auto generate uri for google types

* fix e2e tests

* switch to using Artifact.create instead of _create

* change typing for state to Optional

* change typing for state to Optional in artifact base file

* change to use the Execution.create instead of the private method

* chagne copy to deepcopy for metadata

Co-authored-by: sasha-gitg <[email protected]>
Co-authored-by: Rosie Zou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants