Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

archives: do not create existing schema (#758) #1398

Merged

Conversation

beckjake
Copy link
Contributor

@beckjake beckjake commented Apr 12, 2019

Fixes #758

This is based on #1361

During archive creation, check if a schema exists before creating it.

We should probably cache the results of list_schemas or check_schema_exists.

@cmcarthur cmcarthur requested a review from drewbanin April 18, 2019 15:18
Copy link
Contributor

@drewbanin drewbanin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but one question worth discussing

@@ -84,6 +85,11 @@ def list_schemas(self, database):
include_all=True)
return [ds.dataset_id for ds in all_datasets]

@available
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean that we'll need to do something similar for the Presto and Spark plugins?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This @available exists to make the relation available to bq's macros so it can implement this macro, because on bigquery that functionality is implemented via the connection.

Spark is a SQLAdapter and so it already has the macro available. Though it does look like it's incomplete?

We currently don't support archives on presto, so we don't have to worry about that at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hah, dang. Ok, thanks for the info, we can handle Spark separately from this for sure

@beckjake beckjake force-pushed the feature/archive-blocks-as-regex branch from b00c824 to 416cc72 Compare April 23, 2019 20:49
@beckjake beckjake force-pushed the feature/archive-no-create-existing-schema branch from 7a4dae4 to cc4f285 Compare April 23, 2019 21:00
@beckjake
Copy link
Contributor Author

No changes, just a rebase onto the other PR!

@beckjake beckjake changed the base branch from feature/archive-blocks-as-regex to dev/wilt-chamberlain April 26, 2019 03:42
@beckjake beckjake merged commit 3f18b93 into dev/wilt-chamberlain Apr 26, 2019
@beckjake beckjake deleted the feature/archive-no-create-existing-schema branch April 26, 2019 03:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

archive: don't create schema if it already exists
2 participants