Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dbt fails if destination schema contains a LOCAL TEMPORARY in Snowflake #1869

Closed
danieldicker opened this issue Oct 28, 2019 · 2 comments · Fixed by #1872
Closed

dbt fails if destination schema contains a LOCAL TEMPORARY in Snowflake #1869

danieldicker opened this issue Oct 28, 2019 · 2 comments · Fixed by #1872
Labels
bug Something isn't working

Comments

@danieldicker
Copy link

Describe the bug

dbt fails to compile or run when the destination schema contains a LOCAL TEMPORARY table

Steps To Reproduce

Run dbt compile while someone else is running dbt run

Expected behavior

No error occurs.

Screenshots and log output

Running with dbt=0.14.3

* Deprecation Warning: The adapter function `adapter.already_exists` is deprecated and will be removed in
 a future release of dbt. Please use `adapter.get_relation` instead.
 Documentation for get_relation can be found here:
 https://docs.getdbt.com/docs/adapter

Unhandled error while executing model.snowflake.inventory
connection never acquired for thread (15906, 123145576628224), have []
Encountered an error:
Runtime Error
  Runtime Error in model inventory (models/inventory.sql)
    Invalid arguments passed to "SnowflakeRelation" instance: type.'LOCAL TEMPORARY' is not one of ['table', 'view', 'cte', 'materializedview', 'externaltable', None]

System information

Database: Snowflake

The output of dbt --version: 0.14.3

The operating system you're using: OS X

The output of python --version: 3.7

Additional context

While we don't have any tables explicitly specified as LOCAL TEMPORARY, we believe this is happening because other processes are running dbt and LOCAL TEMPORARY tables are being created in the process of merging or replacing the tables. As such, the specific tables that throw the error are different each time we encounter this error. When we look in the information schema after getting this error, the LOCAL TEMPORARY tables reported in the error text are shown as regular tables.

Similar to the fix for #1431, we created a fork and added a LOCAL TEMPORARY relation type which seems to have fixed the issue. We wanted to report the issue here as well.

@drewbanin
Copy link
Contributor

Thanks for making this report @danieldicker! Adding LOCAL TEMPORARY to the list of known relations is a good idea, but I think that will put us in a perpetual cat-and-mouse position whenever new relation types pop up in Snowflake. Instead, check out how we're approaching this in #1872. Here, we'll record the table as being "external" as in "external to dbt" -- that should be appropriate because dbt doesn't really care about the existence of tables generated outside the scope of a dbt project - we just need to avoid the validation error you ran into here.

@danieldicker
Copy link
Author

That makes sense to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants