Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pg/redshift database verification case-insensitive #1800

Closed
rpammeraal opened this issue Oct 2, 2019 · 3 comments · Fixed by #1936
Closed

Make pg/redshift database verification case-insensitive #1800

rpammeraal opened this issue Oct 2, 2019 · 3 comments · Fixed by #1936
Labels
enhancement New feature or request good_first_issue Straightforward + self-contained changes, good for new contributors! redshift

Comments

@rpammeraal
Copy link

rpammeraal commented Oct 2, 2019

Database names are case sensitive in Postgres (and other databases).
When running DBT I got the following:
ERROR: Cross-db references not allowed in postgres (stg_rammeraal_82136 vs STG_rammeraal_82136)

To avoid this, please apply the following patch:

index af27aef2..f442281e 100644
--- a/plugins/postgres/dbt/adapters/postgres/impl.py
+++ b/plugins/postgres/dbt/adapters/postgres/impl.py
@@ -23,10 +23,8 @@ class PostgresAdapter(SQLAdapter):
     def verify_database(self, database):
         if database.startswith('"'):
             database = database.strip('"')
-        else:
-            database = database.lower()
         expected = self.config.credentials.database
-        if database != expected:
+        if database.lower() != expected.lower():
             raise dbt.exceptions.NotImplementedException(
                 'Cross-db references not allowed in {} ({} vs {})'
                 .format(self.type(), database, expected)```

This may be an issue with other database vendors as well.
@drewbanin
Copy link
Contributor

drewbanin commented Oct 8, 2019

Thanks for the report @rpammeraal! Please feel free to submit a pull request for this change -- it's a very good one :)

We've recently added a CLA to the dbt contribution process. This CLA ensures that contributions to the dbt codebase are "original work that does not violate any third-party license agreement." As such, we're not well-suited to copy/paste code into patches to the dbt-core repository. If you're able to submit a PR for this, I'd be very happy to merge it! If not, we can retitle this issue to "Make Postgres database comparisons case-insensitive" and prioritize it on our own accord. Let us know!

@rpammeraal
Copy link
Author

rpammeraal commented Oct 8, 2019 via email

@drewbanin drewbanin added enhancement New feature or request good_first_issue Straightforward + self-contained changes, good for new contributors! labels Oct 9, 2019
@drewbanin drewbanin changed the title Please apply the following patch to avoid cross-db when database names are case sensitive. Make pg/redshift database verification case-insensitive Oct 9, 2019
@drewbanin
Copy link
Contributor

I don't believe the fix in #1918 fully addresses this issue. While the cross-db error message has indeed been addressed, dbt now fails with an error like:

Compilation Error in model debug2 (models/debug2.sql)
  When searching for a relation, dbt found an approximate match. Instead of guessing
  which relation to use, dbt will move on. Please delete "mydb"."test_schema"."debug2", or rename it to be less ambiguous.
  Searched for: "MyDB"."test_schema"."debug2"
  Found: "mydb"."test_schema"."debug2"

Here, my database is named MyDB on Postgres. We can reproduce this failure mode with:

-- models/my_model.sql
select 1 as id

And a profile configured with:

debug:
  target: dev
  outputs:
    dev:
      type: postgres
      host: localhost
      user: drew
      pass: password
      port: 5433
      dbname: MyDB
      schema: test_schema
      threads: 8

Followed by:

dbt run
dbt run

The first dbt run will succeed while the second dbt run will fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good_first_issue Straightforward + self-contained changes, good for new contributors! redshift
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants