Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Postgresql dependency search is stalled on large databases #1193

Closed
patrickgoss opened this issue Dec 19, 2018 · 0 comments
Closed

Postgresql dependency search is stalled on large databases #1193

patrickgoss opened this issue Dec 19, 2018 · 0 comments
Labels
enhancement New feature or request redshift

Comments

@patrickgoss
Copy link

Please make sure to fill out either the issue template or the feature template and delete the other one!

Issue

Issue description

In general terms, please describe the issue. What command did you run?
dbt run

Results

What happened? What did you expect to happen?
dbt run is stuck by a long running query thats looking for dependency when running on a postgresql database with an excessive number of relations. The query called by the macro postgres__get_relations in postgresql_relations.sql

System information

The output of dbt --version:

installed version: 0.12.1
   latest version: 0.12.1

The operating system you're running on:
MacOS Mojave
Python 3.6.5

Steps to reproduce

In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example models, etc are all very helpful here.
I think this was due to the number of relations we have, we have over 20k relations and almost all of them have at least one dependency. When running this on a smaller postgresql database - ie test with a docker container - I didnt see the issue and dbt run worked as expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request redshift
Projects
None yet
Development

No branches or pull requests

2 participants