Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On ephemeral compile errors that lead to skips, generate real errors (#1037) #1053

Merged
merged 2 commits into from
Oct 15, 2018

Conversation

beckjake
Copy link
Contributor

When an ephemeral model fails and downstream models are skipped, explicitly mark those skipped downstream models as being skipped due to an error, and alter their reporting accordingly, including dbt's exit code.

Old output:

> dbt run
echo $?Found 3 models, 0 tests, 0 archives, 0 analyses, 90 macros, 1 operations, 1 seed files

11:29:21 | Concurrency: 2 threads (target='default')
11:29:21 |
11:29:21 | 1 of 2 START view model dbt_postgres.moby_dick_base.................. [RUN]
11:29:21 | 1 of 2 OK created view model dbt_postgres.moby_dick_base............. [CREATE VIEW in 0.04s]
11:29:21 | 2 of 2 SKIP relation dbt_postgres.word_count......................... [SKIP]
11:29:21 |
11:29:21 | Finished running 2 view models in 0.11s.

Completed successfully

Done. PASS=1 ERROR=0 SKIP=1 TOTAL=2
> echo $?
0

New output:

> dbt run
Found 3 models, 0 tests, 0 archives, 0 analyses, 90 macros, 1 operations, 1 seed files

10:32:18 | Concurrency: 2 threads (target='default')
10:32:18 |
10:32:18 | 1 of 2 START view model dbt_postgres.moby_dick_base.................. [RUN]
10:32:18 | 1 of 2 OK created view model dbt_postgres.moby_dick_base............. [CREATE VIEW in 0.04s]
10:32:19 | 2 of 2 SKIP relation dbt_postgres.word_count due to ephemeral model error [ERROR SKIP]
Compilation Error in model ephemeral_word_count (models/ephemeral_word_count.sql)
  'dbt.context.common.DatabaseWrapper object' has no attribute 'invalid_method'
10:32:19 |
10:32:19 | Finished running 2 view models in 0.11s.

Completed with 1 errors:

Compilation Error in referenced ephemeral model dbt_postgres.ephemeral_word_count

Done. PASS=1 ERROR=1 SKIP=0 TOTAL=2
> echo $?
1

Jacob Beck added 2 commits October 12, 2018 10:33
On an ephemeral model failure mark dependencies as being skipped due to errors, and log them.
@beckjake beckjake requested a review from drewbanin October 12, 2018 17:32
@beckjake beckjake changed the title On ephemeral compile errors that lead to skips, generate real errors On ephemeral compile errors that lead to skips, generate real errors (#1037) Oct 12, 2018
@beckjake
Copy link
Contributor Author

Fixes #1037

Copy link
Contributor

@drewbanin drewbanin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good to me. I gave it a couple of spins and I think the stdout logging looks great!

:shipit:

self.skip = True
self.skip_cause = cause
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like this

@beckjake beckjake merged commit dd25750 into dev/guion-bluford Oct 15, 2018
@beckjake beckjake deleted the fix/ephemeral-compile-errors branch October 15, 2018 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants