Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle a number of bad failure-path behaviors (#1807) #1839

Merged
merged 1 commit into from
Oct 17, 2019

Conversation

beckjake
Copy link
Contributor

Fixes #1807

This is one of those bugs where I don't see the bad behavior anymore so I assume it's ok. This was hard to write tests for, I had to use our internal analytics project with 8 threads to reproduce the issue: I assume it needs a large and complex test case to trigger. And I don't know where I'd even pull debugging information from.

Anyway, my attempts at fixing the issue described:

  • When a connection is missing, raise a special exception
    • instead of RuntimeError, this derives from our RuntimeException, so we catch/suppress it better
  • Be graceful if a connection does not exist during nice_connection_name
  • Handle the case where exceptions caught by the Snowflake exception handler do not have a 'msg' attr
  • Re-raise exceptions in the adapter exception handlers "from" the originating error so we don't get big double stack traces about "during handling... another exception was raised"

- When a connection is missing, raise a special exception
   - instead of RuntimeError, so we catch it better
- Be graceful if a connection does not exist during nice_connection_name
- Handle the case where exceptions caught by the Snowflake exception handler do not have a 'msg' attr
- Re-raise exceptions in the adapter exception handlers "from" the originating error
@beckjake beckjake requested a review from drewbanin October 17, 2019 13:45
@cla-bot cla-bot bot added the cla:yes label Oct 17, 2019
@clrcrl
Copy link
Contributor

clrcrl commented Oct 17, 2019

Tested this out with the problematic dbt project, and it LGTM:

on dev/louisa-may-alcott

$ dbt compile
Running with dbt=0.15.0-a1
Found 106 models, 71 tests, 1 snapshot, 0 analyses, 278 macros, 1 operation, 6 seed files, 23 sources

10:00:14 | Concurrency: 8 threads (target='dev')
10:00:14 |
Unhandled error while executing model.fishtown_internal_analytics.stg_dbt_platform_contexts
connection never acquired for thread (38571, 123145532657664), have []
Unhandled error while executing model.fishtown_internal_analytics.stg_snowplow_web_page_contexts
connection never acquired for thread (38571, 123145527402496), have []
Unhandled error while executing model.fishtown_internal_analytics.stg_dbt_invocation_env_contexts
connection never acquired for thread (38571, 123145495871488), have []
Unhandled error while executing model.fishtown_internal_analytics.stg_dbt_run_model_contexts
connection never acquired for thread (38571, 123145511636992), have []
Encountered an error:
Runtime Error
  Database Error in model fct_opportunities (models/marts/sales/fct_opportunities.sql)
    002003 (02000): SQL compilation error:
    Schema 'ANALYTICS.DBT_CLAIRE' does not exist.

on fix/snowflake-query-bad-errors

$ dbt compile
Running with dbt=0.15.0-a1
Found 106 models, 71 tests, 1 snapshot, 0 analyses, 278 macros, 1 operation, 6 seed files, 23 sources

09:59:04 | Concurrency: 8 threads (target='dev')
09:59:04 |
Encountered an error:
Runtime Error
  Database Error in model fct_opportunities (models/marts/sales/fct_opportunities.sql)
    002003 (02000): SQL compilation error:
    Schema 'ANALYTICS.DBT_CLAIRE' does not exist.

Copy link
Contributor

@drewbanin drewbanin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this LGTM - ship it!

@beckjake beckjake merged commit a7b0d05 into dev/louisa-may-alcott Oct 17, 2019
@beckjake beckjake deleted the fix/snowflake-query-bad-errors branch October 17, 2019 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Weird error messages on 0.14.2 && Snowflake
3 participants