Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Render sql_header for dbt show #8436

Closed
wants to merge 3 commits into from
Closed

Render sql_header for dbt show #8436

wants to merge 3 commits into from

Conversation

jtcohen6
Copy link
Contributor

@jtcohen6 jtcohen6 commented Aug 16, 2023

resolves #8417

Problem

sql_header is not rendered for dbt show. It is while running dbt models, as {{ sql_header }} is included within the materialization logic.

Solution

Always render sql_header, if configured, before prepending to the model's compiled_code

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@cla-bot cla-bot bot added the cla:yes label Aug 16, 2023
@github-actions
Copy link
Contributor

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@codecov
Copy link

codecov bot commented Aug 16, 2023

Codecov Report

Patch coverage: 22.22% and project coverage change: -2.72% ⚠️

Comparison is base (5182e3c) 86.61% compared to head (4b46087) 83.89%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8436      +/-   ##
==========================================
- Coverage   86.61%   83.89%   -2.72%     
==========================================
  Files         175      175              
  Lines       25595    25602       +7     
==========================================
- Hits        22168    21478     -690     
- Misses       3427     4124     +697     
Flag Coverage Δ
integration 80.07% <22.22%> (-3.40%) ⬇️
unit 65.11% <22.22%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed Coverage Δ
core/dbt/task/show.py 33.87% <22.22%> (-64.32%) ⬇️

... and 55 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jtcohen6 jtcohen6 marked this pull request as ready for review August 17, 2023 14:47
@jtcohen6 jtcohen6 requested a review from a team as a code owner August 17, 2023 14:47
@jtcohen6 jtcohen6 requested a review from MichelleArk August 17, 2023 14:47
if "sql_header" in compiled_node.unrendered_config:
compiled_node.compiled_code = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we were mutating the compiled_code attribute on compiled_node directly in addition to executing it, whereas the changes here just execute it.

I imagine modifying compiled_node.compiled_code had some downstream effects such as surfacing the sql_header in the target/compiled files. Is the motivation for this change to be more consistent with what target/compiled files typically contain for run models?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the motivation for this change to be more consistent with what target/compiled files typically contain for run models?

Yes, this is what I was thinking. The sql_header isn't part of a node's compiled_code when it's compiled or materialized, so it feels inconsistent that it would be included just for the manifest produced by show.

Copy link
Member

@aranke aranke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one potential optimization.

# See dbt-core issues #2793, #3264, #7151
# So technically this should be "generate_parser_model_context" (I think) instead of "generate_runtime_model_context"
# Generating the context will be slower if we don't actually need to render the sql_header (if it contains no Jinja)
context = generate_runtime_model_context(compiled_node, self.config, manifest)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One dumb optimization we could make here: only use this slower code path if there is a { in the string.

Copy link
Contributor Author

@jtcohen6 jtcohen6 Sep 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we have some prior art for that: _HAS_RENDER_CHARS_PAT.search().

We bake this into our Jinja rendering, to no-op if there are no render characters. What do you think? Worth doing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up to you, honestly. Timeboxing to 15 minutes seems fine to me.

@MichelleArk
Copy link
Contributor

I think this will be resolved by #8641, but have copied over the tests: d77bfe2

@jtcohen6
Copy link
Contributor Author

Closing in favor of #8641

@jtcohen6 jtcohen6 closed this Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-2994] [Bug] sql_header with jinja not parsed correctly with dbt show
3 participants