[CT-559] [Bug] Partial parser does not recognize default values in envvars

[ajbosco]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

The partial parser does not support default values for envvars. As a result, it will consider all sources deleted since the current value is None even if the default value and previous value are the same and unchanged.

dbt-core/core/dbt/parser/partial.py

Line 959 in 220d8b8

if current_value is None:

Expected Behavior

Since nothing has been changed in the files, I would expect the partial parser to detect the default value matches the previous value and not reparse all the files.

Steps To Reproduce

Include an envvar like this in a source YAML file:

schema: '{{ env_var("OUR_SCHEMA_ENV_VAR", "default_value") }}'

Relevant log output

No response

Environment

- OS: Ubuntu 20.04
- Python: 3.9.12
- dbt: 1.0.4

What database are you using dbt with?

No response

Additional Context

No response

[jtcohen6]

@ajbosco Thanks for opening! Original thread with dbt parse output: #3164 (comment). It sounds like this might save your team up to 2 min on each re-parse.

I'll let someone from the Language team offer their sense of what a fix here might require.

[gshank]

The code that stores the env_var value will actually use the default value if one is provided. So I'm not exactly sure why this wouldn't be working. I'll have to do some investigation.

[gshank]

Oh, I see. We do save the default, but os.getenv won't find it, so it looks different.

I think in order to change that we'd have to save the fact that it uses a default. It would be difficult to handle multiple calls to the same env_var with different defaults.

[gshank]

Maybe instead of saving the default value in the env_var, we could save something like "DEFAULT" and then not compare those env_vars, since presumably if the default changed, the file itself would be marked as changed.