[CT-2432] [CT-2431] [Bug] sqlparse patch version 0.4.4 breaks ephemeral models

[leahwicz]

Description

This is just for Ubuntu and not Mac or Windows.

Failing errors can be seen here:
https://github.com/dbt-labs/dbt-core/actions/runs/4731122949

Environment

- OS:
- Python:
- dbt:all dbt versions

[AuddyPostman]

Just wanted to post an example of this issue that we have faced.
This was hapenning to models which were beginning with WITH and were referring to other models down the lines viaref.

In a normal process, DBT should first create internal CTEs for the ref and then add the given CTE from the WITH clause separated by a comma and remoging the actal WITH. It should look like this:
WITH _dbt_internal_cte1 as (...), _dbt_internal_cte2 as (...), _model_cte as (...), ...

The error that we faced was DBT was wrongly keeping the WITH command. Our model used to look like:
WITH _dbt_internal_cte1 as (...), _dbt_internal_cte2 as (...), WITH _model_cte as (...), ...
A clear error at the start of _model_cte.
Target was redshift.

Solution was to go back to the older version of sqlparse 0.4.3 forcefully in our build script. Waiting for this to be fixed in the next patch release!

[jtcohen6]

@AuddyPostman Thank you so much for the report of the reproduction case! We had trouble reliably reproducing this, except on a specific operating system (Ubuntu) in our functional testing suite. For now, we've pinned to sqlparse<0.4.4, and released patches of all supported dbt-core versions with that pin.

Could you share a bit more about the OS you're using, installed version of Python, etc?

[AuddyPostman]

@jtcohen6 observed this issue in our airflow environment. This is the image that we use for running airflow: https://hub.docker.com/layers/apache/airflow/2.1.2-python3.8/images/sha256-8594489708dcfd8950f1a223f44ce9cec15ba31b1f547d9e59a61ba1ec28cb36?context=explore

we can get the os details from this image!