[CT-185] Remove use of temporary table for merge incremental strategy in Snowflake

[jnatkins]

Describe the feature

The MERGE command in Snowflake supports the use of arbitrary SELECT queries in the USING clause. If we were to adopt this approach for the merge incremental strategy, it remove a database write step that we are currently performing due to the creation of a temporary table to store the data to-be-merged.

Describe alternatives you've considered

The alternative is leaving the status quo. This would represent a performance improvement for the adapter when using the merge strategy.

Additional context

Please include any other relevant context here.

Who will this benefit?

Any customer who is working with incremental tables that have very large incremental loads stand to see a performance improvement with this.

Are you interested in contributing this feature?

Yes, to an extent. I can get some of the way, but would probably need assistance getting this to a merge-able state.

[jtcohen6]

Thanks for opening the issue + PR @jnatkins! I'm about making this change for merge. Also linking a community slack thread about this from last month.

I don't think we can support this change for delete+insert, since the intermediate relation helps us ensure absolute consistency between delete + insert.

I'll take a look at the PR, and leave a few quick comments.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please remove the stale label or comment on the issue, or it will be closed in 7 days.

[jnatkins]

@jtcohen6 Should this be reopened? It looked like there was movement on my PR, but not sure if that's still in flight.