Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshotting with check_cols='all' doesn't handle schema changes when new columns are added #1797

Closed
1 task done
joshpeng opened this issue Oct 1, 2019 · 0 comments · Fixed by #2001
Closed
1 task done
Labels
bug Something isn't working snapshots Issues related to dbt's snapshot functionality
Milestone

Comments

@joshpeng
Copy link

joshpeng commented Oct 1, 2019

Describe the bug

When the source table has schema change via a new column addition, dbt doesn't handle the change when using check_cols='all'. The check_cols is likely trying to hash the non-existent column in the snapshot table before adding the column to the table.

Steps To Reproduce

  1. Snapshot a source table with strategy='check' and check_cols='all'
  2. Add a column to the source table
  3. Run dbt snapshot

Expected behavior

dbt to add the new column and continue along snapshotting gracefully.

Screenshots and log output

Database Error in snapshot pallets_snapshot (snapshots/snapshot_teal_pallets.sql)
  000904 (42000): 018f405d-0215-21e7-0000-5a0102fe42da: SQL compilation error: error line 206 at position 12
  invalid identifier 'SNAPSHOTTED_DATA.PALLET_TYPE_ID'

System information

Which database are you using dbt with?

  • snowflake

The output of dbt --version:

installed version: 0.14.2
   latest version: 0.14.2

Up to date!

The operating system you're using:
macOS

The output of python --version:
Python 3.7.3

@joshpeng joshpeng added bug Something isn't working triage labels Oct 1, 2019
@drewbanin drewbanin added snapshots Issues related to dbt's snapshot functionality and removed triage labels Oct 1, 2019
@drewbanin drewbanin added this to the Louisa May Alcott milestone Oct 10, 2019
@drewbanin drewbanin modified the milestones: Louisa May Alcott, 0.15.1 Nov 4, 2019
beckjake added a commit that referenced this issue Dec 13, 2019
…ll-added-column

Fix snapshot check all with an added column (#1797)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working snapshots Issues related to dbt's snapshot functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants