Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

generate_surrogate_key macro is missing #717

Closed
1 of 5 tasks
dragos-cojocari opened this issue Oct 28, 2022 · 4 comments
Closed
1 of 5 tasks

generate_surrogate_key macro is missing #717

dragos-cojocari opened this issue Oct 28, 2022 · 4 comments
Labels
bug Something isn't working triage

Comments

@dragos-cojocari
Copy link

Describe the bug

Using generate_surrogate_key macro results in errors:

Compilation Error in model ...
13:32:59    'dict object' has no attribute 'generate_surrogate_key'. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

Steps to reproduce

  1. See packages.yaml below
  2. run dbt deps
  3. check the content of dbt_packages/dbt_utils/macros/sql
  4. run a dbt job using dbt_utils.generate_surrogate_key

Also check the content of the 0.9.2 and 1.0.0-beta github release archives, none contains the file.

Expected results

The macro works.

Actual results

DBT errors:

Compilation Error in model ...
13:32:59    'dict object' has no attribute 'generate_surrogate_key'. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

Screenshots and log output

image

System information

The contents of your packages.yml file:

packages:
  - package: dbt-labs/dbt_utils
    version: [">=0.9.2", "<1.0.0"]
  - package: calogica/dbt_expectations
    version: [">=0.6.1", "<0.7.0"]

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.3.0
  - latest:    1.3.0 - Up to date!

Plugins:
  - bigquery: 1.3.0 - Up to date!

Additional context

Are you interested in contributing the fix?
@dragos-cojocari dragos-cojocari added bug Something isn't working triage labels Oct 28, 2022
@dragos-cojocari
Copy link
Author

Installing from git works but it's not a solution as many other packages refer the hub version of dbt-utils

packages:
  - git: "https://github.com/dbt-labs/dbt-utils.git"
    revision: main

@dragos-cojocari
Copy link
Author

a diff between the 2 installations:

diff -rq dbt_packages/dbt_utils.hub dbt_packages/dbt_utils.git

Files dbt_packages/dbt_utils.hub/.circleci/config.yml and dbt_packages/dbt_utils.git/.circleci/config.yml differ
Only in dbt_packages/dbt_utils.git: .git
Files dbt_packages/dbt_utils.hub/CHANGELOG.md and dbt_packages/dbt_utils.git/CHANGELOG.md differ
Files dbt_packages/dbt_utils.hub/LICENSE and dbt_packages/dbt_utils.git/LICENSE differ
Files dbt_packages/dbt_utils.hub/README.md and dbt_packages/dbt_utils.git/README.md differ
Files dbt_packages/dbt_utils.hub/dbt_project.yml and dbt_packages/dbt_utils.git/dbt_project.yml differ
Only in dbt_packages/dbt_utils.hub/integration_tests/data: cross_db
Only in dbt_packages/dbt_utils.hub/integration_tests/data: materializations
Files dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests/data_test_fewer_rows_than_table_1.csv and dbt_packages/dbt_utils.git/integration_tests/data/schema_tests/data_test_fewer_rows_than_table_1.csv differ
Files dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests/data_test_fewer_rows_than_table_2.csv and dbt_packages/dbt_utils.git/integration_tests/data/schema_tests/data_test_fewer_rows_than_table_2.csv differ
Files dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests/data_test_not_constant.csv and dbt_packages/dbt_utils.git/integration_tests/data/schema_tests/data_test_not_constant.csv differ
Only in dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests: data_test_not_null_where.csv
Files dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests/data_test_sequential_values.csv and dbt_packages/dbt_utils.git/integration_tests/data/schema_tests/data_test_sequential_values.csv differ
Only in dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests: data_test_unique_where.csv
Files dbt_packages/dbt_utils.hub/integration_tests/data/schema_tests/schema.yml and dbt_packages/dbt_utils.git/integration_tests/data/schema_tests/schema.yml differ
Only in dbt_packages/dbt_utils.git/integration_tests/data/sql: data_generate_surrogate_key.csv
Only in dbt_packages/dbt_utils.hub/integration_tests/data/sql: data_surrogate_key.csv
Only in dbt_packages/dbt_utils.git/integration_tests/data/sql: data_union_exclude_expected.csv
Only in dbt_packages/dbt_utils.hub/integration_tests/models: cross_db_utils
Files dbt_packages/dbt_utils.hub/integration_tests/models/generic_tests/schema.yml and dbt_packages/dbt_utils.git/integration_tests/models/generic_tests/schema.yml differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/generic_tests/test_fewer_rows_than.sql and dbt_packages/dbt_utils.git/integration_tests/models/generic_tests/test_fewer_rows_than.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/generic_tests/test_recency.sql and dbt_packages/dbt_utils.git/integration_tests/models/generic_tests/test_recency.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/geo/test_haversine_distance_km.sql and dbt_packages/dbt_utils.git/integration_tests/models/geo/test_haversine_distance_km.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/geo/test_haversine_distance_mi.sql and dbt_packages/dbt_utils.git/integration_tests/models/geo/test_haversine_distance_mi.sql differ
Only in dbt_packages/dbt_utils.hub/integration_tests/models: materializations
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/schema.yml and dbt_packages/dbt_utils.git/integration_tests/models/sql/schema.yml differ
Only in dbt_packages/dbt_utils.hub/integration_tests/models/sql: test_deduplicate_deprecated.sql
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_generate_surrogate_key.sql
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_get_column_values.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_get_column_values.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_get_column_values_where.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_get_column_values_where.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_groupby.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_groupby.sql differ
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_not_empty_string_failing.sql
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_not_empty_string_passing.sql
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_nullcheck_table.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_nullcheck_table.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_safe_add.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_safe_add.sql differ
Only in dbt_packages/dbt_utils.hub/integration_tests/models/sql: test_surrogate_key.sql
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_union_exclude_base_lowercase.sql
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_union_exclude_base_uppercase.sql
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_union_exclude_lowercase.sql
Only in dbt_packages/dbt_utils.git/integration_tests/models/sql: test_union_exclude_uppercase.sql
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_unpivot.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_unpivot.sql differ
Files dbt_packages/dbt_utils.hub/integration_tests/models/sql/test_unpivot_bool.sql and dbt_packages/dbt_utils.git/integration_tests/models/sql/test_unpivot_bool.sql differ
Only in dbt_packages/dbt_utils.hub/integration_tests/models/sql: test_unpivot_original_api.sql
Files dbt_packages/dbt_utils.hub/integration_tests/tests/sql/test_get_column_values_use_default.sql and dbt_packages/dbt_utils.git/integration_tests/tests/sql/test_get_column_values_use_default.sql differ
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils: array_append.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils: array_concat.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils: array_construct.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils: cast_array_to_string.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils: current_timestamp.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: any_value.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: bool_or.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: cast_bool_to_text.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: concat.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: datatypes.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: date_trunc.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: dateadd.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: datediff.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: escape_single_quotes.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: except.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: hash.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: identifier.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: intersect.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: last_day.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: length.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: listagg.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: literal.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: position.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: replace.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: right.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: safe_cast.sql
Only in dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated: split_part.sql
Files dbt_packages/dbt_utils.hub/macros/cross_db_utils/deprecated/xdb_deprecation_warning.sql and dbt_packages/dbt_utils.git/macros/cross_db_utils/deprecated/xdb_deprecation_warning.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/at_least_one.sql and dbt_packages/dbt_utils.git/macros/generic_tests/at_least_one.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/equal_rowcount.sql and dbt_packages/dbt_utils.git/macros/generic_tests/equal_rowcount.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/expression_is_true.sql and dbt_packages/dbt_utils.git/macros/generic_tests/expression_is_true.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/fewer_rows_than.sql and dbt_packages/dbt_utils.git/macros/generic_tests/fewer_rows_than.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/mutually_exclusive_ranges.sql and dbt_packages/dbt_utils.git/macros/generic_tests/mutually_exclusive_ranges.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/not_constant.sql and dbt_packages/dbt_utils.git/macros/generic_tests/not_constant.sql differ
Only in dbt_packages/dbt_utils.git/macros/generic_tests: not_empty_string.sql
Files dbt_packages/dbt_utils.hub/macros/generic_tests/not_null_proportion.sql and dbt_packages/dbt_utils.git/macros/generic_tests/not_null_proportion.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/recency.sql and dbt_packages/dbt_utils.git/macros/generic_tests/recency.sql differ
Files dbt_packages/dbt_utils.hub/macros/generic_tests/sequential_values.sql and dbt_packages/dbt_utils.git/macros/generic_tests/sequential_values.sql differ
Only in dbt_packages/dbt_utils.hub/macros/generic_tests: test_not_null_where.sql
Only in dbt_packages/dbt_utils.hub/macros/generic_tests: test_unique_where.sql
Only in dbt_packages/dbt_utils.hub/macros: materializations
Files dbt_packages/dbt_utils.hub/macros/sql/deduplicate.sql and dbt_packages/dbt_utils.git/macros/sql/deduplicate.sql differ
Only in dbt_packages/dbt_utils.git/macros/sql: generate_surrogate_key.sql
Files dbt_packages/dbt_utils.hub/macros/sql/pivot.sql and dbt_packages/dbt_utils.git/macros/sql/pivot.sql differ
Files dbt_packages/dbt_utils.hub/macros/sql/safe_add.sql and dbt_packages/dbt_utils.git/macros/sql/safe_add.sql differ
Files dbt_packages/dbt_utils.hub/macros/sql/surrogate_key.sql and dbt_packages/dbt_utils.git/macros/sql/surrogate_key.sql differ
Files dbt_packages/dbt_utils.hub/macros/sql/union.sql and dbt_packages/dbt_utils.git/macros/sql/union.sql differ
Files dbt_packages/dbt_utils.hub/macros/sql/unpivot.sql and dbt_packages/dbt_utils.git/macros/sql/unpivot.sql differ
Only in dbt_packages/dbt_utils.hub/tests: functional

@joellabes
Copy link
Contributor

@dragos-cojocari generate_surrogate_key is the new version of the currently-shipping surrogate_key (see #685 for the backstory).

main does not reflect the code shipping in 0.9.5, see https://github.com/dbt-labs/dbt-utils/tree/0.9.5

dbt utils 1.0.0 will ship very soon with the new generate_surrogate_key macro

@dragos-cojocari
Copy link
Author

Got it, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working triage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@joellabes @dragos-cojocari