Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix] Add limit 0 to credits_used and usage_cost to fix Postgres errors #109

Merged
merged 13 commits into from
Jan 10, 2024

Conversation

fivetran-avinash
Copy link
Contributor

@fivetran-avinash fivetran-avinash commented Jan 4, 2024

PR Overview

This PR will address the following Issue/Feature: [#104]

This PR will result in the following new package version: 1.4.2

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Bug Fixes

  • Adjusted staging models stg_fivetran_platform__credits_used and stg_fivetran_platform__usage_cost to return empty tables if the respective fivetran_platform__credits_pricing and the fivetran_platform__usage_pricing variables for each model are disabled to avoid Postgres data type errors if those tables are null.

Under the Hood

  • Added additional scripts to run_models.sh for the package with the variables fivetran_platform__credits_pricing set to false and fivetran_platform__usage_pricing set to true to recreate the error.
  • Updated seed files to ensure downstream models properly populate into fivetran_platform__usage_mar_destination_history for validating this PR.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • [NA] dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:
See Height ticket

If you had to summarize this PR in an emoji, which would it be?

💳

@fivetran-avinash fivetran-avinash self-assigned this Jan 4, 2024
@fivetran-avinash fivetran-avinash changed the title Update seeds, regen docs, merge 1.4.1 changes [Bugfix] Add limit 0 to credits_used and usage_cost to fix Postgres errors Jan 4, 2024
Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash thanks for working through this! This is looking good, but I have a few comments and suggestions below. Let me know if you have any questions - we can sync on the SQL Server one in more detail next week, but you should be able to start building out the bones of a conditional knowing SQL Server likely doesn't support the limit function.

README.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
models/staging/stg_fivetran_platform__credits_used.sql Outdated Show resolved Hide resolved
@fivetran-avinash
Copy link
Contributor Author

@fivetran-joemarkiewicz Changes applied and ready for re-review.

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash these changes look great to me, thanks for applying them! I have approved the PR as your changes look good, but I would like to hold off merging and releasing until we can get the sqlserver warehouse back up and running to fully test.

Once Buildkite is passing on the soon to be recreated sqlserver warehouse, we can initialize the release review process.

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! With one suggestion in order to get BK passing.

integration_tests/ci/sample.profiles.yml Outdated Show resolved Hide resolved
@fivetran-avinash fivetran-avinash merged commit a38b14b into main Jan 10, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] empty credits_used table causing date-casting error in fivetran_platform__usage_mar_destination_history
2 participants