Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add deleted connector statuses to fivetran_platform__connector_status #112

Merged
merged 5 commits into from
Jan 24, 2024

Conversation

fivetran-avinash
Copy link
Contributor

@fivetran-avinash fivetran-avinash commented Jan 19, 2024

PR Overview

This PR will address the following Issue/Feature: [#108]

This PR will result in the following new package version: 1.4.3

Column added in a staging model does not cause breaking changes.

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

  • Updated logic for connector_health dimension in fivetran_platform__connector_status to show deleted for connectors that had been removed. Previously the connector would report the last known status before deletion, which is inaccurate based on the definition of this measure.
  • Brought in the is_deleted dimension (based on the _fivetran_deleted value) to stg_fivetran__platform__connector to capture connectors that are deleted in the downstream fivetran_platform__connector_status model.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

  • dbt run –full-refresh && dbt test
  • dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

  • The appropriate issue has been linked, tagged, and properly assigned
  • All necessary documentation and version upgrades have been applied
  • docs were regenerated (unless this PR does not include any code or yml updates)
  • BuildKite integration tests are passing
  • Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps: See Height ticket.

If you had to summarize this PR in an emoji, which would it be?

🍵

Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash thanks for working through this PR! This is looking great, I just have a few considerations that we should apply before being approved. Let me know if you have any questions. Thanks!

models/staging/stg_fivetran_platform__connector.sql Outdated Show resolved Hide resolved
Copy link
Contributor Author

@fivetran-avinash fivetran-avinash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-joemarkiewicz Changes applied! Ran into an error re: SQL Server, but resolved it I think.

models/staging/stg_fivetran_platform__connector.sql Outdated Show resolved Hide resolved
Copy link
Contributor

@fivetran-joemarkiewicz fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fivetran-avinash thanks for making the requested changes. This looks good to go with just one small suggestion to simplify the conditional logic in the staging model.

@@ -25,7 +25,8 @@ final as (
destination_id,
connecting_user_id,
paused as is_paused,
signed_up as set_up_at
signed_up as set_up_at,
coalesce(_fivetran_deleted, {% if target.type == 'sqlserver' %} 0 {% else %} false {% endif %}) as is_deleted
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small suggestion to consolidate the conditional code here.

Suggested change
coalesce(_fivetran_deleted, {% if target.type == 'sqlserver' %} 0 {% else %} false {% endif %}) as is_deleted
coalesce(_fivetran_deleted,{{ ' 0 ' if target.type == 'sqlserver' else ' false'}}) as is_deleted

@fivetran-avinash fivetran-avinash merged commit a1a4662 into main Jan 24, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants