bugfix/redshift-large-int

[fivetran-joemarkiewicz]

PR Overview

This PR will address the following Issue/Feature: Internally raised issue

This PR will result in the following new package version: v1.6.0

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

Breaking Changes

⚠️ Since the following changes result in a field changing datatype, we recommend running a --full-refresh after upgrading to this version to avoid possibly incremental failures.

• The following fields in the fivetran_platform__audit_table model have been updated to be cast as dbt.type_bigint() (previously was dbt.type_int())
  • sum_rows_replaced_or_inserted
  • sum_rows_updated
  • sum_rows_deleted

Bug Fixes

• The following fields in the fivetran_platform__connector_daily_events model have been updated to be cast as dbt.type_bigint() (previously was dbt.type_int())
  • count_api_calls
  • count_record_modifications
  • count_schema_changes

Under the Hood

• Modified log seed data within the integration tests folder to ensure a large integers are being tested as part of our integration tests.

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

• dbt run –full-refresh && dbt test
• dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

• [n/a] The appropriate issue has been linked, tagged, and properly assigned
• All necessary documentation and version upgrades have been applied
• docs were regenerated (unless this PR does not include any code or yml updates)
• BuildKite integration tests are passing
• Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

I was able to reproduce this error locally by editing the seed data to insert a count record in the message_data field within the log data to be a large integer that exists outside the normal range (Integer valid range -2147483648 to 2147483647). See the below screenshot where I was able to reproduce this error using the live version of the package.

Additionally, I was able to see this error be resolved when using this branch version of the package.

Lastly, we can see the resulting audit_table and connector_daily_events end model fields look as we would expect following these updates.

Following these validations, I feel comfortable this should address the issue being reported for the integers being out of range for the respective fields.

If you had to summarize this PR in an emoji, which would it be?

🔢

[fivetran-catfritz]

Just one small typo, but lgtm and approved!

I was able to recreate the issue with the seed data you provided, and using this branch I was able to resolve it.

[fivetran-jamie]

lgtm