Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] JSON parsing error in Redshift #113

Closed
2 of 4 tasks
fivetran-reneeli opened this issue Jan 29, 2024 · 1 comment
Closed
2 of 4 tasks

[Bug] JSON parsing error in Redshift #113

fivetran-reneeli opened this issue Jan 29, 2024 · 1 comment
Assignees
Labels
error:forced priority:p2 Affects most users; fix needed status:in_progress Currently being worked on type:bug Something is broken or incorrect update_type:models Primary focus requires model updates

Comments

@fivetran-reneeli
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Describe the issue

Error downstream in fivetran_platform_connector_status and fivetran_platform_audit_table due to the message_data the log table getting truncated due to lengthy json value. The incomplete json is causing the model runs to fail.

In order to support these json objects in Redshift the SUPER type is necessary but it is complicated.

As a workaround, we should update in the package to filter message_data first for only necessary keys prior to any parsing (event_subtype='status')

Relevant error log or model output

No response

Expected behavior

json to be parsed correctly and package to run

dbt Project configurations

na

Package versions

na

What database are you using dbt with?

redshift

dbt Version

na

Additional Context

No response

Are you willing to open a PR to help address this issue?

  • Yes.
  • Yes, but I will need assistance and will schedule time during our office hours for guidance
  • No.
@fivetran-joemarkiewicz fivetran-joemarkiewicz added type:bug Something is broken or incorrect error:forced labels Jan 30, 2024
@fivetran-catfritz fivetran-catfritz self-assigned this Feb 9, 2024
@fivetran-catfritz fivetran-catfritz added priority:p2 Affects most users; fix needed status:in_progress Currently being worked on update_type:models Primary focus requires model updates labels Feb 9, 2024
@fivetran-catfritz
Copy link
Contributor

This has been addressed in release v1.5.0, which is now live on the dbt hub. Closing the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
error:forced priority:p2 Affects most users; fix needed status:in_progress Currently being worked on type:bug Something is broken or incorrect update_type:models Primary focus requires model updates
Projects
None yet
Development

No branches or pull requests

3 participants