You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#1777 was resolved by #1778 but theres a very similar bug in _process_schema_message in the target where the comparison always returns false when SDC metadata columns are included.
My shopify stream has a bunch of schema messages that cause it to constantly log the fact that it needs to reinitialize a sink because the schema changed. When I added a breakpoint I found that the only difference is the SDC properties.
2023-09-15 16:09:49,858 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:09:50,537 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:09:50,537 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:09:50,537 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:09:50,537 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:09:53,004 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:09:53,004 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:09:53,004 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:09:53,004 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:09:54,800 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:09:54,800 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:09:54,800 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:09:54,800 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:09:56,622 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:09:56,622 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:09:56,622 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:09:56,622 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:09:58,445 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:09:58,445 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:09:58,445 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:09:58,445 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:10:00,275 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:10:00,275 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:10:00,275 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:10:00,275 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:10:02,076 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:10:02,076 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:10:02,076 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:10:02,076 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:10:03,926 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:10:03,926 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:10:03,926 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:10:03,926 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:10:05,793 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:10:05,794 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:10:05,794 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:10:05,794 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:10:07,671 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:10:07,672 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:10:07,672 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:10:07,672 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
2023-09-15 16:10:09,516 | INFO | target-postgres | Schema has changed for stream 'shopify_transactions'. Mapping definitions will be reset.
2023-09-15 16:10:09,516 | INFO | target-postgres | Schema or key properties for 'shopify_transactions' stream have changed. Initializing a new 'shopify_transactions' sink...
2023-09-15 16:10:09,516 | INFO | target-postgres | Initializing 'target-postgres' target sink...
2023-09-15 16:10:09,516 | INFO | target-postgres | Initializing target sink for stream 'shopify_transactions'...
The text was updated successfully, but these errors were encountered:
#1777 was resolved by #1778 but theres a very similar bug in _process_schema_message in the target where the comparison always returns false when SDC metadata columns are included.
My shopify stream has a bunch of schema messages that cause it to constantly log the fact that it needs to reinitialize a sink because the schema changed. When I added a breakpoint I found that the only difference is the SDC properties.
The text was updated successfully, but these errors were encountered: