You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem Description:
Assume we have three tables, inventory (primary key is of type UUID), feature_map and features(Primary key is of type Long)
when we try the update/insert following exception is thrown
opensearchpy.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'failed to create query: For input string: "6cc0eb22-2703-4fce-bf24-eafe6900f059"')
This is due to _meta.features.id being of type long in elasticsearch but pgsync trying to create a terms query with value "6cc0eb22-2703-4fce-bf24-eafe6900f059"
After doing some investigation, the reason behind this failure is this peace of code in symc.py:_root_foreign_key_resolver() function
foreign_values: list = [
payload.new.get(key) for key in foreign_keys[node.name]
]
this code will generate a terms query like [1, '6cc0eb22-2703-4fce-bf24-eafe6900f059'] which is invalid (two different types)
there are two issues with this code, first is the one I pointed out in this ticket with different id types, the other one I believe is since it's adding both foreign key values in feature_map table, even if they are the same type it's looking for the wrong id since it's inserting id for inventory too
I was able to fix this with replacing this with the following code (Column is of sqlalchemy type)
foreign_values: list = []
for column in node.columns:
if isinstance(column, Column):
for foreign_key in column.foreign_keys:
referenced_column = foreign_key.column
base_table = referenced_column.table
if base_table.name == node.parent.table:
foreign_values.append(payload.new.get(column.name))
also, in general, when there is an exception is thrown, the pgsync completely stops processing any data instead of moving to the next data
The text was updated successfully, but these errors were encountered:
PGSync version: main branch
Postgres version: latest
Elasticsearch/OpenSearch version: latest
Redis version: latest
Python version: 3.12.5
Problem Description:
Assume we have three tables, inventory (primary key is of type UUID), feature_map and features(Primary key is of type Long)
when we try the update/insert following exception is thrown
This is due to
_meta.features.id
being of typelong
in elasticsearch but pgsync trying to create a terms query with value"6cc0eb22-2703-4fce-bf24-eafe6900f059"
After doing some investigation, the reason behind this failure is this peace of code in symc.py:_root_foreign_key_resolver() function
this code will generate a terms query like
[1, '6cc0eb22-2703-4fce-bf24-eafe6900f059']
which is invalid (two different types)there are two issues with this code, first is the one I pointed out in this ticket with different id types, the other one I believe is since it's adding both foreign key values in
feature_map
table, even if they are the same type it's looking for the wrong id since it's inserting id for inventory tooI was able to fix this with replacing this with the following code (
Column
is ofsqlalchemy
type)also, in general, when there is an exception is thrown, the pgsync completely stops processing any data instead of moving to the next data
The text was updated successfully, but these errors were encountered: