You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Problem Description:
using a materialized view as source, with 3 base tables with diferent pks, the index gets created on opensearch, but any update on any of the base tables gives the following error:
O:main:2024-08-13 16:24:41.987:DEBUG:pgsync.sync: on_publish len 1
INFO:main:2024-08-13 16:24:41.988:ERROR:pgsync.sync: Primary keys ['sd_secid', 'secid', 'sm_secid'] not subset of payload data dict_keys(['sm_secid']) for table master_ref_data.v_security_stats_mv2
INFO:main:NoneType: None
INFO:main:2024-08-13 16:24:41.988:ERROR:pgsync.search_client: Exception No active exception to reraise
INFO:main:Traceback (most recent call last):
INFO:main:File "/usr/local/lib/python3.12/site-packages/pgsync/search_client.py", line 133, in bulk
INFO:main:self._bulk(
INFO:main:File "/usr/local/lib/python3.12/site-packages/pgsync/search_client.py", line 171, in _bulk
INFO:main:for _ in self.streaming_bulk(
INFO:main:^^^^^^^^^^^^^^^^^^^^
INFO:main:File "/usr/local/lib/python3.12/site-packages/opensearchpy/helpers/actions.py", line 328, in streaming_bulk
INFO:main:for bulk_data, bulk_actions in _chunk_actions(
INFO:main:^^^^^^^^^^^^^^^
INFO:main:File "/usr/local/lib/python3.12/site-packages/opensearchpy/helpers/actions.py", line 168, in _chunk_actions
PGSync version: 3.2
Postgres version: 14.5
Elasticsearch/OpenSearch version: OpenSearch 2.13
Redis version: 6.0.16
Python version: 3.10
Problem Description:
using a materialized view as source, with 3 base tables with diferent pks, the index gets created on opensearch, but any update on any of the base tables gives the following error:
O:main:2024-08-13 16:24:41.987:DEBUG:pgsync.sync: on_publish len 1
INFO:main:2024-08-13 16:24:41.988:ERROR:pgsync.sync: Primary keys ['sd_secid', 'secid', 'sm_secid'] not subset of payload data dict_keys(['sm_secid']) for table master_ref_data.v_security_stats_mv2
INFO:main:NoneType: None
INFO:main:2024-08-13 16:24:41.988:ERROR:pgsync.search_client: Exception No active exception to reraise
INFO:main:Traceback (most recent call last):
INFO:main:File "/usr/local/lib/python3.12/site-packages/pgsync/search_client.py", line 133, in bulk
INFO:main:self._bulk(
INFO:main:File "/usr/local/lib/python3.12/site-packages/pgsync/search_client.py", line 171, in _bulk
INFO:main:for _ in self.streaming_bulk(
INFO:main:^^^^^^^^^^^^^^^^^^^^
INFO:main:File "/usr/local/lib/python3.12/site-packages/opensearchpy/helpers/actions.py", line 328, in streaming_bulk
INFO:main:for bulk_data, bulk_actions in _chunk_actions(
INFO:main:^^^^^^^^^^^^^^^
INFO:main:File "/usr/local/lib/python3.12/site-packages/opensearchpy/helpers/actions.py", line 168, in _chunk_actions
INFO:main:for action, data in actions:
This is my schema.json:
[
{
"database": "client_facing",
"index": "v_security_stats_mv2",
"nodes": {
"table": "v_security_stats_mv2",
"base_tables": ["fi_security", "sec_master", "sec_detl"],
"schema": "master_ref_data",
"columns": [
"isfixedincome",
"status",
"secid",
"sd_secid",
"sm_secid",
"comp_id",
"cusip",
"company_name",
"symbol"
],
"primary_key": ["sd_secid","secid","sm_secid"]
}
}
]
sd_secid is the PK of sec_detl, secid is the PK of fi_security and sm_secid is the pk of sec_master
Error Message (if any):
The text was updated successfully, but these errors were encountered: