Reconciliation

The reconciliation feature was created to ensure data consistency between the synchronizer server and the synchronizer client. It addresses scenarios where objects may have been deleted or have been modified without proper handling, providing a mechanism for periodic reconciliation.

As part of this feature, we have added support for Batch message, which is simply an array of the existing supported message. A Batch can have a Batch type, which means that the client processing the batch can implement a specific processing function for a specific batch.

sequenceDiagram
    participant Kubernetes API
    participant Sync InCluster
    Sync InCluster->Sync BE: Connection Established
    loop Every X minutes
        Sync BE->>+Pulsar: send ReconciliationRequestMessage
        Note over Sync BE,Pulsar: Payload: <br/> {"Initiator": "Sync BE", "Time": "2024-01-01 09:40"}
    end
        Pulsar->>+Ingester: Consume ReconciliationRequestMessage
        Ingester->>+Postgres: SELECT ResourceID, ResourceVersion
        Postgres->>+Ingester: ^All Resources
        Note over Postgres,Ingester:  ^Only those which are NOT managed by the initiator
        Ingester->>+Pulsar: send ReconciliationRequestMessage
        Note over Ingester,Pulsar: Payload:  <br />{"kind-1": ["R1-v1", "R2-v3"]}
        Pulsar->>+Sync BE: Consume ReconciliationRequestMessage

        Sync BE->>+Sync InCluster: Batch callback - Kind 1
        Sync BE->>+Sync InCluster: Batch callback - Kind 2
        Sync BE->>+Sync InCluster: Batch callback - Kind N
    loop In each Batch
        Sync InCluster->>+Kubernetes API: List 
        Kubernetes API->>+Sync InCluster: Resources
        loop For each object
            Sync InCluster->>Sync InCluster: Compare Resource Version
            Sync InCluster->>+Sync BE: DeleteObject (if missing) 
            Sync BE->>+Pulsar: DeleteObject 
            Sync InCluster->>+Sync BE: PutObject (if changed)
            Sync BE->>+Pulsar: PutObject
        end
    end
        
    
    Pulsar->>+Ingester: Consume DeleteObject / PutObject
    Ingester->>+Postgres: DELETE / UPSERT
    

Loading