Can't set `externalClusters` #424

Ofirfr · 2024-10-25T11:10:43Z

Just as title, the helm chart does not support setting externalClusters field, the field can be seen in docs:
https://cloudnative-pg.io/documentation/1.24/database_import/

The text was updated successfully, but these errors were encountered:

itay-grudev · 2024-10-25T12:56:53Z

The chart is responsible for filling in the externalClusters section based on your recovery configuration. I don't want to expose it.

charts/charts/cluster/templates/_bootstrap.tpl

Lines 43 to 90 in 3c5dd5d

    
           externalClusters: 
        
           - name: pgBaseBackupSource 
        
             connectionParameters: 
        
               host: {{ .Values.recovery.pgBaseBackup.source.host | quote }} 
        
               port: {{ .Values.recovery.pgBaseBackup.source.port | quote }} 
        
               user: {{ .Values.recovery.pgBaseBackup.source.username | quote }} 
        
               dbname: {{ .Values.recovery.pgBaseBackup.source.database | quote }} 
        
               sslmode: {{ .Values.recovery.pgBaseBackup.source.sslMode | quote }} 
        
             {{- if .Values.recovery.pgBaseBackup.source.passwordSecret.name }} 
        
             password: 
        
               name: {{ default (printf "%s-pg-basebackup-password" (include "cluster.fullname" .)) .Values.recovery.pgBaseBackup.source.passwordSecret.name }} 
        
               key: {{ .Values.recovery.pgBaseBackup.source.passwordSecret.key }} 
        
             {{- end }} 
        
             {{- if .Values.recovery.pgBaseBackup.source.sslKeySecret.name }} 
        
             sslKey: 
        
               name: {{ .Values.recovery.pgBaseBackup.source.sslKeySecret.name }} 
        
               key: {{ .Values.recovery.pgBaseBackup.source.sslKeySecret.key }} 
        
             {{- end }} 
        
             {{- if .Values.recovery.pgBaseBackup.source.sslCertSecret.name }} 
        
             sslCert: 
        
               name: {{ .Values.recovery.pgBaseBackup.source.sslCertSecret.name }} 
        
               key: {{ .Values.recovery.pgBaseBackup.source.sslCertSecret.key }} 
        
             {{- end }} 
        
             {{- if .Values.recovery.pgBaseBackup.source.sslRootCertSecret.name }} 
        
             sslRootCert: 
        
               name: {{ .Values.recovery.pgBaseBackup.source.sslRootCertSecret.name }} 
        
               key: {{ .Values.recovery.pgBaseBackup.source.sslRootCertSecret.key }} 
        
             {{- end }} 
        
           {{- else }} 
        
             recovery: 
        
               {{- with .Values.recovery.pitrTarget.time }} 
        
               recoveryTarget: 
        
                 targetTime: {{ . }} 
        
               {{- end }} 
        
               {{- if eq .Values.recovery.method "backup" }} 
        
               backup: 
        
                 name: {{ .Values.recovery.backupName }} 
        
               {{- else if eq .Values.recovery.method "object_store" }} 
        
               source: objectStoreRecoveryCluster 
        
               {{- end }} 
        
           externalClusters: 
        
             - name: objectStoreRecoveryCluster 
        
               barmanObjectStore: 
        
                 serverName: {{ .Values.recovery.clusterName }} 
        
                 {{- $d := dict "chartFullname" (include "cluster.fullname" .) "scope" .Values.recovery "secretPrefix" "recovery" -}} 
        
                 {{- include "cluster.barmanObjectStoreConfig" $d | nindent 4 }}

Import is not yet supported. There is a PR that adds support for it, but I probably won't get around to merging that soon.

If you need import, for now, I would suggest you interact with the operator directly i.e. don't use the chart.

Closes: #424, #425 Signed-off-by: Itay Grudev <[email protected]>

soorajshetty604 · 2025-01-09T13:13:16Z

But this chart does not contain the externalClusters field for distributed topology configuration, which should have the scope to add multiple cluster details under externalClusters and replica section as well. So that we can have primary and disaster recovery data centres in multiple regions as mentioned in the blog https://www.gabrielebartolini.it/articles/2024/06/making-the-world-my-single-point-of-failure-with-postgresql
As of now we can have standalone cluster and recovery cluster from the primary, but switchover between the two clusters in two regions (distributed topology)is not possible using this helm chart.

externalClusters:
- name: pg16-eu-central
barmanObjectStore:
destinationPath: s3://pg16-eu-central/
# ...
- name: pg16-eu-western
barmanObjectStore:
destinationPath: s3://pg16-eu-western/
# ...
replica:
primary: pg16-eu-central
source: pg16-eu-western

The chart is responsible for filling in the externalClusters section based on your recovery configuration. I don't want to expose it.

charts/charts/cluster/templates/_bootstrap.tpl

Lines 43 to 90 in 3c5dd5d

externalClusters:

name: pgBaseBackupSource
connectionParameters:
host: {{ .Values.recovery.pgBaseBackup.source.host | quote }}
port: {{ .Values.recovery.pgBaseBackup.source.port | quote }}
user: {{ .Values.recovery.pgBaseBackup.source.username | quote }}
dbname: {{ .Values.recovery.pgBaseBackup.source.database | quote }}
sslmode: {{ .Values.recovery.pgBaseBackup.source.sslMode | quote }}
{{- if .Values.recovery.pgBaseBackup.source.passwordSecret.name }}
password:
name: {{ default (printf "%s-pg-basebackup-password" (include "cluster.fullname" .)) .Values.recovery.pgBaseBackup.source.passwordSecret.name }}
key: {{ .Values.recovery.pgBaseBackup.source.passwordSecret.key }}
{{- end }}
{{- if .Values.recovery.pgBaseBackup.source.sslKeySecret.name }}
sslKey:
name: {{ .Values.recovery.pgBaseBackup.source.sslKeySecret.name }}
key: {{ .Values.recovery.pgBaseBackup.source.sslKeySecret.key }}
{{- end }}
{{- if .Values.recovery.pgBaseBackup.source.sslCertSecret.name }}
sslCert:
name: {{ .Values.recovery.pgBaseBackup.source.sslCertSecret.name }}
key: {{ .Values.recovery.pgBaseBackup.source.sslCertSecret.key }}
{{- end }}
{{- if .Values.recovery.pgBaseBackup.source.sslRootCertSecret.name }}
sslRootCert:
name: {{ .Values.recovery.pgBaseBackup.source.sslRootCertSecret.name }}
key: {{ .Values.recovery.pgBaseBackup.source.sslRootCertSecret.key }}
{{- end }}

{{- else }}
recovery:
{{- with .Values.recovery.pitrTarget.time }}
recoveryTarget:
targetTime: {{ . }}
{{- end }}
{{- if eq .Values.recovery.method "backup" }}
backup:
name: {{ .Values.recovery.backupName }}
{{- else if eq .Values.recovery.method "object_store" }}
source: objectStoreRecoveryCluster
{{- end }}

externalClusters:

name: objectStoreRecoveryCluster
barmanObjectStore:
serverName: {{ .Values.recovery.clusterName }}
{{- $d := dict "chartFullname" (include "cluster.fullname" .) "scope" .Values.recovery "secretPrefix" "recovery" -}}
{{- include "cluster.barmanObjectStoreConfig" $d | nindent 4 }}
Import is not yet supported. There is a PR that adds support for it, but I probably won't get around to merging that soon.

If you need import, for now, I would suggest you interact with the operator directly i.e. don't use the chart.

itay-grudev · 2025-01-09T21:52:30Z

The chart is supposed to serve as a level of abstraction over the the CNPG CRDs. But I'm accepting PRs for adding replica cluster support to the chart.

It's been on my radar but I am a bit low on time to add it. That being said if someone is willing to add that I'll work with them to assist and prioritize that PR.

soorajshetty604 · 2025-01-30T12:56:56Z

The chart is supposed to serve as a level of abstraction over the the CNPG CRDs. But I'm accepting PRs for adding replica cluster support to the chart.

It's been on my radar but I am a bit low on time to add it. That being said if someone is willing to add that I'll work with them to assist and prioritize that PR.

Yes, Thanks. I’m interested in contributing to the Helm chart by adding support for replica clusters (distributed topology) . I’d love to collaborate on this and get your guidance on the approach.

Could you provide any initial pointers on how you envision this being implemented?

itay-grudev · 2025-01-31T08:21:09Z

@soorajshetty604 It's designed to work with mode=replica.

charts/charts/cluster/values.yaml

Line 28 in 0a6940f

mode: standalone

To follow the chart configuration method you'll want to add a global section for replica configuration.

Here is a link to the CNPG documentation on Replica Clusters.

I am on the CloudNativePG Slack if you have any further questions.

Ofirfr mentioned this issue Oct 25, 2024

support externalClusters https://github.com/cloudnative-pg/charts/issues/424 #425

Open

NiccoloFei added the chart( cluster ) Related to the cluster chart label Oct 31, 2024

itay-grudev added a commit that referenced this issue Dec 19, 2024

feat( cluster ): Adds support for recovery.mode=import

2ef41e9

Closes: #424, #425 Signed-off-by: Itay Grudev <[email protected]>

itay-grudev mentioned this issue Dec 19, 2024

feat( cluster ): Adds support for recovery.mode=import #475

Merged

itay-grudev closed this as completed in #475 Feb 26, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't set `externalClusters` #424

Can't set `externalClusters` #424

Ofirfr commented Oct 25, 2024

itay-grudev commented Oct 25, 2024 •

edited

Loading

soorajshetty604 commented Jan 9, 2025

itay-grudev commented Jan 9, 2025

soorajshetty604 commented Jan 30, 2025 •

edited

Loading

itay-grudev commented Jan 31, 2025

Can't set externalClusters #424

Can't set externalClusters #424

Comments

Ofirfr commented Oct 25, 2024

itay-grudev commented Oct 25, 2024 • edited Loading

soorajshetty604 commented Jan 9, 2025

itay-grudev commented Jan 9, 2025

soorajshetty604 commented Jan 30, 2025 • edited Loading

itay-grudev commented Jan 31, 2025

Can't set `externalClusters` #424

Can't set `externalClusters` #424

itay-grudev commented Oct 25, 2024 •

edited

Loading

soorajshetty604 commented Jan 30, 2025 •

edited

Loading