Unable to change satellite ip address

[paraddise]

I have kubernetes node with 2 interfaces, 1 for use with kubernets, another is infiniband interface for drbd traffic only.
I have external linstor controller and piraes deployed to kubernetes.
Currently piraeus when creating satellite in controller takes ip from status.nodeIPs of Pod.
So I can't change IP address of registered satellite.
I propose to create field in LinstorSatelliteConfiguration where you can pass interface name or IP address of sattelite that will be created.

[paraddise]

Found workaround

1. You: Adding new node to kubernets
2. Operator: creates satellite in controller
3. You: Adding new interface ib0 via `linstor n i create ib0 ...
4. You: Changing LinstorSatelliteConfiguration and adding PrefNic=ib0
5. Operator: updates Node Properties in Linstor Controller

Now, I'm searching the way to automate this at node registration at linstor.

Maybe there are plans to add Interfaces in LinstorSatelliteConfiguration?

[WanzenBug]

Currently, the Operator is the one registering the Satellites. So it can only really use what Kubernetes exposes on the Pod (perhaps also on the K8s Node resource). So we would need a way to expose the IP address of this secondary interface to Kubernetes, and then a way for the Operator to pick up this information.

You could probably somehow coble together an initContainer using the LinstorSatelliteConfiguration.spec.podTemplate that scans the output of ip addr show dev ... and registers that interface on Pod startup.

[paraddise]

I haven't thought about patches, thank you, will try it

[paraddise]

Wrote this workaround

apiVersion: piraeus.io/v1
kind: LinstorSatelliteConfiguration
metadata:
  name: register-infiniband
spec:
  podTemplate:
    spec:
      containers:
        - name: register-indiniband
          command:
          - bash
          - -c
          - |
            echo "Checking that node exists in linstor"
            while true; do
              current_node=`linstor -m n list | jq --arg HOSTNAME $HOSTNAME '.[][] | select(.name==$HOSTNAME)'`
              if [[ -z "$current_node" ]]; then
                echo "Node not registered yet, waiting"
              else
                echo "Node exists"
                break
              fi
              sleep 10
            done

            echo "Start reconcile of ib0 interface, with interval 180s"
            while true; do
              IB_ADDR=`ip --json a show dev ib0 | jq -r '.[0].addr_info[] | select(.family=="inet").local'`
              echo "Check interface ib0 exists"
              interface=`echo $current_node | jq '.net_interfaces[]| select(.name=="ib0")'`

              if [[ -z "$interface" ]]; then
                echo "Creating interface ib0 with address $IB_ADDR"
                linstor n interface create "$HOSTNAME" ib0 $IB_ADDR --port 3366 --communication-type plain --active
                exit 0;
              else
                echo "Check interface in a right state"

                if [[ "$(echo $interface | jq -r '.address')" != "$IB_ADDR" ]]; then
                  echo "Address differs, changing"
                  linstor n interface modify $HOSTNAME ib0 --ip "$IB_ADDR"
                fi

                # We will ignore active state and use PrefNic parameter
                # if [[ "$(echo $interface | jq -r '.is_active')" != "true" ]]; then
                #   echo "Interface not active, enabling"
                #   linstor n interface modify $HOSTNAME ib0 --active
                # fi
              fi

              echo "Check node PrefNic is ib0"
              if [[ "$(echo $current_node | jq -r '.props.PrefNic')" != "ib0" ]]; then
                echo "Update node PrefNic property, set to ib0"
                linstor n set-property $HOSTNAME PrefNic ib0
              fi

              current_node=`linstor -m n list | jq --arg HOSTNAME $HOSTNAME '.[][] | select(.name==$HOSTNAME)'`
              sleep 180
            done
          image: quay.io/piraeusdatastore/piraeus-server:v1.29.2
          securityContext:
            capabilities:
              drop:
              - ALL
            readOnlyRootFilesystem: true
          env:
            - name: LS_CONTROLLERS
              value: {{ .Values.externalControllerUrl | quote }} # paste here your controller endpoint
---
# for external linstor controller set hostNetwork and chacne dnsPolicy to have ability resolve pod and service names in kubernetes.
apiVersion: piraeus.io/v1
kind: LinstorSatelliteConfiguration
metadata:
  name: host-network
spec:
  podTemplate:
    spec:
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
  properties:
    - name: PrefNic
      value: "ib0"
---