Skip to content
This repository has been archived by the owner on Jan 19, 2022. It is now read-only.

Basic multi-cluster support #104

Merged
merged 3 commits into from
Oct 27, 2020

Conversation

libby
Copy link
Contributor

@libby libby commented Oct 27, 2020

Initial support for running a quorum k8s network across multiple clusters.

To run a multi-cluster Quorum k8s network, a quorum network must initially be started in a single cluster and the genesis.json needs to be shared with all other cluster that wish to connect.

There is a new (optional) entry in the qubernetes config external-nodes, e.g.

external_nodes:
  - Node_UserIdent:  quorum-node1
    Tm_Url:  192.168.64.49:30853
    Enode_Url: "enode://e6ccd013b90e17ea7692905609f2a8a7922d855ebcb4476f1afc6aa4ad4261a2af40df9f1b6f4c6a83c96fd40bbad4d63352047d890a278d6cd410a225a1a236@192.168.64.49:31500?discport=0&raftport=50401"
    Node_Acct_Addr: "0xC7868aA6C261aAaDA3E370315B6CF752056F54d1"
  - Node_UserIdent:  quorum-node2
    Tm_Url:  192.168.64.49:31680

This is the information that a cluster needs to connect to another cluster. There is a qctl command to obtain this info from inside a cluster, e.g. qctl ls nodes --asexternal -b --node-ip=$(minikube ip)

If cluster A (C-A) wishes to add nodes from cluster B (C-B), C-A must:

  1. Obtain its running node information as an external node, e.g. qctl ls nodes --asexternal -b --node-ip=$(minikube ip) and give this it C-A.
  2. C-A must add C-B's external_node info to its qubernetes config and generate the appropriate resources, qctl generate network --update.
  3. Once C-A has been deploy, C-A must do the same: obtain the external_nodes info for its running nodes, and share that with C-B.
  4. When C-B has updated its qubernetes config with C-A external_nodes and redeployed, the nodes should be able to connect.
  • note: currently only supporting NodePort so the external_node info must be obtained from a running cluster, and note the NodePort may change if the node goes down.
  • todo: Ingress support should be implemented next, so that the external_node info will not change if a node goes down and is restarted.

libby added 3 commits October 27, 2020 02:14
* update qctl to support displaying nodes for inclusion in a separate
cluster:
qctl ls nodes --asexternal -b --node-ip=$(minikube ip)
basic support for multi-cluster quorum k8s networks, adding
external-node config support which will regenerate the necessary config
files (permissioned-nodes.json, tessera config, etc.).

the external node infomation for a cluster can be obtained via
qctl ls nodes --asexternal -b --node-ip=$(minikube ip)

tested across namespaces minikube.

currently have to copy external nodes over manually to the config file
this will be automated by qctl at a later date.

a quorum k8s initial cluster must be started before a new spanning
cluster can be added, they must share the same genesis.json.

currently only supporting NodePort URL.
This enables direct copy and paste of necessary external node info
 to qubernetes config.
@libby libby changed the title Basic multi cluster support Basic multi-cluster support Oct 27, 2020
@libby libby merged commit 222227a into Consensys:master Oct 27, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant