Centos 7 kubernetes and cassandra installation guide (Docker engine runtime)

NOTE: here is the example how to prepare Kubernetes cluster based on Docker runtime. If you would like to use ContainerD(recomended) please refer to RHEL 7 kubernetes and cassandra installation guide (ContainerD runtime)

---

Kubernetes installation

• yum-utils must be installed

•  yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
  

• cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
  [kubernetes]
  name=Kubernetes
  baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
  enabled=1
  gpgcheck=1
  repo_gpgcheck=1
  gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
  exclude=kubelet kubeadm kubectl
  EOF
  

• Set SELinux in permissive mode (effectively disabling it)

  • setenforce 0

  • sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
    

• yum check-update -y

• yum install -y docker-ce-19.03.15-3.el7

• systemctl enable --now docker

• sudo yum install -y kubelet-1.21.13 kubeadm-1.21.13 kubectl-1.21.13 --disableexcludes=kubernetes

• systemctl enable --now kubelet

• swapoff <my_swap_device_or_file> (also swap should be disabled in /etc/fstab )

• cat <<EOF | sudo tee /etc/docker/daemon.json
  {
    "exec-opts": ["native.cgroupdriver=systemd"],
    "log-driver": "json-file",
    "log-opts": {
      "max-size": "100m"
    },
    "storage-driver": "overlay2"
  }
  EOF
  

• systemctl restart docker

• disable firewall systemctl disable firewalld && systemctl stop firewalld

• or add rules:

# Master
firewall-cmd --permanent --add-port=6443/tcp # Kubernetes API server
firewall-cmd --permanent --add-port=2379-2380/tcp # etcd server client API
firewall-cmd --permanent --add-port=10250/tcp # Kubelet API
firewall-cmd --permanent --add-port=10251/tcp # kube-scheduler
firewall-cmd --permanent --add-port=10252/tcp # kube-controller-manager
firewall-cmd --permanent --add-port=8285/udp # Flannel
firewall-cmd --permanent --add-port=8472/udp # Flannel
firewall-cmd --add-masquerade --permanent
firewall-cmd --permanent --add-port=30000-32767/tcp # if NodePort on master is required
firewall-cmd --reload
systemctl restart firewalld

# Worker
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=8285/udp # Flannel
firewall-cmd --permanent --add-port=8472/udp # Flannel
firewall-cmd --permanent --add-port=30000-32767/tcp # NodePort range
firewall-cmd --add-masquerade --permanent
firewall-cmd --reload
systemctl restart firewalld

• Letting iptables see bridged traffic

  cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
  net.bridge.bridge-nf-call-ip6tables = 1
  net.bridge.bridge-nf-call-iptables = 1
  net.ipv4.ip_forward                 = 1
  EOF
  

• sudo sysctl --system
• kubeadm init --pod-network-cidr=10.244.0.0/16
• mkdir -p <exactpro_user_home_dir>/.kube
• cp -i /etc/kubernetes/admin.conf <exactpro_user_home_dir>/.kube/config
• chown $(id <exactpro_user> -u):$(id <exactpro_user> -g) <exactpro_user_home_dir>/.kube/config
• openssl genrsa -out th2-adm.key 2048
• openssl req -new -key th2-adm.key -out th2-adm.csr -subj "/CN=th2-adm"
• openssl x509 -req -in th2-adm.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out th2-adm.crt -days 500
• mkdir <exactpro_user_home_dir>/.certs
• mv th2-adm.crt th2-adm.key <exactpro_user_home_dir>/.certs/
• chown $(id <exactpro_user> -u):$(id <exactpro_user> -g) <exactpro_user_home_dir>/.certs/*
• kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
• kubectl taint node mymasternode node-role.kubernetes.io/master:NoSchedule-

• cat <<EOF | sudo tee th2-adm_clusterRoleBinding.yaml
  kind: ClusterRoleBinding
  apiVersion: rbac.authorization.k8s.io/v1
  metadata:
    name: th2-adm
    namespace: default
  subjects:
  - kind: User
    name: th2-adm
    apiGroup: rbac.authorization.k8s.io
  roleRef:
    kind: ClusterRole
    name: cluster-admin
    apiGroup: rbac.authorization.k8s.io
  EOF
  

• kubectl apply -f th2-adm_clusterRoleBinding.yaml

Cassandra standalone installation

• sudo yum install java-1.8.0-openjdk

• python 2.7.5 or higher is also required (checking python version: python -V)

• cat <<EOF | sudo tee /etc/yum.repos.d/cassandra.repo
  [cassandra]
  name=Apache Cassandra
  baseurl=https://downloads.apache.org/cassandra/redhat/311x/
  gpgcheck=1
  repo_gpgcheck=1
  gpgkey=https://downloads.apache.org/cassandra/KEYS
  EOF
  

• sudo yum install cassandra -y

• Your /etc/cassandra/conf/cassandra.yaml should also contain the following settings:

  authenticator: PasswordAuthenticator
  authorizer: org.apache.cassandra.auth.CassandraAuthorizer
  auto_bootstrap: true
  cluster_name: test-Universum
  commitlog_directory: "/var/lib/cassandra/commitlog"
  commitlog_sync: periodic
  commitlog_sync_period_in_ms: 10000
  data_file_directories:
  - "/var/lib/cassandra/data"
  endpoint_snitch: GossipingPropertyFileSnitch
  hints_directory: "/var/lib/cassandra/hints"
  listen_interface: <YOUR NETWORK INTERFACE for example eth0>
  num_tokens: 256
  partitioner: org.apache.cassandra.dht.Murmur3Partitioner
  saved_caches_directory: "/var/lib/cassandra/saved_caches"
  read_request_timeout_in_ms: 30000
  range_request_timeout_in_ms: 20000
  seed_provider:
  - class_name: org.apache.cassandra.locator.SimpleSeedProvider
    parameters:
    - seeds: <IP ADDRESS OF THIS NODE>
  start_native_transport: true
  start_rpc: false
  

• sudo service cassandra start

• check cluster status:

  nodetool status
  

  Datacenter: dc1
  ===============
  Status=Up/Down
  |/ State=Normal/Leaving/Joining/Moving
  --  Address      Load       Tokens       Owns (effective)  Host ID                               Rack
  UN  10.50.30.51  253.3 KiB  256          100.0%            51ec0170-17e2-4a54-91f5-9a578d12e54e  rack1
  

• sudo chkconfig cassandra on

Change cassandra superuser password:

ALTER USER cassandra WITH PASSWORD '<password>';

Create user for th2:

CREATE USER th2 WITH PASSWORD '<password>' NOSUPERUSER;
GRANT CREATE PERMISSION ON ALL KEYSPACES TO th2;

Check users:

SELECT * FROM system_auth.roles;