Skip to content

Commit

Permalink
docs,hack: support manual repair the Invalid Pod radondb#331
Browse files Browse the repository at this point in the history
  • Loading branch information
acekingke committed Dec 13, 2021
1 parent b478be2 commit ad5ee9c
Showing 2 changed files with 55 additions and 0 deletions.
14 changes: 14 additions & 0 deletions docs/rebuild.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Why need rebuild ?
RadonDB cluster is semisynchronous replication mysql cluster. Because MySQL Semisynchronous Replication, It has a chance that the slave has more data than master, So when the xenon check it, It will lable the pod INVALID. When it happend, You
need rebuild the INVALID pod.

# How to use ?
Before you want to rebuild the pod, you need to manually check the security and consistency of the cluster.

```shell
./hack/rebuild.sh PODNAME
```
**for example**
```shell
./hack/rebuild.sh sample-mysql-2
```
41 changes: 41 additions & 0 deletions hack/rebuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#!/bin/bash
## This script is used to rebuild the pod.
## useage : ./rebuild.sh pod_name
function check_pod_status(){
POD_STATUS=$(kubectl get pods $POD_NAME -o jsonpath='{.status.phase}')
if [ "$POD_STATUS" == "Running" ]; then
echo "Pod $POD_NAME is running."
else
echo "Pod $POD_NAME is not running."
exit 1
fi
}
function rebuild_pod(){
INVALID=$(kubectl exec -it $POD_NAME -c xenon -- xenoncli raft status | grep "INVALID")
if [ -z $INVALID ]; then
echo "Pod in Raft is not in invalid state, please check your cluster status."
exit 1
fi

kubectl exec -it $POD_NAME -c mysql -- rm -rf /var/lib/mysql/*

sleep 5
kubectl delete pod $POD_NAME
echo "Rebuild mysql success."
}

if [ "$#" -ne 1 ]; then
echo "Usage: $0 POD_NAME." >&2
exit 1
fi
POD_NAME=$1
echo "Rebuilding is danagerous job, Before doing this, please make sure your cluster data is safe and consistent."
while true; do
read -p "Do you want to do it (y/n)?" yn
case $yn in
[Yy]* ) check_pod_status; rebuild_pod;break;;
[Nn]* ) exit;;
* ) echo "Please answer yes or no.";;
esac
done

0 comments on commit ad5ee9c

Please sign in to comment.