Skip to content

Commit

Permalink
Improve README, minor config changes
Browse files Browse the repository at this point in the history
  • Loading branch information
double16 committed Jan 5, 2019
1 parent 7cf81da commit 1d52996
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 5 deletions.
3 changes: 1 addition & 2 deletions cruise-control/11cruise-control-config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -352,8 +352,7 @@ data:
clusterConfigs.json: |-
{
"min.insync.replicas": 2,
"an.example.cluster.config": false
"min.insync.replicas": 2
}
log4j2.xml: |-
Expand Down
2 changes: 1 addition & 1 deletion cruise-control/20kafka-broker-reporter-patch.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# meant to be applied using
# kubectl --namespace kafka patch statefulset kafka --patch "$(cat cruise-control/20kafka-broker-reporter-patch.yml )"
# kubectl --namespace kafka patch statefulset kafka --patch "$(cat cruise-control/20kafka-broker-reporter-patch.yml)"
metadata:
name: kafka
namespace: kafka
Expand Down
25 changes: 23 additions & 2 deletions cruise-control/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,26 @@
## Cruise Control

Cruise Control is used to automate the dynamic workload rebalance and self-healing of a Kafka cluster. This tool will allow you to add, replace or remove nodes and the cluster will be automatically adjusted. Partitions will be rebalanced based on resource usage.
Cruise Control is used to automate the dynamic workload rebalance and self-healing of a Kafka cluster. This tool will allow you to add, replace or remove nodes and the cluster will be automatically adjusted. Partitions will be rebalanced based on resource usage of CPU, network, disk, etc.

The default configuration has self healing enabled.
*Disclaimer*: It is important to understand Cruise Control will modify the Kafka cluster without operator intervention. Bugs or misconfiguration may cause loss of data or denial of service. You bear the responsibility of configuring and testing properly and taking precautions based on the importance of your data.

### Configuration

There are several configuration files that need to be mounted in `/opt/cruise-control/config`. The files in `11cruise-control-config.yml` are the defaults from [the Cruise Control GitHub repo, migrate_to_kafka_2_0 branch](https://github.com/linkedin/cruise-control/tree/migrate_to_kafka_2_0/config). The significant modification from the GitHub repo is that self healing has been enabled using `self.healing.enabled=true`.

Following are the files in `11cruise-control-config.yml`. Nearly all changes you would make are in `cruisecontrol.properties`.

- cruisecontrol.properties
- capacityJBOD.json
- capacity.json
- clusterConfigs.json
- log4j2.xml
- log4j.properties

### Patching

Cruise control requires broker metrics to make informed decisions. Each broker runs a metric collector that pushes metrics into a topic, by default named `__CruiseControlMetrics`. Configuring the collector requires patching the broker StatefulSet. An example command to apply this patch is below.

```shell
$ kubectl --namespace kafka patch statefulset kafka --patch "$(cat cruise-control/20kafka-broker-reporter-patch.yml)"
```

0 comments on commit 1d52996

Please sign in to comment.