diff --git a/README.md b/README.md index 2182432..8b8e29b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ - + # NuoDB Collector @@ -28,87 +28,75 @@ Built into this container are 4 input plugins to collect metrics from the NuoDB Collects host machine resource consumption statistics on a regular 10s interval. # NuoDB Collector Page Outline -[Setup on Bare Metal](#Setup-on-bare-metal) +[Quick Start with Docker Compose](#Quick-start-with-docker-compose) -[Setup in Docker](#Setup-in-docker) +[Setup Manually in Docker](#Setup-in-docker) [Setup in Kubernetes](#Setup-in-Kubernetes) +[Setup on Bare Metal](#Setup-on-bare-metal) -# Setup on bare metal +[Check Collection Status](#Check-collection-status) -## Installation +# Quick Start with Docker Compose -### 1) Install dependencies -``` -pidof - installed - sysvinit-tools package -python2.7 -pip -``` +For a complete example on how to set up the NuoDB domain with NuoDB collector, you can use `docker compose`. +This repository contains a Docker Compose file (`docker-compose.yml`) which will start: -### 2) Download and install `telegraf` -These steps are for RedHat or CentOS. For other platforms, see [Telegraf Documentation](https://portal.influxdata.com/downloads/). +- 2 Admin Processes +- 1 Storage Manager +- 2 Transaction Engines +- 2 NuoDB Collector containers (1 for SM, 1 for TE) +- InfluxDB database -``` -wget https://dl.influxdata.com/telegraf/releases/telegraf-1.15.2-1.x86_64.rpm -sudo yum localinstall telegraf-1.15.2-1.x86_64.rpm -``` +Clone the NuoDB Collector repository and `cd` into it: -### 3) Download and install NuoDB Collector ``` git clone git@github.com:nuodb/nuodb-collector.git cd nuodb-collector -pip install -r requirements.txt -sudo cp -r nuocd /opt/ ``` -## Configuration -The `conf/nuodb.conf` file in this repository configures all 4 input plugins for NuoDB running on localhost as described in the section above. -The `conf/outputs.conf` file configures an output plugin to a InfluxDB instance defined by the `$INFLUXURL` environmental variable. -Replace the `` placeholder with the URL of a running InfluxDB instance. -``` -sudo cp conf/nuodb.conf /etc/telegraf/telegraf.d -sudo cp conf/outputs.conf /etc/telegraf/telegraf.d -sudo edit /etc/telegraf/telegraf.conf -sudo chown -R telegraf.telegraf /etc/telegraf -sudo cat >> /etc/default/telegraf <:8086 -PYTHONPATH=/opt/nuocd/pylib -EOF -``` +Then run `docker-compose up` to start the processes specified in the Docker Compose file: -### Start NuoDB Collector ``` -sudo systemctl daemon-reload -sudo systemctl restart telegraf +DOCKER_IMAGE=nuodb/nuodb-ce:latest docker-compose up -d ``` -**NOTE:** If not starting telegraf via `systemd` then the variables set in `/etc/default/telegraf` are not picked up automatically. -Instead you can start telegraf with the following command: +Stop processes started with `docker-compose up` by running the following command: + ``` -sh -c "$(cat /etc/default/telegraf | tr '\n' ' ') telegraf --config /etc/telegraf/telegraf.conf --config-directory /etc/telegraf/telegraf.d" +DOCKER_IMAGE=nuodb/nuodb-ce:latest docker-compose down ``` -## Setup in Docker +# Setup Manually in Docker + +## Download the NuoDB Collector Docker Image -### Download the NuoDB Collector Docker Image ``` docker pull docker.pkg.github.com/nuodb/nuodb-collector/nuocd:latest ``` -### Building docker image from source +## Building docker image from source (Optional) + +Instead of pulling a pre-built Docker image, you can build it yourself from source. + ``` git clone git@github.com:nuodb/nuodb-collector.git cd nuodb-collector -docker build . +docker build . docker tag ``` -### Running Prerequisites +## Running Prerequisites + +### NuoDB Domain + As a prerequisite you must have a running NuoDB domain. To start NuoDB in Docker, follow the [NuoDB Docker Blog Part I](https://nuodb.com/blog/deploy-nuodb-database-docker-containers-part-i). +Following this tutorial will also create the Docker network `nuodb-net`. + +After following the [NuoDB Docker Blog Part I](https://nuodb.com/blog/deploy-nuodb-database-docker-containers-part-i) tutorial, verify your domain by running `nuocmd`: -To verify your domain, run `nuocmd`: ``` $ docker exec -it nuoadmin1 nuocmd show domain server version: 4.1.1-3-2203dab8dd, server license: Community @@ -123,8 +111,11 @@ Databases: [TE] test-te-1/172.20.0.6:48006 [start_id = 1] [server_id = nuoadmin1] [pid = 39] [node_id = 2] [last_ack = 0.14] MONITORED:RUNNING ``` -This collector also assumes that an InfluxDB instance is already running. -If you do not already have InfluxDB running, you can start a simple install like so. +### InfluxDB + +The collector assumes that an InfluxDB instance is running. +Start InfluxDB with Docker: + ``` docker run -d --name influxdb \ --network nuodb-net \ @@ -133,15 +124,16 @@ docker run -d --name influxdb \ influxdb:latest ``` -### Running NuoDB Collector +## Running NuoDB Collector + Each NuoDB Collector runs colocated with a NuoDB engine in the same process namespace. As such, you must start a NuoDB collector docker container for every running NuoDB engine you want to monitor. The following value replacement must be done to start a NuoDB Collector container: +- Replace the `` with the hostname of the monitored engine container. The hostnames must match. In our example it will be `test-sm-1` - Replace the `` placeholder with the URL of a running InfluxDB container. In our example, it will be `influxdb`. - Replace the `` placeholder with the URL of a running NuoDB admin container. In our example, it will be `nuoadmin1`. - Replace the `` placeholder with the URL of a running NuoDB Engine container. In our example, it will be `test-sm-1`. -- Replace the `` with the hostname of the monitored engine container. The hostnames must match. In our example it will be `test-sm-1` ``` docker run -d --name nuocd-sm \ @@ -155,19 +147,92 @@ docker run -d --name nuocd-sm \ Repeat the steps above for all running NuoDB engine containers you want to monitor. -### Docker Example -For a complete example on how to set up the NuoDB domain with NuoDB collector, you can use `docker compose`. -The following command will start: -- 2 Admin Processes -- 1 Storage Manager -- 2 Transaction Engines -- 2 NuoDB Collector containers (1 for SM, 1 for TE) -- InfluxDB time-series database +# Setup in Kubernetes + +NuoDB Helm Chart support for the NuoDB Collector coming soon. + +# Setup on Bare Metal + +## Installation + +### 1) Install dependencies ``` -DOCKER_IMAGE=nuodb/nuodb-ce:latest docker-compose up -d +pidof +sysvinit-tools +python2.7 +pip ``` -## Setup in Kubernetes +### 2) Download and install `telegraf` -Follow the documentation in the [NuoDB Helm Charts](https://github.com/nuodb/nuodb-helm-charts) repository. +These steps are for RedHat or CentOS. For other platforms, see [Telegraf Documentation](https://portal.influxdata.com/downloads/). + +``` +wget https://dl.influxdata.com/telegraf/releases/telegraf-1.15.2-1.x86_64.rpm +sudo yum localinstall telegraf-1.15.2-1.x86_64.rpm +``` + +### 3) Download and install NuoDB Collector + +``` +git clone git@github.com:nuodb/nuodb-collector.git +cd nuodb-collector +pip install -r requirements.txt +sudo cp -r nuocd /opt/ +``` + +## Configuration + +The `conf/nuodb.conf` file in this repository configures all 4 input plugins for NuoDB running on localhost as described in the section above. +The `conf/outputs.conf` file configures an output plugin to a InfluxDB instance defined by the `$INFLUXURL` environment variable. +Replace the `` placeholder with the hostname for a running InfluxDB instance. +``` +sudo cp conf/nuodb.conf /etc/telegraf/telegraf.d +sudo cp conf/outputs.conf /etc/telegraf/telegraf.d +sudo edit /etc/telegraf/telegraf.conf +sudo chown -R telegraf.telegraf /etc/telegraf +cat </dev/null +INFLUXURL=http://:8086 +PYTHONPATH=/opt/nuocd/pylib +EOF +``` + +## Start NuoDB Collector + +``` +sudo systemctl daemon-reload +sudo systemctl restart telegraf +``` + +**NOTE:** If not starting telegraf via `systemd` then the variables set in `/etc/default/telegraf` are not picked up automatically. +Instead you can start telegraf with the following command: +``` +sh -c "$(cat /etc/default/telegraf | tr '\n' ' ') telegraf --config /etc/telegraf/telegraf.conf --config-directory /etc/telegraf/telegraf.d" +``` + + +# Check Collection Status + +After starting collection, your InfluxDB instance should contain the `nuodb_internal` and `nuodb` databases. +You can check if `nuodb_internal` and `nuodb` exist by launching the InfluxDB CLI: + +``` +influx +``` + +Then, from the InfluxDB CLI, run the `SHOW DATABASES` command and check that `nuodb_internal` exists (it may take a minute for the database to be created): + +``` +$ influx +Connected to http://localhost:8086 version 1.8.3 +InfluxDB shell version: 1.8.3 +> show databases +name: databases +name +---- +_internal +telegraf +nuodb_internal +nuodb +``` diff --git a/docker-compose.yml b/docker-compose.yml index 57fd9dc..8b5a62c 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,4 +1,4 @@ -version: "3.7" +version: "3.3" services: nuoadmin1: image: $DOCKER_IMAGE @@ -11,7 +11,7 @@ services: hostname: sm environment: NUOCMD_API_SERVER: https://nuoadmin1:8888 - depends_on: + depends_on: - nuoadmin1 command: ["nuodocker", "start", "sm", "--db-name", "hockey", "--server-id", "nuoadmin1", "--dba-user", "dba", "--dba-password", "goalie"] te: @@ -19,7 +19,7 @@ services: hostname: te environment: NUOCMD_API_SERVER: https://nuoadmin1:8888 - depends_on: + depends_on: - nuoadmin1 - sm command: ["nuodocker", "start", "te", "--db-name", "hockey", "--server-id", "nuoadmin1"] @@ -50,4 +50,4 @@ services: INFLUXURL: http://influxdb:8086 NUOCMD_API_SERVER: https://nuoadmin1:8888 NUOCD_HOSTNAME: te - pid: 'service:te' \ No newline at end of file + pid: 'service:te'