Skip to content

Commit

Permalink
Focus README on docker-compose; README fixups; reduce docker-compose …
Browse files Browse the repository at this point in the history 
…dep (#7)

= README.md

Move Docker Compose instructions to the top of the types of setup, and
name it "Quick Start with Docker Compose".

Fix bug in instructions to append to /etc/default/telegraf in the bare
metal instructions (use sudo tee).

Added instructions for verifying installation by checking that
nuodb_internal and nuodb databases exist in influx.

Changed Kubernetes section to say "coming soon".

Fix header depth so that each of the Setup types are at the top level.

Fix whitespace.

= docker-compose.yml

Reduce dependency from 3.7 to 3.3, seems to work just fine and is
required for the versions of docker and docker-compose on Ubuntu.
  • Loading branch information
@rshaull
rshaull authored Oct 26, 2020
1 parent fe1503a commit 5939c60
Show file tree
Hide file tree
Showing 2 changed files with 130 additions and 65 deletions.
187 changes: 126 additions & 61 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<img src="images/nuodb.svg" width="200" height="200" />
<img src="images/nuodb.svg" width="200" height="200" />

# NuoDB Collector

Expand Down Expand Up @@ -28,87 +28,75 @@ Built into this container are 4 input plugins to collect metrics from the NuoDB
Collects host machine resource consumption statistics on a regular 10s interval.

# NuoDB Collector Page Outline
[Setup on Bare Metal](#Setup-on-bare-metal)
[Quick Start with Docker Compose](#Quick-start-with-docker-compose)

[Setup in Docker](#Setup-in-docker)
[Setup Manually in Docker](#Setup-in-docker)

[Setup in Kubernetes](#Setup-in-Kubernetes)

[Setup on Bare Metal](#Setup-on-bare-metal)

# Setup on bare metal
[Check Collection Status](#Check-collection-status)

## Installation
# Quick Start with Docker Compose

### 1) Install dependencies
```
pidof - installed - sysvinit-tools package
python2.7
pip
```
For a complete example on how to set up the NuoDB domain with NuoDB collector, you can use `docker compose`.
This repository contains a Docker Compose file (`docker-compose.yml`) which will start:

### 2) Download and install `telegraf`
These steps are for RedHat or CentOS. For other platforms, see [Telegraf Documentation](https://portal.influxdata.com/downloads/).
- 2 Admin Processes
- 1 Storage Manager
- 2 Transaction Engines
- 2 NuoDB Collector containers (1 for SM, 1 for TE)
- InfluxDB database

```
wget https://dl.influxdata.com/telegraf/releases/telegraf-1.15.2-1.x86_64.rpm
sudo yum localinstall telegraf-1.15.2-1.x86_64.rpm
```
Clone the NuoDB Collector repository and `cd` into it:

### 3) Download and install NuoDB Collector
```
git clone [email protected]:nuodb/nuodb-collector.git
cd nuodb-collector
pip install -r requirements.txt
sudo cp -r nuocd /opt/
```

## Configuration
The `conf/nuodb.conf` file in this repository configures all 4 input plugins for NuoDB running on localhost as described in the section above.
The `conf/outputs.conf` file configures an output plugin to a InfluxDB instance defined by the `$INFLUXURL` environmental variable.
Replace the `<hostinflux>` placeholder with the URL of a running InfluxDB instance.
```
sudo cp conf/nuodb.conf /etc/telegraf/telegraf.d
sudo cp conf/outputs.conf /etc/telegraf/telegraf.d
sudo edit /etc/telegraf/telegraf.conf
sudo chown -R telegraf.telegraf /etc/telegraf
sudo cat >> /etc/default/telegraf <<EOF
INFLUXURL=http://<hostinflux>:8086
PYTHONPATH=/opt/nuocd/pylib
EOF
```
Then run `docker-compose up` to start the processes specified in the Docker Compose file:

### Start NuoDB Collector
```
sudo systemctl daemon-reload
sudo systemctl restart telegraf
DOCKER_IMAGE=nuodb/nuodb-ce:latest docker-compose up -d
```

**NOTE:** If not starting telegraf via `systemd` then the variables set in `/etc/default/telegraf` are not picked up automatically.
Instead you can start telegraf with the following command:
Stop processes started with `docker-compose up` by running the following command:

```
sh -c "$(cat /etc/default/telegraf | tr '\n' ' ') telegraf --config /etc/telegraf/telegraf.conf --config-directory /etc/telegraf/telegraf.d"
DOCKER_IMAGE=nuodb/nuodb-ce:latest docker-compose down
```

## Setup in Docker
# Setup Manually in Docker

## Download the NuoDB Collector Docker Image

### Download the NuoDB Collector Docker Image
```
docker pull docker.pkg.github.com/nuodb/nuodb-collector/nuocd:latest
```

### Building docker image from source
## Building docker image from source (Optional)

Instead of pulling a pre-built Docker image, you can build it yourself from source.

```
git clone [email protected]:nuodb/nuodb-collector.git
cd nuodb-collector
docker build .
docker build .
docker tag <SHA> <TAG>
```

### Running Prerequisites
## Running Prerequisites

### NuoDB Domain

As a prerequisite you must have a running NuoDB domain.
To start NuoDB in Docker, follow the [NuoDB Docker Blog Part I](https://nuodb.com/blog/deploy-nuodb-database-docker-containers-part-i).
Following this tutorial will also create the Docker network `nuodb-net`.

After following the [NuoDB Docker Blog Part I](https://nuodb.com/blog/deploy-nuodb-database-docker-containers-part-i) tutorial, verify your domain by running `nuocmd`:

To verify your domain, run `nuocmd`:
```
$ docker exec -it nuoadmin1 nuocmd show domain
server version: 4.1.1-3-2203dab8dd, server license: Community
Expand All @@ -123,8 +111,11 @@ Databases:
[TE] test-te-1/172.20.0.6:48006 [start_id = 1] [server_id = nuoadmin1] [pid = 39] [node_id = 2] [last_ack = 0.14] MONITORED:RUNNING
```

This collector also assumes that an InfluxDB instance is already running.
If you do not already have InfluxDB running, you can start a simple install like so.
### InfluxDB

The collector assumes that an InfluxDB instance is running.
Start InfluxDB with Docker:

```
docker run -d --name influxdb \
--network nuodb-net \
Expand All @@ -133,15 +124,16 @@ docker run -d --name influxdb \
influxdb:latest
```

### Running NuoDB Collector
## Running NuoDB Collector

Each NuoDB Collector runs colocated with a NuoDB engine in the same process namespace.
As such, you must start a NuoDB collector docker container for every running NuoDB engine you want to monitor.

The following value replacement must be done to start a NuoDB Collector container:
- Replace the `<hostname>` with the hostname of the monitored engine container. The hostnames must match. In our example it will be `test-sm-1`
- Replace the `<hostinflux>` placeholder with the URL of a running InfluxDB container. In our example, it will be `influxdb`.
- Replace the `<nuoadmin>` placeholder with the URL of a running NuoDB admin container. In our example, it will be `nuoadmin1`.
- Replace the `<enginecontainer>` placeholder with the URL of a running NuoDB Engine container. In our example, it will be `test-sm-1`.
- Replace the `<hostname>` with the hostname of the monitored engine container. The hostnames must match. In our example it will be `test-sm-1`

```
docker run -d --name nuocd-sm \
Expand All @@ -155,19 +147,92 @@ docker run -d --name nuocd-sm \

Repeat the steps above for all running NuoDB engine containers you want to monitor.

### Docker Example
For a complete example on how to set up the NuoDB domain with NuoDB collector, you can use `docker compose`.
The following command will start:
- 2 Admin Processes
- 1 Storage Manager
- 2 Transaction Engines
- 2 NuoDB Collector containers (1 for SM, 1 for TE)
- InfluxDB time-series database
# Setup in Kubernetes

NuoDB Helm Chart support for the NuoDB Collector coming soon.

# Setup on Bare Metal

## Installation

### 1) Install dependencies

```
DOCKER_IMAGE=nuodb/nuodb-ce:latest docker-compose up -d
pidof
sysvinit-tools
python2.7
pip
```

## Setup in Kubernetes
### 2) Download and install `telegraf`

Follow the documentation in the [NuoDB Helm Charts](https://github.com/nuodb/nuodb-helm-charts) repository.
These steps are for RedHat or CentOS. For other platforms, see [Telegraf Documentation](https://portal.influxdata.com/downloads/).

```
wget https://dl.influxdata.com/telegraf/releases/telegraf-1.15.2-1.x86_64.rpm
sudo yum localinstall telegraf-1.15.2-1.x86_64.rpm
```

### 3) Download and install NuoDB Collector

```
git clone [email protected]:nuodb/nuodb-collector.git
cd nuodb-collector
pip install -r requirements.txt
sudo cp -r nuocd /opt/
```

## Configuration

The `conf/nuodb.conf` file in this repository configures all 4 input plugins for NuoDB running on localhost as described in the section above.
The `conf/outputs.conf` file configures an output plugin to a InfluxDB instance defined by the `$INFLUXURL` environment variable.
Replace the `<hostinflux>` placeholder with the hostname for a running InfluxDB instance.
```
sudo cp conf/nuodb.conf /etc/telegraf/telegraf.d
sudo cp conf/outputs.conf /etc/telegraf/telegraf.d
sudo edit /etc/telegraf/telegraf.conf
sudo chown -R telegraf.telegraf /etc/telegraf
cat <<EOF | sudo tee -a /etc/default/telegraf >/dev/null
INFLUXURL=http://<hostinflux>:8086
PYTHONPATH=/opt/nuocd/pylib
EOF
```

## Start NuoDB Collector

```
sudo systemctl daemon-reload
sudo systemctl restart telegraf
```

**NOTE:** If not starting telegraf via `systemd` then the variables set in `/etc/default/telegraf` are not picked up automatically.
Instead you can start telegraf with the following command:
```
sh -c "$(cat /etc/default/telegraf | tr '\n' ' ') telegraf --config /etc/telegraf/telegraf.conf --config-directory /etc/telegraf/telegraf.d"
```


# Check Collection Status

After starting collection, your InfluxDB instance should contain the `nuodb_internal` and `nuodb` databases.
You can check if `nuodb_internal` and `nuodb` exist by launching the InfluxDB CLI:

```
influx
```

Then, from the InfluxDB CLI, run the `SHOW DATABASES` command and check that `nuodb_internal` exists (it may take a minute for the database to be created):

```
$ influx
Connected to http://localhost:8086 version 1.8.3
InfluxDB shell version: 1.8.3
> show databases
name: databases
name
----
_internal
telegraf
nuodb_internal
nuodb
```
8 changes: 4 additions & 4 deletions docker-compose.yml
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
version: "3.7"
version: "3.3"
services:
nuoadmin1:
image: $DOCKER_IMAGE
Expand All @@ -11,15 +11,15 @@ services:
hostname: sm
environment:
NUOCMD_API_SERVER: https://nuoadmin1:8888
depends_on:
depends_on:
- nuoadmin1
command: ["nuodocker", "start", "sm", "--db-name", "hockey", "--server-id", "nuoadmin1", "--dba-user", "dba", "--dba-password", "goalie"]
te:
image: $DOCKER_IMAGE
hostname: te
environment:
NUOCMD_API_SERVER: https://nuoadmin1:8888
depends_on:
depends_on:
- nuoadmin1
- sm
command: ["nuodocker", "start", "te", "--db-name", "hockey", "--server-id", "nuoadmin1"]
Expand Down Expand Up @@ -50,4 +50,4 @@ services:
INFLUXURL: http://influxdb:8086
NUOCMD_API_SERVER: https://nuoadmin1:8888
NUOCD_HOSTNAME: te
pid: 'service:te'
pid: 'service:te'

0 comments on commit 5939c60

Please sign in to comment.