Standalone Monasca for demo purposes with Docker Compose

[pgillich]

Running Monasca standalone for demo purposes requires different configuration, comparing to automatic tests. It's an initial version.

We have several alternatives for the future:
a) maintain this repo parallel with the automatic test configuration
b) merging this repo with with the automatic test configuration (https://github.com/hpcloud-mon/monasca-docker/blob/master/docker-compose.yml) and maintaining configuration for several use cases (automatic test, demo, etc) by combining several alternate compose files.

I feel b) is better, but I'm not sure, because I don't know the long-term aims of monasca-docker.

[timothyb89]

I definitely like the idea of having multiple docker-compose configurations available, and fully separating out the init containers is an improvement over the "

We're mainly using monasca-docker as a DevStack replacement and in CI/CD, so I don't think any of our needs conflict with what you're trying to do here. Is there any reason these changes couldn't be applied to the main docker-compose.yml to avoid duplication between the "main" and standalone environments?

[timothyb89]

Any reason not to use monasca/grafana here? It installs keystone auth and the monasca datasource plugin automatically (and can be configured with grafana-init)

[pgillich]

Not too strong reason, see my long answer.

[timothyb89]

As a side note, we are planning to move away from :latest and :master tags as soon as we can, the main blocker on that right now is that a lot of required patches haven't yet been merged upstream. Once those are merged and included in some stable release we'd like to move to proper release versions everywhere.

[pgillich]

I had zero competence with docker-compose 3 weeks ago. So I decided the most safe and most independent alternative: using own directory as a sandbox. It has a few disadvantages, for example parent directory must be added to the paths (service.build.context, service.volumes), but more flexible. In the future, when the compose file structure will be worked out, this directory may be thrown out. I don't know yet, which alternative will be the best. Without trying, we won't know.

My goal is creating a Monasca delivery which is easy to install on an already installed OpenStack (not DevStack) in order to make demo, prototyping or custom characteristic measurement. In the past, I tried a VM for Monasca server, but I would like to try the Docker way, too. The easy install means harmless as possible on an installed OpenStack. Practically it means, Monasca server (Kafka+API+InfluxDB+etc.) runs in Docker containers (on a CIC or out of OpenStack), Monasca Agent runs in Python virtenv. The Keystone is a good question: I'm not sure, using different Keystone for Monasca is a good idea, but currently it's not important for me yet, because I'd like to know more deeply the Monasca, Kafka and InfluxDB protocols/schemas.

I have already tried both of official Grafana and monasca/grafana. Official Grafana already has Monasca datasource plugin, meanwhile monasca/grafana is a branch on a Grafana fork. I'd like to see the differences in the future, currently the official Grafana is enough me, because I'm using it only to understand the concept InfluxDB schema. This 2 different Grafana is a good chance to figure out, how the Compose configuration can handle 2 or more different distributions for same thing (and TCP port: only one of them can run).

There is another chance for playing different Compose configuration: compose-sidecar.yml, which set monasca service dependency to monasca-sidecar. Prometheus is also interesting for me, but I don't know when I can spend time for it.

I'm not sure, Compose file version 3 is required. But it's compatible to swarm mode. Who knows...

There are several open questions. I'm not sure, I follow the right way, but let's see it. Please take this Pull request as a first draft of an idea. The current state of this request is not good enough for merging to the master. If you mind, you can pick good idea to the master Compose configuration.

But anyway, I has some findings:

• It's possible to list more Compose configuration file in the Compose CLI (-f/--file). It's good, but maybe not enough: include directive in a master Compose configuration would be better, instead of making the set of managed services by a list of Compose configuration file in each CLI command.
• It's possible to set the env file(s), but only in the Compose configuration file (env_file), it's not possible in CLI. It's not enough flexible, if the tags and versions are defined in env files and we would like to make stable/testing/development images with different tags and versions. Currently, this kind of info should be set in environment variables before executing CLI commands.

[matrixik]

@timothyb89 How do you see way forward with this PR?