093_prometheus.md

Monitoring a Node with Prometheus

Latest documentation has been moved to https://docs.cardano.org/en/latest/ .

1. In contrast to EKG, Prometheus allows monitoring a Cardano Node running on an AWS instance without the need for port forwarding. We do however need to configure the AWS firewall to open the right port. If we want to use port 12789, we have to add a new inbound rule to the launch wizard of our browser.

   You can find the link in the Instances dashboard of the AWS console

   

   Click "Edit inbound rules".

   

   Then add a new rule for "Custom TCP", port range 12789, source "Anywhere".

   

2. On the AWS instance we edit the node configuration file shelley_testnet-config.json providing the host and port:

    hasPrometheus:
      - "0.0.0.0"
      - 12789
   

   (Using 0.0.0.0 as host will bind to all provided interfaces, all of which you can list with ifconfig. You can be more selective if you want and provide a specific IP-address instead.)

3. We restart the node, and it will now make Prometheus metrics available at port 12789 (or whatever port you specified in shelley_testnet-config.json).

4. You can get the best out of prometheus if you have both Prometheus on your local machine or monitoring server and Prometheus Node Exporter on your node's server. How to do this depends on your platform and setup, but you can find documentation:

   • Prometheus.

   • Node exporter

5. Prometheus needs to be configured to monitor your Cardano Node. A minimalistic configuration file doing this could look like this:

    global:
      scrape_interval:     15s
      external_labels:
        monitor: 'codelab-monitor'
   
    scrape_configs:
      - job_name: 'cardano' # To scrape data from the cardano node
        scrape_interval: 5s
        static_configs:
          - targets: ['a.b.c.d:12789']
      - job_name: 'node' # To scrape data from a node exporter to monitor your linux host metrics.
        scrape_interval: 5s
        static_configs:
          - targets: ['a.b.c.d:9100']
   

   You have to replace a.b.c.d with the public IP-address of your Cardano Node server, which you can find on the dashboard under IPv4 Public IP.

6. Start Prometheus on your local machine or monitoring server with this configuration, for example:

   $ ./prometheus --config.file=prometheus.yml

7. Start Prometheus node exporter on your Cardano node server, for example:

   $ ./node_exporter

8. On your browser open a.b.c.d:9090, pick one or more interesting metrics to graph and enjoy!

   

**NOTE: Security configurations you should perform on your monitoring server are out of scope for this tutorial.