Prometheus & Grafana

Prerequisites

1. The ElastiFlow Unified Collector is running and the metrics endpoint is reachable. (Reachable @ http://0.0.0.0:8080/metrics)

2. Prometheus is installed & running and can reach the NetObserv Flow Metrics endpoint. (Prometheus default @ http://0.0.0.0:9090)

3. Grafana is installed & running and can reach Prometheus. (Grafana default @ http://0.0.0.0:3000)

ElastiFlow Metrics Endpoint

• Verify the ElastiFlow Unified Collector & Metrics Endpoint have successfully started (logs in /var/log/elastiflow/flowcoll/flowcoll.log):

2022-10-21T08:31:23.078-0700  info  pipeline/appserver.go:44  metrics exposed at http://0.0.0.0:8080/metrics

If another service is already using port 8080, you can change the port on which the collector listens by setting EF_API_PORT.

Grafana Installation Guide

• OS specific Grafana installation guides: https://grafana.com/docs/grafana/latest/setup-grafana/installation/

• Debian/Ubuntu installation guide: https://grafana.com/docs/grafana/latest/setup-grafana/installation/debian/

• RPM-based installation guide: https://grafana.com/docs/grafana/latest/setup-grafana/installation/rpm/

Prometheus Installation Guide

• Main Installation guides: https://prometheus.io/docs/prometheus/latest/installation/

• Precompiled binaries: https://prometheus.io/download/

Prometheus Configuration

1. Once Prometheus is installed edit prometheus.yml and add a job pointing to the ElastiFlow Unified Collector as in the following example:

Example default Prometheus configuration:

# my global config
global:
  scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any time series scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

An ElastiFlow-specific Job:

    job_name: elastiflow-metrics-1
    scrape_interval: 15s
    metrics_path: /metrics

    scheme: http
    static_configs:
      - targets: ['192.168.3.18:8080']

Connect Grafana To Prometheus

1. Once Grafana is up and running; from the "Welcome To Grafana" page click the cogwheel in the bottom left corner, then click "Data Sources", and finally click "Add Data Source":

2. Select Prometheus

3. Name this Prometheus Datasource, type your Prometheus endpoint IP address and port in the "URL" field, and configure the authentication and "scrape interval" you have set up:

1. Click "Save & Test"

Import The ElastiFlow Metrics Dashboard

The ElastiFlow Metrics Dashboard can be found by searching Grafana Labs Dashboards: https://grafana.com/grafana/dashboards/

Direct Link: https://grafana.com/grafana/dashboards/17306-elastiflow-metrics/

Image

Image

1. Import the Dashboard by entering the dashboard ID (17306) found after searching ElastiFlow Metrics in the Grafana Labs Dashboards search by going to Dashboards -> Import and clicking "Load":

1. Accept the default or give the dashboard a name and place it in the folder you'd like it to reside in. After this point to the Prometheus Datasource you previously connected above click "Import" and you are done.

Image