Prometheus & Grafana
This page explains how to setup Prometheus and Grafana to monitor the ElastiFlow Unified Collector.
Prerequisites
- The ElastiFlow Unified Collector is running and the metrics endpoint is reachable. (Reachable @ http://0.0.0.0:8080/metrics)
- Prometheus is installed & running and can reach the NetObserv Flow Metrics endpoint. (Prometheus default @ http://0.0.0.0:9090)
- Grafana is installed & running and can reach Prometheus. (Grafana default @ http://0.0.0.0:3000)
ElastiFlow Metrics Endpoint
- Verify the ElastiFlow Unified Collector & Metrics Endpoint have successfully started (logs in /var/log/elastiflow/flowcoll/flowcoll.log):
2022-10-21T08:31:23.078-0700 info pipeline/appserver.go:44 metrics exposed at http://0.0.0.0:8080/metrics
tip
If another service is already using port 8080
, you can change the port on which the collector listens by setting EF_API_PORT
.
Grafana Installation Guide
- OS specific Grafana installation guides: https://grafana.com/docs/grafana/latest/setup-grafana/installation/
- Debian/Ubuntu installation guide: https://grafana.com/docs/grafana/latest/setup-grafana/installation/debian/
- RPM-based installation guide: https://grafana.com/docs/grafana/latest/setup-grafana/installation/rpm/
Prometheus Installation Guide
- Main Installation guides: https://prometheus.io/docs/prometheus/latest/installation/
- Precompiled binaries: https://prometheus.io/download/
Prometheus Configuration
- Once Prometheus is installed edit
prometheus.yml
and add a job pointing to the ElastiFlow Unified Collector as in the following example:
Example default Prometheus configuration:
# my global config
global:
scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
# - alertmanager:9093
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first_rules.yml"
# - "second_rules.yml"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any time series scraped from this config.
- job_name: "prometheus"
# metrics_path defaults to '/metrics'
# scheme defaults to 'http'.
static_configs:
- targets: ["localhost:9090"]
An ElastiFlow-specific Job:
job_name: elastiflow-metrics-1
scrape_interval: 15s
metrics_path: /metrics
scheme: http
static_configs:
- targets: ['192.168.3.18:8080']
Connect Grafana To Prometheus
- Once Grafana is up and running; from the "Welcome To Grafana" page click the cogwheel in the bottom left corner, then click "Data Sources", and finally click "Add Data Source":
- Select Prometheus
- Name this Prometheus Datasource, type your Prometheus endpoint IP address and port in the "URL" field, and configure the authentication and "scrape interval" you have set up: Click "Save & Test"
Import The ElastiFlow Metrics Dashboard
The ElastiFlow Metrics Dashboard can be found by searching Grafana Labs Dashboards: https://grafana.com/grafana/dashboards/
Direct Link: https://grafana.com/grafana/dashboards/17306-elastiflow-metrics/
- Import the Dashboard by entering the dashboard ID (17306) found after searching ElastiFlow Metrics in the Grafana Labs Dashboards search by going to Dashboards -> Import and clicking "Load":
2. Accept the default or give the dashboard a name and place it in the folder you'd like it to reside in. After this point to the Prometheus Datasource you previously connected above click "Import" and you are done.