Docker

A Docker container for NetObserv SNMP is available on Docker Hub. docker-compose is a good way to run the container. It allows for the various environment variables, used to configure the collector, to be easily managed in one place without having to enter them on the command line.

docker-compose.yml

The following docker-compose.yml file provides a starting point that can be further customized for your environment and needs.

version: '3'
services:
# ElastiFlow NetObserv SNMP
snmp-collector:
  image: elastiflow/snmp-collector:undefined
  container_name: snmp-collector
  restart: 'unless-stopped'
  volumes:
    - /etc/elastiflow:/etc/elastiflow
  environment:
    EF_LICENSE_ACCEPTED: 'false'
    #EF_ACCOUNT_ID: ''

    #EF_SNMP_LICENSE_KEY: ''
    #EF_SNMP_LICENSED_UNITS:

    #EF_INSTANCE_NAME: default
    #EF_API_PORT: 8080
    #EF_API_TLS_ENABLE: ''
    #EF_API_TLS_CERT_FILEPATH: ''
    #EF_API_TLS_KEY_FILEPATH: ''
    #EF_API_BASIC_AUTH_ENABLE: 'false'
    #EF_API_BASIC_AUTH_USERNAME: ''
    #EF_API_BASIC_AUTH_PASSWORD: ''

    #EF_LOGGER_LEVEL: 'info'
    #EF_LOGGER_ENCODING: 'json'
    #EF_LOGGER_FILE_LOG_ENABLE: 'false'
    #EF_LOGGER_FILE_LOG_FILENAME: '/var/log/elastiflow/flowcoll/flowcoll.log'
    #EF_LOGGER_FILE_LOG_MAX_SIZE: 100
    #EF_LOGGER_FILE_LOG_MAX_AGE: ''
    #EF_LOGGER_FILE_LOG_MAX_BACKUPS: 4
    #EF_LOGGER_FILE_LOG_COMPRESS: 'false'

    #EF_INPUT_SNMP_POLLER_WORKER_POOL_SIZE:  # defaults to the number of CPU threads * 4
    #EF_INPUT_SNMP_POLLER_ERROR_HANDLING: 'partial'
    #EF_INPUT_SNMP_DEVICE_DEFINITIONS_DIRECTORY_PATH: '/etc/elastiflow/snmp/devices'
    #EF_INPUT_SNMP_DEVICE_GROUP_DEFINITIONS_DIRECTORY_PATH: '/etc/elastiflow/snmp/device_groups'
    #EF_INPUT_SNMP_OBJECT_GROUP_DEFINITIONS_DIRECTORY_PATH: '/etc/elastiflow/snmp/object_groups'
    #EF_INPUT_SNMP_OBJECT_DEFINITIONS_DIRECTORY_PATH: '/etc/elastiflow/snmp/objects'
    #EF_INPUT_SNMP_PERSIST_ENABLE: 'true'
    #EF_INPUT_SNMP_PERSIST_DIRECTORY_PATH: '/usr/share/elastiflow/snmpcoll'

    #EF_PROCESSOR_SNMP_ENUM_DEFINITIONS_DIRECTORY_PATH: '/etc/elastiflow/snmp/enums'
    #EF_PROCESSOR_POOL_SIZE:
    #EF_PROCESSOR_TRANSLATE_KEEP_IDS: 'default'

    #EF_PROCESSOR_ENRICH_IPADDR_TTL: 7200

    EF_PROCESSOR_ENRICH_IPADDR_METADATA_ENABLE: 'false'
    #EF_PROCESSOR_ENRICH_IPADDR_METADATA_USERDEF_PATH: '/etc/elastiflow/metadata/ipaddrs.yml'
    #EF_PROCESSOR_ENRICH_IPADDR_METADATA_REFRESH_RATE: 15

    EF_PROCESSOR_ENRICH_IPADDR_DNS_ENABLE: 'false'
    EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_IP: ''
    EF_PROCESSOR_ENRICH_IPADDR_DNS_NAMESERVER_TIMEOUT: 3000
    #EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PRIVATE: 'true'
    #EF_PROCESSOR_ENRICH_IPADDR_DNS_RESOLVE_PUBLIC: 'true'
    #EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_PATH: '/etc/elastiflow/hostname/user_defined.yml'
    #EF_PROCESSOR_ENRICH_IPADDR_DNS_USERDEF_REFRESH_RATE: 15
    #EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_PATH: '/etc/elastiflow/hostname/incl_excl.yml'
    #EF_PROCESSOR_ENRICH_IPADDR_DNS_INCLEXCL_REFRESH_RATE: 15

    #EF_PROCESSOR_DURATION_PRECISION: 'ms'
    #EF_PROCESSOR_TIMESTAMP_PRECISION: 'ms'
    #EF_PROCESSOR_PERCENT_NORM: 100
    #EF_PROCESSOR_KEEP_CPU_TICKS: 'false'

    #EF_PROCESSOR_DROP_FIELDS: ''

    # stdout
    #EF_OUTPUT_STDOUT_ENABLE: 'false'
    #EF_OUTPUT_STDOUT_FORMAT: 'json_pretty'

    # monitor
    #EF_OUTPUT_MONITOR_ENABLE: 'false'
    #EF_OUTPUT_MONITOR_INTERVAL: 300

    # Elasticsearch
    EF_OUTPUT_ELASTICSEARCH_ENABLE: 'false'
    EF_OUTPUT_ELASTICSEARCH_ECS_ENABLE: 'false'
    #EF_OUTPUT_ELASTICSEARCH_BATCH_DEADLINE: 2000
    #EF_OUTPUT_ELASTICSEARCH_BATCH_MAX_BYTES: 8388608
    #EF_OUTPUT_ELASTICSEARCH_TIMESTAMP_SOURCE: 'collect'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_PERIOD: 'rollover'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_SUFFIX: ''
    #EF_OUTPUT_ELASTICSEARCH_DROP_FIELDS: ''
    #EF_OUTPUT_ELASTICSEARCH_ALLOWED_RECORD_TYPES: 'as_path_hop,flow_option,flow,telemetry'

    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_ENABLE: 'true'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_OVERWRITE: 'true'
    EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_SHARDS: 1
    EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_REPLICAS: 0
    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_REFRESH_INTERVAL: '10s'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_CODEC: 'best_compression'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_ILM_LIFECYCLE: 'elastiflow'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_PIPELINE_DEFAULT: '_none'
    #EF_OUTPUT_ELASTICSEARCH_INDEX_TEMPLATE_PIPELINE_FINAL: '_none'

    # A comma separated list of Elasticsearch nodes to use. DO NOT include "http://" or "https://"
    EF_OUTPUT_ELASTICSEARCH_ADDRESSES: '127.0.0.1:9200'
    EF_OUTPUT_ELASTICSEARCH_USERNAME: 'elastic'
    EF_OUTPUT_ELASTICSEARCH_PASSWORD: 'changeme'
    #EF_OUTPUT_ELASTICSEARCH_CLOUD_ID: ''
    #EF_OUTPUT_ELASTICSEARCH_API_KEY: ''
    #EF_OUTPUT_ELASTICSEARCH_CLIENT_CA_CERT_FILEPATH:
    #EF_OUTPUT_ELASTICSEARCH_CLIENT_CERT_FILEPATH:
    #EF_OUTPUT_ELASTICSEARCH_CLIENT_KEY_FILEPATH:

    EF_OUTPUT_ELASTICSEARCH_TLS_ENABLE: 'false'
    EF_OUTPUT_ELASTICSEARCH_TLS_SKIP_VERIFICATION: 'false'
    EF_OUTPUT_ELASTICSEARCH_TLS_CA_CERT_FILEPATH: ''

    #EF_OUTPUT_ELASTICSEARCH_RETRY_ENABLE: 'true'
    #EF_OUTPUT_ELASTICSEARCH_RETRY_ON_TIMEOUT_ENABLE: 'true'
    #EF_OUTPUT_ELASTICSEARCH_MAX_RETRIES: 3
    #EF_OUTPUT_ELASTICSEARCH_RETRY_BACKOFF: 1000

    # OpenSearch
    EF_OUTPUT_OPENSEARCH_ENABLE: 'false'
    EF_OUTPUT_OPENSEARCH_ECS_ENABLE: 'false'
    #EF_OUTPUT_OPENSEARCH_BATCH_DEADLINE: 2000
    #EF_OUTPUT_OPENSEARCH_BATCH_MAX_BYTES: 8388608
    #EF_OUTPUT_OPENSEARCH_TIMESTAMP_SOURCE: 'collect'
    #EF_OUTPUT_OPENSEARCH_INDEX_PERIOD: 'daily'
    #EF_OUTPUT_OPENSEARCH_INDEX_SUFFIX: ''
    #EF_OUTPUT_OPENSEARCH_DROP_FIELDS: ''
    #EF_OUTPUT_OPENSEARCH_ALLOWED_RECORD_TYPES: 'as_path_hop,flow_option,flow,telemetry'

    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ENABLE: 'true'
    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_OVERWRITE: 'true'
    EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_SHARDS: 1
    EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_REPLICAS: 0
    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_REFRESH_INTERVAL: '10s'
    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_CODEC: 'best_compression'
    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ISM_POLICY: 'elastiflow'
    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_PIPELINE_DEFAULT: '_none'
    #EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_PIPELINE_FINAL: '_none'

    # A comma separated list of OpenSearch nodes to use. DO NOT include "http://" or "https://"
    EF_OUTPUT_OPENSEARCH_ADDRESSES: '127.0.0.1:9200'
    EF_OUTPUT_OPENSEARCH_USERNAME: 'elastic'
    EF_OUTPUT_OPENSEARCH_PASSWORD: 'changeme'
    #EF_OUTPUT_OPENSEARCH_CLIENT_CA_CERT_FILEPATH:
    #EF_OUTPUT_OPENSEARCH_CLIENT_CERT_FILEPATH:
    #EF_OUTPUT_OPENSEARCH_CLIENT_KEY_FILEPATH:

    EF_OUTPUT_OPENSEARCH_TLS_ENABLE: 'false'
    EF_OUTPUT_OPENSEARCH_TLS_SKIP_VERIFICATION: 'false'
    EF_OUTPUT_OPENSEARCH_TLS_CA_CERT_FILEPATH: ''

    #EF_OUTPUT_OPENSEARCH_RETRY_ENABLE: 'true'
    #EF_OUTPUT_OPENSEARCH_RETRY_ON_TIMEOUT_ENABLE: 'true'
    #EF_OUTPUT_OPENSEARCH_MAX_RETRIES: 3
    #EF_OUTPUT_OPENSEARCH_RETRY_BACKOFF: 1000

    # Splunk
    EF_OUTPUT_SPLUNK_HEC_ENABLE: 'false'
    #EF_OUTPUT_SPLUNK_HEC_CIM_ENABLE: 'false'
    EF_OUTPUT_SPLUNK_HEC_ADDRESSES: '127.0.0.1:8088'
    EF_OUTPUT_SPLUNK_HEC_TOKEN: ''
    #EF_OUTPUT_SPLUNK_HEC_BATCH_MAX_BYTES: 8388608
    #EF_OUTPUT_SPLUNK_HEC_BATCH_DEADLINE: 2000
    #EF_OUTPUT_SPLUNK_HEC_TLS_ENABLE: 'true'
    #EF_OUTPUT_SPLUNK_HEC_TLS_SKIP_VERIFICATION: 'false'
    #EF_OUTPUT_SPLUNK_HEC_TLS_CA_CERT_FILEPATH: ''
    #EF_OUTPUT_SPLUNK_HEC_DROP_FIELDS: ''

    # Kafka
    EF_OUTPUT_KAFKA_ENABLE: 'false'
    EF_OUTPUT_KAFKA_BROKERS: ''
    #EF_OUTPUT_KAFKA_VERSION: '1.0.0'
    #EF_OUTPUT_KAFKA_TOPIC: 'elastiflow-flow-codex'
    #EF_OUTPUT_KAFKA_PARTITION_KEY: 'flow.export.ip.addr'
    #EF_OUTPUT_KAFKA_CLIENT_ID: 'elastiflow-flowcoll'
    #EF_OUTPUT_KAFKA_RACK_ID: ''
    #EF_OUTPUT_KAFKA_TIMEOUT: 30
    #EF_OUTPUT_KAFKA_DROP_FIELDS: ''
    #EF_OUTPUT_KAFKA_ALLOWED_RECORD_TYPES: 'as_path_hop,flow_option,flow,telemetry'
    #EF_OUTPUT_KAFKA_FLAT_RECORD_ENABLE: 'true'

    EF_OUTPUT_KAFKA_SASL_ENABLE: 'false'
    #EF_OUTPUT_KAFKA_SASL_USERNAME: ''
    #EF_OUTPUT_KAFKA_SASL_PASSWORD: ''

    #EF_OUTPUT_KAFKA_TLS_ENABLE: 'false'
    #EF_OUTPUT_KAFKA_TLS_CA_CERT_FILEPATH: ''
    #EF_OUTPUT_KAFKA_TLS_CERT_FILEPATH: ''
    #EF_OUTPUT_KAFKA_TLS_KEY_FILEPATH: ''
    #EF_OUTPUT_KAFKA_TLS_SKIP_VERIFICATION: 'false'

    #EF_OUTPUT_KAFKA_PRODUCER_MAX_MESSAGE_BYTES: 1000000
    #EF_OUTPUT_KAFKA_PRODUCER_REQUIRED_ACKS: 1
    #EF_OUTPUT_KAFKA_PRODUCER_TIMEOUT: 10
    #EF_OUTPUT_KAFKA_PRODUCER_COMPRESSION: 3
    #EF_OUTPUT_KAFKA_PRODUCER_COMPRESSION_LEVEL: -1000
    #EF_OUTPUT_KAFKA_PRODUCER_FLUSH_BYTES: 1000000
    #EF_OUTPUT_KAFKA_PRODUCER_FLUSH_MESSAGES: 1024
    #EF_OUTPUT_KAFKA_PRODUCER_FLUSH_FREQUENCY: 1000
    #EF_OUTPUT_KAFKA_PRODUCER_FLUSH_MAX_MESSAGES: 0
    #EF_OUTPUT_KAFKA_PRODUCER_RETRY_MAX: 3
    #EF_OUTPUT_KAFKA_PRODUCER_RETRY_BACKOFF: 100

    # Cribl
    EF_OUTPUT_CRIBL_ENABLE: 'false'
    EF_OUTPUT_CRIBL_ADDRESSES: '127.0.0.1:10080'
    EF_OUTPUT_CRIBL_TOKEN: ''
    #EF_OUTPUT_CRIBL_BATCH_DEADLINE: 2000
    #EF_OUTPUT_CRIBL_BATCH_MAX_BYTES: 8388608
    #EF_OUTPUT_CRIBL_TLS_ENABLE: 'false'
    #EF_OUTPUT_CRIBL_TLS_SKIP_VERIFICATION: 'false'
    #EF_OUTPUT_CRIBL_TLS_CA_CERT_FILEPATH: ''
    #EF_OUTPUT_CRIBL_DROP_FIELDS: ''

    # Generic HTTP
    EF_OUTPUT_GENERIC_HTTP_ENABLE: 'false'
    EF_OUTPUT_GENERIC_HTTP_ECS_ENABLE: 'false'
    #EF_OUTPUT_GENERIC_HTTP_BATCH_DEADLINE: 2000
    #EF_OUTPUT_GENERIC_HTTP_BATCH_MAX_BYTES: 8388608
    EF_OUTPUT_GENERIC_HTTP_ADDRESSES: ''
    #EF_OUTPUT_GENERIC_HTTP_USERNAME: ''
    #EF_OUTPUT_GENERIC_HTTP_PASSWORD: ''
    #EF_OUTPUT_GENERIC_HTTP_TLS_ENABLE: 'false'
    #EF_OUTPUT_GENERIC_HTTP_TLS_SKIP_VERIFICATION: 'false'
    #EF_OUTPUT_GENERIC_HTTP_TLS_CA_CERT_FILEPATH: ''
    #EF_OUTPUT_GENERIC_HTTP_DROP_FIELDS: ''
    #EF_OUTPUT_GENERIC_HTTP_TIMESTAMP_SOURCE: 'collect'

image

The name of the current released image is elastiflow/snmp-collector:.

restart

restart is set to unless-stopped so that the collector will restart automatically if it fails for some reason.

volumes

There are a few scenarios where it is necessary to make files on the host file system available to the collector.

In the example above, /etc/elastiflow on the host's filesystem is mapped into the same path within the container. It is recommend to place the SNMP poller definition files in /etc/elastiflow/snmp.

note

It is also possible to build a new container, adding additional files as needed. This may the best choice if running the container in a dynamically orchestrated environment (e.g. running in Kubernetes). However for an instance dedicated to a specific host, using bind mounted volumes can be very convenient.

environment variables

NetObserv SNMP is configured using environment variables.

For a complete reference of all configuration options please refer to the Configuration Reference.

Running the Container

After completing configuration of the collector in the docker-compose.yml file, you can start the container using one of the following commands...

From within the same path as the docker-compose.yml file:

docker-compose up -d

From a path different from the location of the docker-compose.yml file:

docker-compose -f /PATH/TO/docker-compose.yml up -d

To view the logs written by the container run:

docker logs -f NAME_OF_CONTAINER

To stop the container run:

docker-compose down

or:

docker-compose -f /PATH/TO/docker-compose.yml down