Kafka

This page outlines the required steps to connect ElastiFlow to Kafka.

In the ElastiFlow configuration /etc/elastiflow/flowcoll.yml you need to configure the following lines

To view all configuration options please refer to:

https://www.elastiflow.com/docs/config_ref/common/output_kafka/#overview

Specify the IP for the Kafka broker

EF_OUTPUT_KAFKA_BROKERS: 10.101.2.179:9093

If you want to export data from ElastiFlow in Elasticsearch Common Schema enable the following. If this is not set field names will default to the ElastiFlow codex schema.

EF_OUTPUT_KAFKA_ECS_ENABLE: "true"

Enable Kafka output

EF_OUTPUT_KAFKA_ENABLE: "true"

If you want to flatten the JSON record set this to true

EF_OUTPUT_KAFKA_FLAT_RECORD_ENABLE: "true"

Set the Kafka topic name

EF_OUTPUT_KAFKA_TOPIC

Enable TLS or MTLS

EF_OUTPUT_KAFKA_TLS_KEY_FILEPATH: "/etc/elastiflow/certs/flowcoll.key" EF_OUTPUT_KAFKA_TLS_SKIP_VERIFICATION: "false"

EF_OUTPUT_KAFKA_TLS_CA_CERT_FILEPATH: "/etc/elastiflow/certs/ca.crt" EF_OUTPUT_KAFKA_TLS_CERT_FILEPATH: "/etc/elastiflow/certs/flowcoll.crt" EF_OUTPUT_KAFKA_TLS_ENABLE: "true"