Output Configuration

Outputs can be figured using a global scope or within a namespace scope. Global allows one instance of an output to be run. Namespace allows for multiple instances of an output to be run.

Global Configuration

Global configuration only allows you to create a single instance of an output.

For example:

EF_FLOW_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector sends data to a single instance of an output.

2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [default] Output: running

Namespace Configuration

Namespaces allow multiple instances of an output for flows to be sent.

To define a namespace, include the name of your namespace after EF_FLOW_.

For example:

To namespace a configuration defined as EF_FLOW_OUTPUT_ELASTICSEARCH_ENABLE. Add your namespace between EF_FLOW_ and _OUTPUT_ELASTICSEARCH_ENABLE. As shown below:

EF_FLOW_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true

EF_FLOW_NAMESPACE2_OUTPUT_ELASTICSEARCH_ENABLE=true

EF_FLOW_NAMESPACE3_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector will send data to three(3) separate instances of an output.

2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [NAMESPACE1] Output: running

2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [NAMESPACE2] Output: running

2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [NAMESPACE3] Output: running

note

The collector must be configured with a global OR a namespace configuration. Both cannot be used simultaneously.

For example:

When an elasticsearch output is configured with both a global and a namespace configuration.

EF_FLOW_OUTPUT_ELASTICSEARCH_ENABLE=true # global configuration

EF_FLOW_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true # namespace configuration

The flow collector will fail to start.

2022-07-07T08:27:39.536-0500 fatal flowcoll/main.go:108 exiting flowcoll because of invalid configuration {"code": "config/invalid", "reason": "both global and namespace environment variables cannot be used simultaneously"}

main.main

/go/src/elastiflow.com/flowcoll/cmd/flowcoll/main.go:108

runtime.main

/usr/local/go/src/runtime/proc.go:255

exit status 1