Skip to main content

Output Configuration

Outputs can be figured using a global scope or within a namespace scope. Global allows one instance of an output to be run. Namespace allows for multiple instances of an output to be run.

Global Configuration#

Global configuration only allows you to create a single instance of an output.

For example:

EF_FLOW_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector sends data to a single instance of an output.

2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [default] Output: running

Namespace Configuration#

Namespaces allow multiple instances of an output for flows to be sent.

To define a namespace, include the name of your namespace after EF_FLOW_.

For example:

To namespace a configuration defined as EF_FLOW_OUTPUT_ELASTICSEARCH_ENABLE. Add your namespace between EF_FLOW_ and _OUTPUT_ELASTICSEARCH_ENABLE. As shown below:

EF_FLOW_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true
EF_FLOW_NAMESPACE2_OUTPUT_ELASTICSEARCH_ENABLE=true
EF_FLOW_NAMESPACE3_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector will send data to three(3) separate instances of an output.

2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [NAMESPACE1] Output: running
2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [NAMESPACE2] Output: running
2022-07-07T08:22:29.875-0500 info httpoutput/httpoutput.go:98 Elasticsearch [NAMESPACE3] Output: running
note

The collector must be configured with a global OR a namespace configuration. Both cannot be used simultaneously.

For example:

When an elasticsearch output is configured with both a global and a namespace configuration.

EF_FLOW_OUTPUT_ELASTICSEARCH_ENABLE=true # global configuration
EF_FLOW_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true # namespace configuration

The flow collector will fail to start.

2022-07-07T08:27:39.536-0500 fatal flowcoll/main.go:108 exiting flowcoll because of invalid configuration {"code": "config/invalid", "reason": "both global and namespace environment variables cannot be used simultaneously"}
main.main
/go/src/elastiflow.com/flowcoll/cmd/flowcoll/main.go:108
runtime.main
/usr/local/go/src/runtime/proc.go:255
exit status 1