Output Configuration

Outputs can be figured using a global scope or within a namespaced scope. Global scope allows for one instance of an output. Namespaced scope allows for multiple instances of an output.

Global Configuration

Global configuration only allows you to create a single instance of an output.

For example:

EF_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector sends data to a single instance of an output.

2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [default] Output: running

Namespace Configuration

Namespaces allow multiple instances of an output for flows to be sent.

To define a namespace, include the name of your namespace after EF_.

For example:

To namespace a configuration defined as EF_OUTPUT_ELASTICSEARCH_ENABLE. Add your namespace between EF_ and _OUTPUT_ELASTICSEARCH_ENABLE. As shown below:

EF_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true
EF_NAMESPACE2_OUTPUT_ELASTICSEARCH_ENABLE=true
EF_NAMESPACE3_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector will send data to three(3) separate instances of an output.

2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [NAMESPACE1] Output: running
2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [NAMESPACE2] Output: running
2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [NAMESPACE3] Output: running

note

The collector must be configured with a global OR a namespaced configuration. Both cannot be used simultaneously.

For example:

When an Elasticsearch output is configured with both a global and a namespace configuration.

EF_OUTPUT_ELASTICSEARCH_ENABLE=true # global configuration
EF_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true # namespace configuration

The flow collector will fail to start.

2022-07-07T08:27:39.536-0500    fatal   flowcoll/main.go:108    exiting flowcoll because of invalid configuration       {"code": "config/invalid", "reason": "both global and namespace environment variables cannot be used simultaneously"}
main.main
        /go/src/elastiflow.com/flowcoll/cmd/flowcoll/main.go:108
runtime.main
        /usr/local/go/src/runtime/proc.go:255
exit status 1