Output Configuration

Outputs can be figured using a global scope or within a namespace scope. Global allows one instance of an output to be run. Namespace allows for multiple instances of an output to be run.

Global Configuration

Global configuration only allows you to create a single instance of an output.

For example:

EF_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector sends data to a single instance of an output.

2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [default] Output: running

Namespace Configuration

Namespaces allow multiple instances of an output for flows to be sent.

To define a namespace, include the name of your namespace after EF_.

For example:

To namespace a configuration defined as EF_OUTPUT_ELASTICSEARCH_ENABLE. Add your namespace between EF_ and _OUTPUT_ELASTICSEARCH_ENABLE. As shown below:

EF_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true
EF_NAMESPACE2_OUTPUT_ELASTICSEARCH_ENABLE=true
EF_NAMESPACE3_OUTPUT_ELASTICSEARCH_ENABLE=true

The collector will send data to three(3) separate instances of an output.

2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [NAMESPACE1] Output: running
2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [NAMESPACE2] Output: running
2022-07-07T08:22:29.875-0500    info    httpoutput/httpoutput.go:98     Elasticsearch [NAMESPACE3] Output: running

note

The collector must be configured with a global OR a namespace configuration. Both cannot be used simultaneously.

For example:

When an elasticsearch output is configured with both a global and a namespace configuration.

EF_OUTPUT_ELASTICSEARCH_ENABLE=true # global configuration
EF_NAMESPACE1_OUTPUT_ELASTICSEARCH_ENABLE=true # namespace configuration

The flow collector will fail to start.

2022-07-07T08:27:39.536-0500    fatal   flowcoll/main.go:108    exiting flowcoll because of invalid configuration       {"code": "config/invalid", "reason": "both global and namespace environment variables cannot be used simultaneously"}
main.main
        /go/src/elastiflow.com/flowcoll/cmd/flowcoll/main.go:108
runtime.main
        /usr/local/go/src/runtime/proc.go:255
exit status 1