Skip to main content
Version: 5.6

Decoder

EF_FLOW_DECODER_POOL_SIZE​

Specifies the number of flow packet decoders to start. You will need at least one (1) decoder for every 2000 records/second. Increasing the number of decoders will allow the collector to better handle a high volume of high latency enrichment tasks such as DNS lookups for IP addresses and SNMP polls for network interfaces.

note

While increasing the number of decoders can be beneficial, there are diminishing returns at higher decoder counts. This is especially true when the number of decoders exceeds the number of available CPU threads (real cores + SMT threads) or vCPUs. If you require more than 64 decoders it may be more beneficial to use multiple collector instances.

  • Default
    • 4 * EF_FLOW_LICENSED_UNITS

EF_FLOW_DECODER_SETTINGS_PATH​

The path where any files used by the collector's decoder functions are loacted.

  • Default
    • /etc/elastiflow

EF_FLOW_DECODER_IPFIX_ENABLE​

Set to true to enable decoding of IPFIX records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_NETFLOW1_ENABLE​

Set to true to enable decoding of Netflow v1 records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_NETFLOW5_ENABLE​

Set to true to enable decoding of Netflow v5 records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_NETFLOW6_ENABLE​

Set to true to enable decoding of Netflow v6 records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_NETFLOW7_ENABLE​

Set to true to enable decoding of Netflow v7 records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_NETFLOW9_ENABLE​

Set to true to enable decoding of Netflow v9 records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_SFLOW5_ENABLE​

Set to true to enable decoding of sFlow v5 records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_SFLOW_FLOWS_ENABLE​

Set to true to enable decoding of sFlow flow_sample and flow_sample_expanded records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_SFLOW_FLOWS_KEEP_SAMPLES​

When set to true, the packet data from an sFlow sampled_header record will be stored in l2.section.sample as a hex-encoded string.

  • Valid Values
    • true, false
  • Default
    • false

EF_FLOW_DECODER_SFLOW_COUNTERS_ENABLE​

Set to true to enable decoding of sFlow counters_sample and counters_sample_expanded records.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_TRANSLATE_KEEP_IDS​

Specifies which identifier values will be included in the final dataset.

  • Valid Values
    • none - All identifiers are removed from the final dataset.
    • default - Most identifiers are removed from the final dataset. However some identifiers which are required for common use-cases (e.g. raw protocol port values) are included.
    • all - All identifiers are included in the final dataset.
  • Default
    • default

EF_FLOW_DECODER_ENRICH_ASN_PREF​

If enrichment with autonomous system attributes is enabled, but the autonomous system is already indicated directly in the flow record data, this setting specifies which source is prefered. If the preferred source is not available for a given record, the decoder will fall-back to the alternate option.

  • Valid Values
    • lookup - prefer the autonomous system determined by lookup.
    • flow - prefer the autonomous system indicated directly in the flow record data.
  • Default
    • lookup

EF_FLOW_DECODER_ENRICH_JOIN_ASN​

Some features require that related values from separate fields are stored as an array in a single field. Such a "join" of autonomous system related fields is enabled when this setting is true.

info

If records are being output to Elasticsearch this setting should be set to true.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_ENRICH_JOIN_GEOIP​

Some features require that related values from separate fields are stored as an array in a single field. Such a "join" of GeoIP related fields is enabled when this setting is true.

info

If records are being output to Elasticsearch this setting should be set to true.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_ENRICH_JOIN_NETATTR​

Some features require that related values from separate fields are stored as an array in a single field. Such a "join" of network attribute related fields is enabled when this setting is true.

info

If records are being output to Elasticsearch this setting should be set to true.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_ENRICH_JOIN_SUBNETATTR​

Some features require that related values from separate fields are stored as an array in a single field. Such a "join" of IP subnetwork attribute related fields is enabled when this setting is true.

info

If records are being output to Elasticsearch this setting should be set to true.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_ENRICH_JOIN_SEC​

Some features require that related values from separate fields are stored as an array in a single field. Such a "join" of security attribute related fields is enabled when this setting is true.

info

If records are being output to Elasticsearch this setting should be set to true.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_DURATION_PRECISION​

The desired precision of duration-related values. Values received at a different precision than specified will be converted to the desired precision.

  • Valid Values
    • sec - seconds
    • ds - deciseconds
    • cs - centiseconds
    • ms - millseconds
    • us - microseconds
    • ns - nanoseconds
  • Default
    • ms
tip

For most data sources this should millseconds (ms)

EF_FLOW_DECODER_TIMESTAMP_PRECISION​

The desired precision of timestamp values. Values received at a different precision than specified will be converted to the desired precision.

  • Valid Values
    • sec - seconds
    • ds - deciseconds
    • cs - centiseconds
    • ms - millseconds
    • us - microseconds
    • ns - nanoseconds
  • Default
    • ms
tip

For most data stores, e.g. Elasticsearch, this should millseconds (ms)

EF_FLOW_DECODER_PERCENT_NORM​

The desired representation of percentages. Values received with a different representation than specified will be converted to the desired representation.

  • Valid Values
    • 1 - values will be based on a scale of 0-1.
    • 100 - values will be based on a scale of 0-100.
  • Default
    • 100

EF_FLOW_DECODER_ENRICH_EXPAND_CLISRV​

The collector will infer the client/server relationship of two source/destination endpoints. The is setting determines whether such inferrence is enabled or not.

  • Valid Values
    • true, false
  • Default
    • true

EF_FLOW_DECODER_ENRICH_KEEP_CPU_TICKS​

For telemetry sources which provide CPU usage as timeticks, utilization percentages will be calculated. If this setting is set false the timetick values will be removed from the final dataset. If true they will be kept, in addition to the utilization values.

  • Valid Values
    • true, false
  • Default
    • false

EF_FLOW_RECORD_STREAM_MAX_SIZE​

Processed records are queued prior to being processed by an available output instance. This value specifies the size of the queue as a quantity of records. As a single PDU typically contains multiple flow records, this value will typically be a multiple of EF_FLOW_SERVER_UDP_PACKET_STREAM_MAX_SIZE.

  • Default
    • 16 * EF_FLOW_SERVER_UDP_PACKET_STREAM_MAX_SIZE