Changelog

Latest Version: 6.4.4

Release History

6.4.4

Fixes

• Elasticsearch and OpenSearch Outputs - Corrected the ECS schema issue where "agent.version" incorrectly displayed as 1.2.3.

6.4.3

No Changes from 6.4.2 to 6.4.3.

6.4.2

Fixes

• Elasticsearch and OpenSearch Outputs - Index templates update to include Geo fields for system. (CODEX) and host. (ECS). These fields are also now included in CODEX to ECS conversion.

Updates

• Enhanced SNMP Get Request Error Management - Improved the handling of errors returned by SNMP Get requests and adjusted how these errors impact device object schedules. This update addresses problems that previously occurred when devices failed to respond, often due to performance bottlenecks. The new implementation effectively resolves these issues. For detailed information, please refer to the section EF_INPUT_SNMP_POLLER_ERROR_HANDLING.

6.4.1

Fixes

• An issue was fixed which could cause the collector to stop polling in some circumstances.
• Rediscovery - Object definitions modified to limit rediscovery to system restarts.
• Fixed an issue which caused the collector to panic when the configuration file was not provided.

6.4.0

No Changes from 6.3.7 to 6.4.0.

6.3.7

Updates

• Packaging - Sign the rpm package using a FIPS-compliant GPG key, and provide a FIPS-compliant GPG public key for package signature verification.

6.3.6

No Changes from 6.3.5 to 6.3.6.

6.3.5

New Features

• Support Bundler - Added endpoint and command-line interface to retrieve a support bundle. Support Bundler will collect logs, configs, and metrics for troubleshooting or analysis. See Generating A Support Bundle for more details.

Updates

• OpenSearch Output - The OpenSearch output will automatically bootstrap the initial write index and add the rollover alias when EF_FLOW_OUTPUT_OPENSEARCH_INDEX_PERIOD is set to rollover. If the ISM policy configured in EF_OUTPUT_OPENSEARCH_INDEX_TEMPLATE_ISM_POLICY (default is elastiflow) is not found in OpenSearch, a default policy will be created which deletes data after 7 days. This policy can be changed later using the OpenSearch Dashboards UI or OpenSearch API.

Fixes

• Metrics - Fixed an issue where the collector could panic due to mishandling the parsing of metrics.
• Outputs - Fixed an issue where the collector would not send data to the outputs.

Deprecations

• **Default value of EF_OUTPUT_OPENSEARCH_INDEX_PERIOD - In a future release, the OpenSearch output's default value for this setting will be changed to rollover. This will enable the use of Index State Management (ISM) to manage the retention of ElastiFlow indices. If you wish to continue to use the old default setting of daily, you should ensure that it is specifically set in your configuration.

6.3.4

Fixes

• Logger - Fixed an issue where the configuration options for logging are not recognized when using YAML for configuration. This resulted in the logs not being written.
• OpenSearch Output and Splunk Output - Fixed an issue which caused auto-scaling of the output worker pool not function properly. This could result in a reduction of throughput unless the pool size was set manually.
• 6.3.3 - Fixed an issue which prevented the collector from running on operating systems based on Debian 11 and earlier (e.g. Ubuntu 20.04).

6.3.3

Updates

• Various security updates based on ElastiFlow's Static Application Security Testing (SAST) and Software Composition Analysis (SCA) scanning processes.
• Logging - Added additional logging for poller, scheduler and other internal processes, to assist with regular operation of the collector.
• Elasticsearch Output - Added the managed and managed_by attributes to the _meta section of the Index Templates. This allows Kibana to indicate that they are managed by an external process (The Unified SNMP Collector) and not user-defined.

6.3.2

No Changes from 6.3.1 to 6.3.2.

6.3.1

Updates

• SNMP Definitions - Updated SNMP Definitions to 1.1 from https://github.com/elastiflow/snmp.

6.3.0

Breaking Changes

• Elasticsearch Output: default option value changes

Beginning with ElastiFlow 6.3.0 the default values for the Elasticsearch output have been changed as follows.

Option	Old Value	New Value
EF_OUTPUT_ELASTICSEARCH_TIMESTAMP_SOURCE	end	collect
EF_OUTPUT_ELASTICSEARCH_INDEX_PERIOD	daily	rollover

• Kafka Output: default option value changes

Beginning with ElastiFlow 6.3.0 the default values for the Kafka output have been changed as follows. Performance testing has shown that this change can improve throughput.

Option	Old Value	New Value
EF_OUTPUT_KAFKA_PRODUCER_COMPRESSION	0 (none)	3 (LZ4)
EF_OUTPUT_KAFKA_PRODUCER_FLUSH_FREQUENCY	1000	500
EF_OUTPUT_KAFKA_FLAT_RECORD_ENABLE	false	true
EF_OUTPUT_KAFKA_TIMESTAMP_SOURCE	end	collect

• OpenSearch Output: default option value changes

Beginning with ElastiFlow 6.3.0 the default values for the OpenSearch output have been changed as follows.

Option	Old Value	New Value
EF_OUTPUT_OPENSEARCH_TIMESTAMP_SOURCE	end	collect

Fixes

• Discovery - Fixes a condition where the SNMP poller can rediscover a device continually.
• Elasticsearch Output - Telemetry index templates are now created with the correct rollover alias.
• HTTP-based Outputs - All HTTP-based outputs now set the Host header, as is required by some environments.

Updates

• Logging - Logs related to failed SNMP operations have been improved to include the operation details.

6.2.2

Fixes

• Definitions - device defaults are now applied to all device definitions within a single YAML file
• Discovery: persistence - discovered object inventory is now properly applied when the collector is restarted
• SNMP Error Handling - fixed a possible panic that can occur when objects are being pruned from the discovered inventory

Updates

• Discovery - device discovery now performs a bulk walk, rather than a simple walk, when SNMP v2c and v3 is configured, reducing the number of polls necessary to complete the discovery process

6.2.1

New Features

• Kafka Output: support for ECS - An option has been added to output records in Elastic Common Schema (ECS).

6.2.0

This is the initial generally available (GA) release of the new ElastiFlow Unified SNMP Collector.

New Features

• API: apply-definitions and rediscover-device - endpoints have been added to trigger the collector to re-read all definitions and take necessary actions (e.g. add/remove a device), and the rediscovery of a devices SNMP object instances.
• Discovery: persistence - Discovered device SNMP object instances can now be persisted to disk. This allow the collector to be restarted without rediscovering all devices.
• Elasticsearch and OpenSearch Output: auto-generated index templates - Index templates for Elasticsearch and OpenSearch are now auto-generated from SNMP object definitions.
• Kafka Output: optional flattened field names - An option has been added to use flattened, rather than nested, field names in the JSON records produced to Kafka.
• API (formerly Metrics) Server - Added support for basic authentication to secure the API's HTTP Server.

Deprecations

• Default value of EF_OUTPUT_ELASTICSEARCH_TIMESTAMP_SOURCE - Beginning with ElastiFlow 6.3.0 the Elasticsearch output's default value for EF_OUTPUT_ELASTICSEARCH_TIMESTAMP_SOURCE will be changed to collect. This will allow the collector to handle a wider variety of situations without additional configuration. If you wish to continue to use the current default setting of end, you should ensure that it is specifically set in your configuration prior to the release and deployment of 6.3.0.
• Default value of EF_OUTPUT_ELASTICSEARCH_INDEX_PERIOD - Beginning with ElastiFlow 6.3.0 the Elasticsearch output's default value for EF_OUTPUT_ELASTICSEARCH_INDEX_PERIOD will be changed to rollover. This will enable the use of Index Lifecycle Management (ILM) to manage retention of ElastiFlow indices. If you wish to continue to use the current default setting of daily, you should ensure that it is specifically set in your configuration prior to the release and deployment of 6.3.0.
• Default value of EF_OUTPUT_OPENSEARCH_TIMESTAMP_SOURCE - Beginning with ElastiFlow 6.3.0 the OpenSearch output's default value for EF_OUTPUT_OPENSEARCH_TIMESTAMP_SOURCE will be changed to collect. This will allow the collector to handle a wider variety of situations without additional configuration. If you wish to continue to use the current default setting of end, you should ensure that it is specifically set in your configuration prior to the release and deployment of 6.3.0.
• Kafka output default values - Performance testing has shown that the current default values can be modified for improved throughput. Beginning with ElastiFlow 6.3.0 the default values of various Kafka output configuration options will be changed as in the table below. If you wish to continue to use the current default settings, you should ensure that it is specifically set in your configuration prior to the release and deployment of 6.3.0.

Option	6.2.x and earlier	planned for 6.3.0
EF_OUTPUT_KAFKA_PRODUCER_COMPRESSION	0 (none)	3 (LZ4)
EF_OUTPUT_KAFKA_PRODUCER_FLUSH_FREQUENCY	500ms	1000ms
EF_OUTPUT_KAFKA_FLAT_RECORD_ENABLE	false	true

Updates

• Logging - logs have been improved for improved structure and readability.
• Elasticsearch Output - Bulk index errors returned from Elasticsearch/OpenSearch are now logged.
• Kafka Output - producer pool has been improved for increased performance.