Latest Version: 5.1.1
- sFlow sample_header decoding now supports PPPoE headers.
- Logic for detecting and logging bad packets has been improved. In many scenarios the guilty PDU will be logged as well, to simplify troubleshooting and support.
- DNS reverse lookup for exporter IPs now works when when only exporter lookups are enabled.
- Juniper flows with truncated records no longer cause a panic.
- If you are receiving flow records from Citrix Netscaler environments there have been a few minor updates to the netscaler-specific fields, which may cause field type conflicts. Users with an active support agreement should contact firstname.lastname@example.org to discuss migration options prior to upgrading to
- The RiskIQ integration to enrich flow records with threat details and autonomous system attributes is now generally available, and can be used in large scale production environments.
- The ability to configure
index.lifecycle.rollover_aliashas been added for the Elasticsearch output, when it is used with Elastic's X-Pack ILM rollover features. The configuration option is
- Support for additional IEs from the Antrea Kubernetes CNI have been added, and is current for Antrea v1.0.0.
- Support for additional IEs from Citrix Netscaler has been added. This is the start of a larger Citrix Netscaler update.
- The default log file interval (
EF_FLOW_LOGGER_FILE_LOG_INTERVAL) has been changed to
daily. You may need to remove all existing log files from the log directory.
- Added the initial release of session establishment inference which populates the field
l4.session.establishedsession established field.
- Jobs for Elastic's Machine Learning features are now provided in the
- License details are now logged when the collector starts.
- Added option to install via RPM package.
- Fixed a scenario where flow.locality can be private and should be public.
- Fix an issue with the AppID cache that could cause a panic.
- Initiator/Responder bytes and packets value will now populate flow.bytes and flow.packets when in/out bytes and packets are not available (fixes as issue with flow records from Cisco ASA).
- SNMP enrichment of interface attributes now supports the polled device responding with a different source IP.
- While the documentation explained that the RiskIQ output must be enabled whenever the enricher is enabled, this is now enforced properly at runtime. If
EF_FLOW_OUTPUT_RISKIQ_ENABLEmust also be
EF_FLOW_OUTPUT_RISKIQ_CUSTOMER_ENCRYPTION_KEYmust also be set. If you were previously using the RiskIQ enrichment feature without the RiskIQ output, you must enable and configure the output, or disable enrichment.
- Added the option EF_FLOW_DECODER_ENRICH_RISKIQ_CACHE_FAILED_REQS which when set to
true(the default) improves performance when the RiskIQ service is unavailable.
- Default value of EF_FLOW_DECODER_ENRICH_RISKIQ_API_TIMEOUT changed from
- Corrected handling of signed integers from sFlow counters.
- When interface attributes are learned from option data, and no ifName value is provide, fallback to ifDescr now works correctly.
- The GA version of the collector now enforces the license. If no license key is used, the collector will run as a Community tier subscription. A Basic tier license can be requested from the ElastiFlow website, as we a 30-day trial of the full Premium tier features. The beta collector will no longer run after 31 March 2021. All users who requested access to the beta will be proactively sent a Basic tier license to the email address which they provided.
- Fixed an issue with sFlow sample rate that caused bytes and packets to be overstated.
- Improved rejection of malformed packets.
- Fixed incorrect timestamps for some Netflow v9 Option Data.
- Added RiskIQ IP reputation and routed block data to client and server objects.
- Fixed decoding of microsecond and nanosecond timestamps.
- Interface name enrichment will fallback to ifDescr when ifName is unavailable.
- Improved handling of malformed (non-RFC6759 compliant) Application IDs
- Fixed CA certificate validation for Elasticsearch output when TLS is enabled.
- Improved normalization of bytes and packets when flow direction is unknown.
- The Elasticsearch output pool size (EF_FLOW_OUTPUT_ELASTICSEARCH_POOL_SIZE) now allows the pool size to be defined independent of the number of licensed cores (EF_FLOW_LICENSED_CORES).
- ASN to Organization mappings have been updated.
- Environment variables for configuring the flow collector have changed. Please review the installation documentation for a detailed description of all configuration options.
- Elasticsearch Index names and templates have been changed to be schema-specific (CODEX vs ECS). This prevents some conflicts where schemas could be inadvertently mixed due to misconfiguration.
As we are still in a beta phase, we recommend the installation of v5.0.0-beta.3 in a clean environment that does not contain data of the v5.0.0-beta.2, v5.0.0-beta.1 or previous ElastiFlow (v4.x) installations.
- Integration with RiskIQ PassiveTotal for enrichment and flow data analysis.
- Expanded options for handling sampled flow records.
- Ability to define Applications per IP address and port number.
- Elasticsearch Output support for Index Management and Ingest Pipelines
- Integration with RiskIQ PassiveTotal.
- Sampling rates can be learned from option data records.
- Sampling rates can be statically defined per flow exporter IP address.
- Applications can be statically defined per IP address and port number.
- Hostnames for reverse IP lookups can be statically defined.
- Elasticsearch Ingest Pipelines can now be specified.
- An Elasticsearch ILM Lifecycle can now be specified.
- Open Distro for Elasticsearch ISM Policy can be specified.
- Logging to file with log rotation.
- Refactored reverse name lookups for better performance.
- Fixed issue where flow.export.host.name was not being set using the sFlow agent IP address.
- Fixed a condition where the timestamp was not normalized properly result in indices created in the past.
- Fixed issue where Maxmind ASN and GeoIP cache sizes where not set as configured.
- Enabling both the Elasticsearch and stdout outputs simultaneously no longer causes the collector to exit.
- Environment variables for configuring unicolld have changed.
- Changes to Elasticsearch templates.
As we are still in a beta environment, we recommend the installation of v5.0.0-beta.2 in a new environment that does not contain data of the v5.0.0-beta.1 or previous ElastiFlow (v4.x) installations.
- TLS support on the Elasticsearch output.
- ElastiFlow's documentation site.
- Support for Elastic Common Schema (ECS).
- General bugs and performance updates.
- Adds support for TLS configuration of Elasticsearch output.
- Adds support for configuring the UDP server's kernel buffer size.
- Adds support for enabling the ECS output on the Elasticsearch output.
- Created ECS-based Kibana dashboards.
- Configuration changes for licensing and core allowances.
- Populates *.host.name with IP when DNS disabled.
- Ensures all timestamps are normalized.
- Prevents error that occurred when the TCP header size is too small.
- Prevents record duplication issue.