# ElastiFlow vs. Filebeat and Logstash

### Performance

The following results were obtained with the collectors running on a 16-core (AMD EPYC 7302P) server. The data was output to an Elasticsearch cluster consisting of seven data nodes, with three dedicated master nodes.

As much as possible given the options available, batch sizes and the # of workers were configured to comparable and optimal levels.

To provide a "full-featured" comparison, NetObserv Flow was tested with all enrichment features enabled. Logstash was tested with the legacy ElastiFlow 4.x pipeline to give it better feature parity. Filebeat relies on Elasticsearch ingest pipelines for anything beyond basic functionality. These pipelines were NOT used. This does give Filebeat a bit of an unfair advantage, however it was still many times slower despite its more favorable conditions.

| Throughput   |         ElastiFlow         |          Filebeat          | Logstash |
| ------------ | :------------------------: | :------------------------: | :------: |
| Flows/second | :white\_check\_mark: 78818 | :heavy\_minus\_sign: 21217 | :x: 5205 |

### Network Flow Data Support

| Flow Data Support                      |                                                ElastiFlow                                               |                                                 Filebeat                                                |                                                 Logstash                                                |
| -------------------------------------- | :-----------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------: |
| Netflow                                | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>1562 IEs<br>10 Vendors</p> |   <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>474 IEs<br>1 Vendor</p>  |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>422 IEs<br>2 Vendors</p>  |
| IPFIX                                  | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>4585 IEs<br>44 Vendors</p> | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>1319 IEs<br>11 Vendors</p> | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>1329 IEs<br>12 Vendors</p> |
| sFlow Flows                            |                                           :white\_check\_mark:                                          |                                                   :x:                                                   |                                                   :x:                                                   |
| sFlow Counters                         |                                           :white\_check\_mark:                                          |                                                   :x:                                                   |                                                   :x:                                                   |
| Broadcom IFA                           |                                           :white\_check\_mark:                                          |                                                   :x:                                                   |                                                   :x:                                                   |
| IEs most recently added                |      <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>11 July 2022</p>      |      <p><span data-gb-custom-inline data-tag="emoji" data-code="274c">❌</span><br>1 April 2019</p>      |     <p><span data-gb-custom-inline data-tag="emoji" data-code="274c">❌</span><br>4 January 2019</p>     |
| SLA for supporting new vendors/devices |                                           :white\_check\_mark:                                          |                                                   :x:                                                   |                                                   :x:                                                   |

### Platform Support

| Feature            |                                          ElastiFlow                                          |       Filebeat       |       Logstash       |
| ------------------ | :------------------------------------------------------------------------------------------: | :------------------: | :------------------: |
| Elastic Stack      |                                     :white\_check\_mark:                                     | :white\_check\_mark: | :white\_check\_mark: |
| OpenSearch         |                                     :white\_check\_mark:                                     |          :x:         |          :x:         |
| Apache Kafka       |                                     :white\_check\_mark:                                     | :white\_check\_mark: | :white\_check\_mark: |
| Splunk             |                                     :white\_check\_mark:                                     |          :x:         |          :x:         |
| Cribl              |                                     :white\_check\_mark:                                     |          :x:         |          :x:         |
| ClickHouse/Grafana | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>winter 2022</p> |          :x:         |          :x:         |

### Features

| Feature                                                             |                                                 ElastiFlow                                                 |         Filebeat        |          Logstash         |
| ------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------: | :---------------------: | :-----------------------: |
| ECS schema support                                                  |                                            :white\_check\_mark:                                            |   :white\_check\_mark:  |            :x:            |
| CODEX schema support                                                |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Schema for IEs not covered by ECS                                   |           <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>CODEX</p>           |           :x:           |            :x:            |
| Properly handle Netflow v9/IPFIX Templates                          |                                            :white\_check\_mark:                                            |   :white\_check\_mark:  |            :x:            |
| Support Netflow v9/IPFIX Option Data                                |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Translation ("subtype" handling) of IE values                       |      <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>587 translators</p>      |           :x:           | :heavy\_minus\_sign: \*\* |
| GeoIP Enrichment                                                    |                                            :white\_check\_mark:                                            | :heavy\_minus\_sign: \* | :heavy\_minus\_sign: \*\* |
| Autonomous System Enrichment                                        | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>Maxmind<br>or flow record</p> | :heavy\_minus\_sign: \* | :heavy\_minus\_sign: \*\* |
| Reverse DNS IPs to hostname                                         |                                            :white\_check\_mark:                                            | :heavy\_minus\_sign: \* | :heavy\_minus\_sign: \*\* |
| User-defined IPs to hostname                                        |                                            :white\_check\_mark:                                            |           :x:           | :heavy\_minus\_sign: \*\* |
| User-defined Metadata for IPs                                       |                                            :white\_check\_mark:                                            |           :x:           | :heavy\_minus\_sign: \*\* |
| <p>AS-based include/exclude for DNS<br>resolutions and Metadata</p> |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| <p>IP Block include/exclude for DNS<br>resolutions and Metadata</p> |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Obscure IP addresses and Hostnames                                  |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Threat Intelligence Enrichment                                      |          <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>NetIntel</p>         | :heavy\_minus\_sign: \* |  :heavy\_minus\_sign: \*  |
| Microsoft 365 service enrichment                                    |        <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>winter 2022</p>        |           :x:           |            :x:            |
| SalesForce service enrichment                                       |        <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>winter 2022</p>        |           :x:           |            :x:            |
| Infer Client & Server sides of a conversation                       |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Community ID support                                                |                                            :white\_check\_mark:                                            | :heavy\_minus\_sign: \* |            :x:            |
| Conversation ID support                                             |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| User-defined Metadata for Interfaces                                |                                            :white\_check\_mark:                                            |           :x:           | :heavy\_minus\_sign: \*\* |
| <p>Translate Interface Index values<br>to Interface Names</p>       |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| <p>Translate AppIDs to Application<br>names and attributes</p>      |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| User-defined Application names and attributes                       |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Adjust Bytes/Packets based on Sample Rate                           |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| User-defined sample rates per flow exporter                         |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Normalize timestamp values                                          |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Normalize percentage values                                         |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Normalize byte values                                               |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |
| Configurable timestamp precision                                    |                                            :white\_check\_mark:                                            |           :x:           |            :x:            |

\* Must be done in an Elasticsearch Ingest Pipeline. This puts additional load on Elasticsearch, which is already the primary limiter of overall throughput.

\*\* Can be achieved using a Logstash pipeline. This is not provided out of the box and must be developed and maintained.

### Turnkey Analytics for the Elastic Stack

| Feature                                      |                                      ElastiFlow                                      |                                       Filebeat                                      |                                       Logstash                                       |
| -------------------------------------------- | :----------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------: |
| Dashboards                                   |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>29</p> |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2796">➖</span><br>8</p> |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2796">➖</span><br>8</p>  |
| Visualizations                               | <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>347</p> | <p><span data-gb-custom-inline data-tag="emoji" data-code="2796">➖</span><br>78</p> | <p><span data-gb-custom-inline data-tag="emoji" data-code="2796">➖</span><br>143</p> |
| <p>Machine Learning Jobs<br>Security</p>     |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>84</p> |                                         :x:                                         |                                          :x:                                         |
| <p>Machine Learning Jobs<br>Performance</p>  |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>12</p> |                                         :x:                                         |                                          :x:                                         |
| <p>Machine Learning Jobs<br>Availability</p> |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>14</p> |                                         :x:                                         |                                          :x:                                         |
| <p>Detections<br>Security</p>                |  <p><span data-gb-custom-inline data-tag="emoji" data-code="2705">✅</span><br>84</p> |                                         :x:                                         |                                          :x:                                         |


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.elastiflow.com/data_platforms/elastic/filebeat_logstash.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.