Skip to main content
Version: 7.8

Setting up Azure Flow Logs

Introduction​

Virtual network flow logs are a feature of Azure Network Watcher. You can use them to log information about IP traffic flowing through a virtual network. See Azure documentation for more information.

This guide will walk you through the process of configuring Azure: enabling flow logs for a virtual network, subnet, or network interface, and then forwarding the flow log events to an Azure Event Hub.

Prerequisites​

  • An Azure account with an active subscription.
  • Microsoft.Insights provider must be registered to successfully log traffic flowing through a virtual network (guide).
  • A virtual network, subnet, or network interface for which you want to collect flow logs.

Create a Flow Log​

See Microsoft documentation for how to Create a Flow Log

Forward Flow Log events to an Event Hub​

The following steps create the necessary resources to push events from the storage account holding the VNet Flow Logs to a real-time data streaming platform (Event Hubs). This will allow flowcoll to listen to new events on the configured Event Hub and collect the appropriate Azure VNet Flow Logs from the storage account.

Create an Events Hub namespace​

See Microsoft documentation for how to Create an Events Hub namespace

Create an Event Hub​

See Microsoft documentation for how to Create an Event Hub

Create an Event Grid subscription​

  1. Navigate to the storage account for the Flow Logs you want to collect.

  2. In the left menu, select Events > + Event Subscription.

Adding Event Subscription in Azure

  1. In the Create Event Subscription window within the Basic tab, provide the following values:
SettingSuggested valueField description
Nametest-grid-connectionThe name of the Event Grid subscription that you want to create.
Event SchemaCloud Event Schema v1.0The schema that should be used for the Event Grid. Only Cloud Event Schema v1.0 is supported.
Topic TypeStorage accountThe type of Event Grid topic. Automatically populated.
Source Resourcegridteststorage1The name of your storage account. Automatically populated.
System Topic Namegridteststorage1...The system topic where Azure Storage publishes events. This system topic then forwards the event to a subscriber that receives and processes events. Automatically populated.
Filter to Event TypesBlob CreatedWhich specific events to get notified for.

  1. Under ENDPOINT DETAILS, take the following steps:

    1. Select Event Hubs.

    2. Click Configure an endpoint and select the Event Hub created in the previous section.

  2. Select Create.

At this point if you navigate to the Overview page for the Event Hub created in the previous section, you should see requests coming in from the storage account through the Event Grid subscription.

Register an Application with Role Assignments​

The following steps register an App with the appropriate role assignments that will allow flowcoll to collect Azure VNet Flow Logs from the storage account created earlier.

Create an App registration​

See Microsoft documentation for how to Create an App registration

Make sure to record the Application (client) ID, which uniquely identifies your application and is needed to properly configure flowcoll.

Add a Client Secret​

  1. In the Microsoft Entra admin center, in App registrations, select your application.

  2. Select Certificates & secrets > Client secrets > New client secret.

  3. Add a description for your client secret.

  4. Select an expiration for the secret or specify a custom lifetime.

  5. Select Add.

  6. Record the secret's value as it is needed to properly configure flowcoll. This secret value is never displayed again after you leave this page.

Assign Role Assignments​

  1. Assign the Storage Blob Data Reader to the above App registration for the storage account previously created.

    1. Navigate to the storage account for the Flow Logs you want to collect.

    2. Select Access Control (IAM) > + Add > Add role assignment.

    3. Find and select the Storage Blob Data Reader role.

    4. Select Next.

    5. Select + Select members and select the App registration created in the previous section.

    6. Select Review + assign.

  2. Assign Azure Event Hubs Data Receiver to the above App registration for the Event Hubs namespace created earlier.

    1. Navigate to the Event Hubs namespace created earlier.

    2. Select Access Control (IAM) > + Add > Add role assignment.

    3. Find and select the Azure Event Hubs Data Receiver role.

    4. Select Next.

    5. Select + Select members and select the App registration created in the previous section.

    6. Select Review + assign.

Configure flowcoll​

For configuring flowcoll, see our configuration page