Skip to content

Azure Functions sample for high scale HTTP trigger to Event Hubs using VNET Integration

License

Notifications You must be signed in to change notification settings

Azure-Samples/functions-e2e-http-to-eventhubs

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

description page_type products urlFragment languages
This end-to-end C# sample demonstrates the secure ingestion and processing of HTTP-based inputs in a Flex Consumption plan app using an Event Hubs instance secured in a virtual network.
sample
azure-functions
azure
http-trigger-to-event-hubs
csharp
bicep
azdeveloper

Flex Consumption plan - HTTP trigger to Event Hubs using VNET Integration | Azure Functions

A common scenario for Azure Functions is the ingestion and processing of HTTP-based inputs. For example, an HTTP endpoint that is part of a larger API for ingesting customer feedabck and support information, or a flash sale that needs to ingest customer sign ups interested in the sale. The function app can do some validation and transformation of the provided information before sending it to a secure endpoint for further downstream processing.

This sample demonstrates a function app running in a Flex Consumption plan that connects to Event Hubs running in a virtual network. This sample demonstrates these two key features of the Flex Consumption plan:

  • High scale. Once deployed you can run load tests against the app to see how it handles thousands of requests per second.
  • Virtual network Integration. The Event Hubs instance to which the app sends events is secured behind a private endpoint. The function app can send events to it because it is configured to use virtual network integration.

Diagram showing customer feedback HTTP calls into an Azure Functions Flex Consumption app then sending that data to Event Hubs through a VNet, for further downstream processing

Important

This sample creates several resources, includin an Event Hubs Premium namespace with 100 partitions. Make sure to delete the resource group after testing to minimize charges.

Prerequisites

Before you can run this sample, you must have the following:

Provision the solution on Azure

To set up this sample, follow these steps:

  1. Clone this repository to your local machine.
  2. in the root folder use Azure Developer CLI (azd) to provision a new resource group with the environment name you provide and all the resources for the sample.
azd up

Inspect the solution (optional)

  1. Once the deployment is done, inspect the new resource group. The Flex Consumption function app and plan, storage, App Insights, Event Hubs, and networking services have been created and configured: List of resources created by the bicep template
  2. The Event Hubs namespace public network access has been turned off so nothing outside the VNet can access Event Hubs. You can check this in the Event Hub Namespace's Networking tab and the Public access tab: Event Hubs public network access turned off
  3. The Event Hubs namespace has a private endpoint configured to one of the subnets in the Vnet. This means it can only be accessed from inside the VNet. You can check this in the Event Hub Namespace's Networking tab and the Private Endpoint connections: Event Hubs private endpoint
  4. An outbound virtual network integration has been created in your Function App into another subnet in the same VNet. This means it can access the Event Hubs namespace. You can check this in the function app's Networking tab in the Outbound traffic configuration section: Function App Networking tab
  5. Open the Application Insights instance that was created by the Bicep deploy. Open the Live metrics tab to monitor for live events. Notice that it can't connect to the application. This is expected, because the Flex Consupmtion app is scaled to zero as there's no traffic or executions happening yet. Live Metrics not available

Test the solution

  1. Use a REST client to make HTTP POST calls to the function URL provided by the deployment output. If using VS Code, you can install and use the REST Client extension and use the test.http file. The loadtest folder also has sample JSON files you to use in the testing.
  2. Refresh the Application Insights live metrics, and observe the telemetry and how now it shows successful executions. This means your HTTP calls are making their way through the function app and into you VNet secured Event Hubs instance. Live metrics available
  3. Create a load test against your app using your favorite load testing solution. If you use Azure Load Testing or JMeter, you can use the example loadtest.jmx in your tests, making sure to edit the function name throughput the file.
  4. Run a load test and observe from your load testing solution, or from Application Insights live metrics, how the solution can scale very fast to handle thousands of requests per second, through Flex Consumption and into Event Hubs via the Vnet. Application Insights showing just over 40k RPS and 246 instances

Clean up resources

When you no longer need the resources created in this sample, run the following command to delete the Azure resources:

azd down

Resources

For more information on Azure Functions, Event Hubs, and VNet integration, see the following resources:

About

Azure Functions sample for high scale HTTP trigger to Event Hubs using VNET Integration

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published