Skip to content

Commit

Permalink
[Event Hubs Client] Core Samples Overhaul (#16479)
Browse files Browse the repository at this point in the history 
The focus of these changes is to overhaul the samples, converting the
old-style code samples into markdown while extending them with more context,
better detail, and additional scenarios.

The content enhancements have taken into account feedback from customers
since GA in the form of issues, Stack Overflow questions, and direct conversations.
  • Loading branch information
@jsquire
jsquire authored Nov 3, 2020
1 parent de304ae commit 0bf86dd
Show file tree
Hide file tree
Showing 45 changed files with 3,968 additions and 2,713 deletions.
4 changes: 3 additions & 1 deletion sdk/eventhub/Azure.Messaging.EventHubs.Processor/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ The Event Processor client library is a companion to the Azure Event Hubs client

- **Azure Storage account with blob storage:** To persist checkpoints as blobs in Azure Storage, you'll need to have an Azure Storage account with blobs available. If you are not familiar with Azure Storage accounts, you may wish to follow the step-by-step guide for [creating a storage account using the Azure portal](https://docs.microsoft.com/azure/storage/common/storage-quickstart-create-account?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json&tabs=azure-portal). There, you can also find detailed instructions for using the Azure CLI, Azure PowerShell, or Azure Resource Manager (ARM) templates to create storage accounts.

- **C# 8.0:** The Azure Event Hubs client library makes use of new features that were introduced in C# 8.0. In order to take advantage of the C# 8.0 syntax, it is recommended that you compile using the [.NET Core SDK](https://dotnet.microsoft.com/download) 3.0 or higher with a [language version](https://docs.microsoft.com/dotnet/csharp/language-reference/configure-language-version#override-a-default) of `latest`. It is also possible to compile with the .NET Core SDK 2.1.x using a language version of `preview`. Visual Studio users wishing to take advantage of the C# 8.0 syntax will need to use Visual Studio 2019 or later. Visual Studio 2019, including the free Community edition, can be downloaded [here](https://visualstudio.microsoft.com).
- **C# 8.0:** The Azure Event Hubs client library makes use of new features that were introduced in C# 8.0. In order to take advantage of the C# 8.0 syntax, it is recommended that you compile using the [.NET Core SDK](https://dotnet.microsoft.com/download) 3.0 or higher with a [language version](https://docs.microsoft.com/dotnet/csharp/language-reference/configure-language-version#override-a-default) of `latest`. It is also possible to compile with the .NET Core SDK 2.1.x using a language version of `preview`.

Visual Studio users wishing to take full advantage of the C# 8.0 syntax will need to use Visual Studio 2019 or later. Visual Studio 2019, including the free Community edition, can be downloaded [here](https://visualstudio.microsoft.com). Users of Visual Studio 2017 can take advantage of the C# 8 syntax by making use of the [Microsoft.Net.Compilers NuGet package](https://www.nuget.org/packages/Microsoft.Net.Compilers/) and setting the language version, though the editing experience may not be ideal.

You can still use the library with previous C# language versions, but will need to manage asynchronous enumerable and asynchronous disposable members manually rather than benefiting from the new syntax. You may still target any framework version supported by your .NET Core SDK, including earlier versions of .NET Core or the .NET framework. For more information, see: [how to specify target frameworks](https://docs.microsoft.com/dotnet/standard/frameworks#how-to-specify-target-frameworks).

Expand Down
29 changes: 15 additions & 14 deletions sdk/eventhub/Azure.Messaging.EventHubs/Azure.Messaging.EventHubs.sln
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Messaging.EventHubs",
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Messaging.EventHubs.Tests", "tests\Azure.Messaging.EventHubs.Tests.csproj", "{51A23FD9-A32D-4390-9A5B-1EA7A045F0F9}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Messaging.EventHubs.Samples", "samples\Azure.Messaging.EventHubs.Samples.csproj", "{AD33C619-AC51-4F29-937B-184B4F785F57}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Azure.Core.TestFramework", "..\..\core\Azure.Core.TestFramework\src\Azure.Core.TestFramework.csproj", "{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "External", "External", "{2DB233D3-E757-423C-8F8D-742B0AFF4713}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "System.Memory.Data", "..\..\core\System.Memory.Data\src\System.Memory.Data.csproj", "{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
Expand Down Expand Up @@ -47,18 +47,6 @@ Global
{51A23FD9-A32D-4390-9A5B-1EA7A045F0F9}.Release|x64.Build.0 = Release|Any CPU
{51A23FD9-A32D-4390-9A5B-1EA7A045F0F9}.Release|x86.ActiveCfg = Release|Any CPU
{51A23FD9-A32D-4390-9A5B-1EA7A045F0F9}.Release|x86.Build.0 = Release|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Debug|Any CPU.Build.0 = Debug|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Debug|x64.ActiveCfg = Debug|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Debug|x64.Build.0 = Debug|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Debug|x86.ActiveCfg = Debug|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Debug|x86.Build.0 = Debug|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Release|Any CPU.ActiveCfg = Release|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Release|Any CPU.Build.0 = Release|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Release|x64.ActiveCfg = Release|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Release|x64.Build.0 = Release|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Release|x86.ActiveCfg = Release|Any CPU
{AD33C619-AC51-4F29-937B-184B4F785F57}.Release|x86.Build.0 = Release|Any CPU
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}.Debug|x64.ActiveCfg = Debug|Any CPU
Expand All @@ -71,12 +59,25 @@ Global
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}.Release|x64.Build.0 = Release|Any CPU
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}.Release|x86.ActiveCfg = Release|Any CPU
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433}.Release|x86.Build.0 = Release|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Debug|Any CPU.Build.0 = Debug|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Debug|x64.ActiveCfg = Debug|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Debug|x64.Build.0 = Debug|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Debug|x86.ActiveCfg = Debug|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Debug|x86.Build.0 = Debug|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Release|Any CPU.ActiveCfg = Release|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Release|Any CPU.Build.0 = Release|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Release|x64.ActiveCfg = Release|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Release|x64.Build.0 = Release|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Release|x86.ActiveCfg = Release|Any CPU
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF}.Release|x86.Build.0 = Release|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(NestedProjects) = preSolution
{2CFDB3D6-5CFB-428C-9C89-29DD169B5433} = {2DB233D3-E757-423C-8F8D-742B0AFF4713}
{EDC6F52C-48FA-410A-9EC3-B77C33576ECF} = {2DB233D3-E757-423C-8F8D-742B0AFF4713}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {509F2EE0-3348-4506-8AC7-9945B602CB43}
Expand Down
23 changes: 11 additions & 12 deletions sdk/eventhub/Azure.Messaging.EventHubs/MigrationGuide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This guide is intended to assist in the migration to version 5 of the Event Hubs client library from version 4. It will focus on side-by-side comparisons for similar operations between the v5 packages, [`Azure.Messaging.EventHubs`](https://www.nuget.org/packages/Azure.Messaging.EventHubs/) and [`Azure.Messaging.EventHubs.Processor`](https://www.nuget.org/packages/Azure.Messaging.EventHubs.Processor/) and their v4 equivalents, [`Microsoft.Azure.EventHubs`](https://www.nuget.org/packages/Microsoft.Azure.EventHubs/) and [`Microsoft.Azure.EventHubs.Processor`](https://www.nuget.org/packages/Microsoft.Azure.EventHubs.Processor/).

Familiarity with the v4 client library is assumed. For those new to the Event Hubs client library for .NET, please refer to the [README](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/README.md), [Event Hubs samples](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs/samples), and the [Event Processor samples](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples) for the v5 library rather than this guide.
Familiarity with the v4 client library is assumed. For those new to the Event Hubs client library for .NET, please refer to the [README](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/README.md), [Event Hubs samples](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples), and the [Event Processor samples](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples) for the v5 library rather than this guide.

## Table of contents

Expand Down Expand Up @@ -67,16 +67,16 @@ In order to allow for a single focus and clear responsibility, the core function

- The [PartitionReceiver](https://docs.microsoft.com/dotnet/api/azure.messaging.eventhubs.primitives.partitionreceiver?view=azure-dotnet) is responsible for reading events from a specific partition of an Event Hub, with a greater level of control over communication with the Event Hubs service than is offered by other event consumers. More detail on the design and philosophy for the `PartitionReceiver` can be found in its [design document](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/design/partition-receiver-proposal.md).

- The [EventProcessor<TPartition>](https://docs.microsoft.com/dotnet/api/azure.messaging.eventhubs.primitives.eventprocessor-1?view=azure-dotnet) provides a base for creating a custom processor for for reading and processing events for all partitions of an Event Hub. The `EventProcessor<TPartition>` fills a similar role as the EventProcessorClient, with cooperative load balancing and resiliency as its core features. However, it also offers native batch processing, the ability to customize checkpoint storage, a greater level of control over communication with the Event Hubs service, and a less opinionated API. The caveat is that this comes with additional complexity and exists as of an abstract base, which needs to be extended and the core "handler" activities implemented via override.
- The [EventProcessor&lt;TPartition&gt;](https://docs.microsoft.com/dotnet/api/azure.messaging.eventhubs.primitives.eventprocessor-1?view=azure-dotnet) provides a base for creating a custom processor for reading and processing events for all partitions of an Event Hub. The `EventProcessor<TPartition>` fills a similar role as the EventProcessorClient, with cooperative load balancing and resiliency as its core features. However, it also offers native batch processing, the ability to customize checkpoint storage, a greater level of control over communication with the Event Hubs service, and a less opinionated API. The caveat is that this comes with additional complexity and exists as of an abstract base, which needs to be extended and the core "handler" activities implemented via override.

Generally speaking, the `EventProcessorClient` was designed to provide a familiar API to that of the `EventHubConsumerClient` and offer an intuitive "step-up" experience for developers exploring Event Hubs as they advance to production scenarios. For a large portion of our library users, that covers their needs well. There's definitely a point, however, where an application requires more control to handle higher throughput or unique needs - that's where the `EventProcessor<TPartition>` is intended to help. More on the design and philosophy behind this type can be found in its [design document](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/design/event-processor%7BT%7D-proposal.md).

### Client constructors

| In v4 | Equivalent in v5 | Sample |
|------------------------------------------------|------------------------------------------------------------------|--------|
| `EventHubClient.CreateFromConnectionString()` | `new EventHubProducerClient()` or `new EventHubConsumerClient()` | [Publish Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample03_PublishAnEventBatch.cs), [Read Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample05_ReadEvents.cs) |
| `EventHubClient.CreateWithAzureActiveDirectory()` or `EventHubClient.CreateWithManagedIdentity()` | `new EventHubProducerClient(..., TokenCredential)` or `new EventHubConsumerClient(..., TokenCredential)` | [Authenticate with client secret credential](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample12_AuthenticateWithClientSecretCredential.cs)
| `EventHubClient.CreateFromConnectionString()` | `new EventHubProducerClient()` or `new EventHubConsumerClient()` | [Publishing Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample04_PublishingEvents.md), [Reading Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample05_ReadingEvents.md) |
| `EventHubClient.CreateWithAzureActiveDirectory()` or `EventHubClient.CreateWithManagedIdentity()` | `new EventHubProducerClient(..., TokenCredential)` or `new EventHubConsumerClient(..., TokenCredential)` | [Identity and Shared Access Credentials](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample06_IdentityAndSharedAccessCredentials.md)
| `new EventProcessorHost()` | `new EventProcessorClient(BlobContainerClient, ...)` | [Basic Event Processing](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples/Sample03_BasicEventProcessing.cs) |

### Publish events
Expand All @@ -85,14 +85,14 @@ The v4 client allowed for sending a single event or an enumerable of events, whi

| In v4 | Equivalent in v5 | Sample |
|------------------------------------------------|------------------------------------------------------------------|--------|
| `PartitionSender.SendAsync()` | `EventHubProducerClient.SendAsync()` | [Publish events to a specific partition](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample07_PublishAnEventBatchToASpecificPartition.cs) |
| `EventHubClient.SendAsync()` | `EventHubProducerClient.SendAsync()` | [Publish events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample03_PublishAnEventBatch.cs) |
| `PartitionSender.SendAsync()` | `EventHubProducerClient.SendAsync()` | [Publishing Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample04_PublishingEvents.md) |
| `EventHubClient.SendAsync()` | `EventHubProducerClient.SendAsync()` | [Publishing Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample04_PublishingEvents.md) |

### Read events

| In v4 | Equivalent in v5 | Sample |
|------------------------------------------------|------------------------------------------------------------------|--------|
| `PartitionReceiver.ReceiveAsync()` or `PartitionReceiver.SetReceiveHandler()` | `EventHubConsumerClient.ReadEventsFromPartitionAsync()` | [Read events from a known position](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample10_ReadEventsFromAKnownPosition.cs) |
| `PartitionReceiver.ReceiveAsync()` or `PartitionReceiver.SetReceiveHandler()` | `EventHubConsumerClient.ReadEventsFromPartitionAsync()` | [Reading Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample05_ReadingEvents.md) |
| `new EventProcessorHost()` | `new EventProcessorClient(blobContainerClient, ...)` | [Basic Event Processing](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples/Sample03_BasicEventProcessing.cs) |

## Migration samples
Expand All @@ -103,8 +103,7 @@ In v4, events could be published to a single partition using `PartitionSender`.

In v5, this has been consolidated into a more efficient SendAsync(EventDataBatch) method. Batching merges information from multiple events into a single publish message, reducing the amount of network communication needed vs publishing events one at a time. Events are published to a specific partition when partition id is set in [`CreateBatchOptions`](https://docs.microsoft.com/dotnet/api/azure.messaging.eventhubs.producer.createbatchoptions?view=azure-dotnet) before calling [`CreateBatchAsync(CreateBatchOptions)`](https://docs.microsoft.com/dotnet/api/azure.messaging.eventhubs.producer.eventhubproducerclient.createbatchasync?view=azure-dotnet#Azure_Messaging_EventHubs_Producer_EventHubProducerClient_CreateBatchAsync_Azure_Messaging_EventHubs_Producer_CreateBatchOptions_System_Threading_CancellationToken_).

The code below assumes all events fit into a single batch. For more complete example, see sample: [Publish events
to specific partition](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample07_PublishAnEventBatchToASpecificPartition.cs).
The code below assumes all events fit into a single batch. For more complete example, see: [Publishing events without a batch](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample04_PublishingEvents.md#publishing-events-without-an-explicit-batch).

In v4:
```csharp
Expand Down Expand Up @@ -285,7 +284,7 @@ await using (var consumer = new EventHubConsumerClient(consumerGroup, connection
}
```

See [`ReadEventsFromAKnownPosition.cs`](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample10_ReadEventsFromAKnownPosition.cs) for a sample program demonstrating this.
See [Reading Events](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples/Sample05_ReadingEvents.md) for a deep dive into using the `EventHubConsumerClient` in different scenarios.

### Migrating code from `EventProcessorHost` to `EventProcessorClient` for reading events

Expand Down Expand Up @@ -506,5 +505,5 @@ private class Checkpoint
## Additional samples

More examples can be found at:
- [Event Hubs samples](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs/samples)
- [Event Hubs Processor samples](https://github.com/Azure/azure-sdk-for-net/tree/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples)
- [Event Hubs samples](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs/samples)
- [Event Hubs Processor samples](https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples)
Loading

0 comments on commit 0bf86dd

Please sign in to comment.