Skip to content

Commit

Permalink
update eventhub readme file (microsoft#751)
Browse files Browse the repository at this point in the history
* update eventhub readme file

* Update README.adoc

Co-authored-by: unknown <fenzho@FZHOU.fareast.corp.microsoft.com>
  • Loading branch information
zhoufenqin and unknown authored Aug 7, 2020
1 parent 957fd8a commit e46226f
Show file tree
Hide file tree
Showing 7 changed files with 118 additions and 69 deletions.
108 changes: 67 additions & 41 deletions spring-cloud-azure-samples/eventhubs-binder-sample/README.adoc
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
= Spring Cloud Azure Stream Binder for Event Hub Code Sample

This code sample demonstrates how to use the Spring Cloud Stream binder for Azure Event Hub.
This code sample demonstrates how to use the Spring Cloud Stream Binder for Azure Event Hub.
The sample app exposes a rest api to receive string message. Then message is sent through Azure
Event Hub to a `sink` which simply logs the message.

Running this sample will be charged by Azure. You can check the usage and bill at https://azure.microsoft.com/en-us/account/[this link].

== Connection string based usage
We have several ways to config the Spring Cloud Stream Binder for Azure Event Hub. You can choose anyone of them.

== Method 1: Connection string based usage

1. Create https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-create[Azure Event Hubs]. Please note `Basic` tier is unsupported.
After creating the Azure Event Hub, you can create your own Consumer Group or use the default "$Default" Consumer Group.

2. Create https://docs.microsoft.com/en-us/azure/storage/[Azure Storage] for checkpoint use.

3. Update link:src/main/resources/application.properties[application.properties]
3. Update link:src/main/resources/application.properties[application.properties].

+
....
Expand All @@ -23,9 +26,21 @@ spring.cloud.azure.eventhub.connection-string=[eventhub-namespace-connection-str
spring.cloud.azure.eventhub.checkpoint-storage-account=[checkpoint-storage-account]
spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
# Fill eventhub infomation
spring.cloud.stream.bindings.input.destination=[eventhub-name]
spring.cloud.stream.bindings.input.group=[consumer-group]
spring.cloud.stream.bindings.output.destination=[eventhub-name]
# Use manual checkpoint mode
spring.cloud.stream.eventhub.bindings.input.consumer.checkpoint-mode=MANUAL
# Config this property if you want to create resource automatically
#spring.cloud.azure.auto-create-resources=true
#spring.cloud.azure.region=[region]
....

== Credential file based usage
== Method 2: Credential file based usage

1. Create Azure credential file. Please see https://github.com/Azure/azure-libraries-for-java/blob/master/AUTH.md[how
to create credential file]
Expand Down Expand Up @@ -63,56 +78,68 @@ spring.cloud.azure.resource-group=[resource-group]
spring.cloud.azure.eventhub.namespace=[eventhub-namespace]
spring.cloud.azure.eventhub.checkpoint-storage-account=[checkpoint-storage-account]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
# Fill eventhub infomation
spring.cloud.stream.bindings.input.destination=[eventhub-name]
spring.cloud.stream.bindings.input.group=[consumer-group]
spring.cloud.stream.bindings.output.destination=[eventhub-name]
....

== MSI credential based usage
== Method 3: MSI credential based usage

=== Overview

https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/[MSI] (Managed Service Identity, aka Managed Identity) for Azure resources provides Azure services with an automatically managed identity in https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-whatis[Azure AD]. You can use this identity to authenticate to any service that supports Azure AD authentication without having any credentials in your code.

Please note your application should run in VM (Virtual Machine) or App Services on Azure for support of MSI.

=== Prerequisites
1. Create https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-create[Azure Event Hubs]. Please note `Basic` tier is unsupported.

2. Create https://docs.microsoft.com/en-us/azure/storage/[Azure Storage] for checkpoint use.

=== Setup Application
Please note your application should run in VM (Virtual Machine) or App Services on Azure for support of MSI. Choose any of them.

=== Setup VM and assign identity
==== Method 1: Setup VM and assign identity

3. Create VM in Azure portal.
1. Create VM in Azure portal.
Please refer to https://docs.microsoft.com/en-us/azure/virtual-machines/windows/quick-create-portal[Create a Windows virtual machine in the Azure portal] or https://docs.microsoft.com/en-us/azure/virtual-machines/linux/quick-create-portal[Create a Linux virtual machine in the Azure portal]. Choose any one according to your needs.

4. Create an user-assigned identity in Azure Portal.
2. Create an user-assigned identity in Azure Portal.
Please refer to https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-manage-ua-identity-portal#create-a-user-assigned-managed-identity[Create an user-assigned managed identity].

5. Assign the user-assigned identity to the VM.
3. Assign the user-assigned identity to the VM.
Please refer to https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/qs-configure-portal-windows-vm#assign-a-user-assigned-managed-identity-to-an-existing-vm[Assign an user-assigned managed identity to an existing VM].

=== Setup App Service and assign identity

1. Please refer to https://docs.microsoft.com/en-us/java/azure/spring-framework/deploy-spring-boot-java-app-with-maven-plugin?toc=%2Fazure%2Fapp-service%2Fcontainers%2Ftoc.json&view=azure-java-stable[
Deploy a Spring Boot JAR file to Azure App Service] to deploy a Spring Boot JAR file.
The maven plugin does not support deploying App Service using `user-assigned` type identity now.
Another way to deploy an executable JAR is via FTP/S. Please refer to https://docs.microsoft.com/en-us/azure/app-service/deploy-ftp[
Deploy your app to App Service using FTP/S]. And the JAR file's name must be `app.jar`.
==== Method 2: Setup App Service and assign identity

- 1. Deploy this sample's Spring Boot JAR file to App Service.

You can follow https://docs.microsoft.com/en-us/java/azure/spring-framework/deploy-spring-boot-java-app-with-maven-plugin?toc=%2Fazure%2Fapp-service%2Fcontainers%2Ftoc.json&view=azure-java-stable[
Deploy a Spring Boot JAR file to Azure App Service] to deploy the JAR file.

Another way to deploy an executable JAR is via FTP/S. Follow https://docs.microsoft.com/en-us/azure/app-service/deploy-ftp[
Deploy your app to App Service using FTP/S]. And the JAR file's name must be `app.jar`.

- 2. Create a managed identity for App Service.

2. Create a managed identity for App Service.
Please refer to https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity#adding-a-system-assigned-identity[
Adding a system assigned identity] if you choose to use system-assigned identity,
or https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity#adding-a-user-assigned-identity[
Adding a user assigned identity] if user-assigned identity is chosen.
If you choose system-assigned identity, follow https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity#adding-a-system-assigned-identity[
Adding a system assigned identity].

If you choose user-assigned identity, follow https://docs.microsoft.com/en-us/azure/app-service/overview-managed-identity#adding-a-user-assigned-identity[
Adding a user assigned identity].

=== Role Assignment

1. Add role assignment to event hub, storage account and resource group. You should add managed identity to the `Contributor` role for event hub, and add managed identity to the `Storage Account Key Operator Service Role` role for storage account and add managed identity to the `Reader` role for resource group.
+
--
For how to add role assignment to event hub, please see https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-managed-service-identity[Managed identities for Azure resources with Event Hubs]. Adding role assignment to storage account is almost similar, you can set it in `Access Control (IAM)` page in Azure Portal of your storage account resource by imitating the steps of adding role assignment for event hub.
Role assignments are the way you control access to Azure resources. For different built-in role's descriptions, please see https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles[Built-in role descriptions].
--
Add role assignment to Event hub, Storage Account and Resource Group.

- Event Hub: `Contributor` role.
- Storage Account: `Storage Account Key Operator Service Role` role.
- Resource Group: `Reader` role.

See https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-managed-service-identity[Managed identities for Azure resources with Event Hubs] to add role assignment for Event Hub, Storage Account and Resource Group are similar.

For different built-in role's descriptions, please see https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles[Built-in role descriptions].

=== Add MSI related properties

Expand All @@ -138,38 +165,37 @@ spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
# Fill event hub namespace
spring.cloud.azure.eventhub.namespace=[eventhub-namespace]
....

== How to run
1. Update stream binding related properties in link:src/main/resources/application.properties[application.properties]

+
....
# Fill eventhub infomation
spring.cloud.stream.bindings.input.destination=[eventhub-name]
spring.cloud.stream.bindings.input.group=[consumer-group]
spring.cloud.stream.bindings.output.destination=[eventhub-name]
spring.cloud.stream.eventhub.bindings.input.consumer.checkpoint-mode=MANUAL
....

2. Run the `mvn clean spring-boot:run` in the root of the code sample to get the app running.

3. Send a POST request
== How to run

1. Run the `mvn clean spring-boot:run` in the root of the code sample to get the app running.

2. Send a POST request
+
....
$ curl -X POST http://localhost:8080/messages?message=hello
....
+
or when the app runs on App Service
or when the app runs on App Service or VM
+
....
$ curl -X POST https://[your-app-URL]/messages?message=hello
$ curl -d -X POST https://[your-app-URL]/messages?message=hello
....

4. Verify in your app's logs that a similar message was posted:
3. Verify in your app's logs that a similar message was posted:
+
....
New message received: 'hello'
Message 'hello' successfully checkpointed
....

5. Delete the resources on http://ms.portal.azure.com/[Azure Portal] to avoid unexpected charges.
4. Delete the resources on http://ms.portal.azure.com/[Azure Portal] to avoid unexpected charges.
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,9 @@ public String postMessage(@RequestParam String message) {
public String hello() {
return "hello world";
}

@GetMapping("/")
public String welcome() {
return "welcome";
}
}
Original file line number Diff line number Diff line change
@@ -1,29 +1,28 @@
# MSI Credential based usage

spring.cloud.azure.msi-enabled=true
spring.cloud.azure.subscription-id=[subscription-id]
spring.cloud.azure.managed-identity.client-id=[The ID of the user-assigned identity to be used]
# ------------------------------- Connection string based usage ---------------------------------
# Fill event hub namespace connection string copied from portal
# spring.cloud.azure.eventhub.connection-string=[eventhub-namespace-connection-string]

# Credential file based usage
# Fill checkpoint storage account name and access key
# spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]

# spring.cloud.azure.credential-file-path=[credential-file-path]
# ------------------------------- Credential file based usage -----------------------------------

# For MSI or credential file based usage
# Credential file based usage
spring.cloud.azure.credential-file-path=[credential-file-path]

spring.cloud.azure.resource-group=[resource-group]
spring.cloud.azure.eventhub.namespace=[eventhub-namespace]

# Storage account name length should be between 3 and 24
# and use numbers and lower-case letters only.
spring.cloud.azure.eventhub.checkpoint-storage-account=[checkpoint-storage-account]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
# ------------------------------- MSI Credential based usage -----------------------------------

# Connection string and access key based usage
#spring.cloud.azure.msi-enabled=true
#spring.cloud.azure.subscription-id=[subscription-id]
#spring.cloud.azure.managed-identity.client-id=[The ID of the user-assigned identity to be used]

# Fill event hub namespace connection string copied from portal
# spring.cloud.azure.eventhub.connection-string=[eventhub-namespace-connection-string]
# Fill checkpoint storage account name and access key
# spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]
#spring.cloud.azure.resource-group=[resource-group]
#spring.cloud.azure.eventhub.namespace=[eventhub-namespace]

# ------------------------------- Common Properties --------------------------------------------

# For example here, the destination name of input and output should be the same.
# Eventhub name can contain only lowercase letters, numbers, and the dash (-) character.
Expand All @@ -33,16 +32,25 @@ spring.cloud.stream.bindings.input.destination=[eventhub-name]
spring.cloud.stream.bindings.input.group=[consumer-group]
spring.cloud.stream.bindings.output.destination=[eventhub-name-same-as-above]

# enable producer error channel, it is closed by default
spring.cloud.stream.default.producer.errorChannelEnabled=true

# Storage account name length should be between 3 and 24
# and use numbers and lower-case letters only.
spring.cloud.azure.eventhub.checkpoint-storage-account=[checkpoint-storage-account]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]


# Use manual checkpoint mode
spring.cloud.stream.eventhub.bindings.input.consumer.checkpoint-mode=MANUAL

# Config this property if you want to create resource automatically
# spring.cloud.azure.auto-create-resources=true

# Default environment is AZURE. Provide your own if in another environment
# Example environment is AZURE, AZURE_CHINA
# spring.cloud.azure.environment=[environment]

# Example region is westUS, northchina
# spring.cloud.azure.region=[region]

# enable producer error channel, it is closed by default
spring.cloud.stream.default.producer.errorChannelEnabled=true
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@ This code sample demonstrates how to use Spring Integration for Azure Event Hub.

Running this sample will be charged by Azure. You can check the usage and bill at https://azure.microsoft.com/en-us/account/[this link].

== Connection string based usage
We have several ways to config the Azure Event Hub. You can choose anyone of them.

1. Create https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-create[Azure Event Hubs]. Please note `Basic` tier is unsupported.
== Method 1: Connection string based usage

1. Create https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-create[Azure Event Hubs]. Please note `Basic` tier is unsupported.
After creating the Azure Event Hub, you can create your own Consumer Group or use the default "$Default" Consumer Group.

2. Create https://docs.microsoft.com/en-us/azure/storage/[Azure Storage] for checkpoint use.

Expand All @@ -23,7 +26,7 @@ spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
....

== Credential file based usage
== Method 2: Credential file based usage

1. Create Azure credential file. Please see https://github.com/Azure/azure-libraries-for-java/blob/master/AUTH.md[how
to create credential file]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,19 @@
# Credential file based usage
# ----------------------- Credential file based usage ---------------------------------------

spring.cloud.azure.credential-file-path=[credential-file]
spring.cloud.azure.resource-group=[resource-group]
spring.cloud.azure.eventhub.namespace=[eventhub-namespace]

# Connection string and access key based usage
# ----------------------- Connection string and access key based usage ----------------------

# Fill event hub namespace connection string copied from portal
# spring.cloud.azure.eventhub.connection-string=[eventhub-namespace-connection-string]

# Fill checkpoint storage account name and access key
# spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]


# ----------------------------------- Common Properties -------------------------------------

spring.cloud.azure.eventhub.checkpoint-storage-account=[checkpoint-storage-account]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ This code sample demonstrates how to use https://github.com/Microsoft/spring-clo

Running this sample will be charged by Azure. You can check the usage and bill at https://azure.microsoft.com/en-us/account/[this link].

== Connection string based usage
We have several ways to config the Azure Event Hub. You can choose anyone of them.

== Method 1: Connection string based usage

1. Create https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-create[Azure Event Hubs]. Please note `Basic` tier is unsupported.

Expand All @@ -23,7 +25,7 @@ spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]
....

== Credential file based usage
== Method 2: Credential file based usage

1. Create Azure credential file. Please see https://github.com/Azure/azure-libraries-for-java/blob/master/AUTH.md[how
to create credential file]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
# Credential file based usage
# -------------------------- Credential file based usage ----------------------------------

spring.cloud.azure.credential-file-path=[credential-file]
spring.cloud.azure.resource-group=[resource-group]
spring.cloud.azure.eventhub.namespace=[eventhub-namespace]

# Connection string and access key based usage
# -------------------------- Connection string and access key based usage -----------------

# Fill event hub namespace connection string copied from portal
# spring.cloud.azure.eventhub.connection-string=[eventhub-namespace-connection-string]
# Fill checkpoint storage account name and access key
# spring.cloud.azure.eventhub.checkpoint-access-key=[checkpoint-accesskey]

# -------------------------------- Common Properties -------------------------------------
spring.cloud.azure.eventhub.checkpoint-storage-account=[checkpoint-storage-account]
spring.cloud.azure.eventhub.checkpoint-container=[checkpoint-container]

0 comments on commit e46226f

Please sign in to comment.