forked from debezium/debezium
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
DBZ-6049 Move debezium-server to https://github.com/debezium/debezium…
- Loading branch information
Showing
175 changed files
with
56 additions
and
14,924 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,95 +1 @@ | ||
| # Debezium Server | ||
|
|
||
| Debezium Server is a standalone Java application built on Qurkus framework. | ||
| The application itself contains the `core` module and a set of modules responsible for communication with different target systems. | ||
|
|
||
| The per-module integration tests depend on the availability of the external services. | ||
| It is thus recommended to execute integration tests per-module and set-up necessary pre-requisities beforehand. | ||
|
|
||
| Note: running these tests against external infrastructure may incur cost with your cloud provider. | ||
| We're not going to pay your AWS/GCP/Azure bill. | ||
|
|
||
| ## Amazon Kinesis | ||
|
|
||
| * Execute `aws configure` as described in AWS CLI [getting started](https://github.com/aws/aws-cli#getting-started) guide and setup the account. | ||
| * Create Kinesis stream `aws kinesis create-stream --stream-name testc.inventory.customers --shard-count 1` | ||
| * Build the module and execute the tests `mvn clean install -DskipITs=false -am -pl debezium-server-kinesis` | ||
| * Remove the stream `aws kinesis delete-stream --stream-name testc.inventory.customers` | ||
|
|
||
| ## Google Cloud Pub/Sub | ||
|
|
||
| * Login into your Google Cloud account using `gcloud auth application-default login` as described in the [documentation](https://cloud.google.com/sdk/gcloud/reference/auth/application-default). | ||
| * Build the module and execute the tests `mvn clean install -DskipITs=false -am -pl debezium-server-pubsub` | ||
|
|
||
| ## Azure Event Hubs | ||
|
|
||
| Login into your Azure account and create a resource group, e.g. on the CLI: | ||
|
|
||
| ```shell | ||
| az login | ||
| az group create --name eventhubstest --location westeurope | ||
| ``` | ||
|
|
||
| ### Create an Event Hubs namespace | ||
|
|
||
| Create an [Event Hubs namespace](https://docs.microsoft.com/azure/event-hubs/event-hubs-features#namespace). Check the documentation for options on how do this using the [Azure Portal](https://docs.microsoft.com/azure/event-hubs/event-hubs-create#create-an-event-hubs-namespace), [Azure CLI](https://docs.microsoft.com/azure/event-hubs/event-hubs-quickstart-cli#create-an-event-hubs-namespace) etc., e.g. on the CLI: | ||
|
|
||
| ```shell | ||
| az eventhubs namespace create --name debezium-test --resource-group eventhubstest -l westeurope | ||
| ``` | ||
|
|
||
| ### Create an Event Hub | ||
|
|
||
| Create an Event Hub (equivalent to a topic) with `one` partition. Check the documentation for options on how do this using the [Azure Portal](https://docs.microsoft.com/azure/event-hubs/event-hubs-create#create-an-event-hub), [Azure CLI](https://docs.microsoft.com/azure/event-hubs/event-hubs-quickstart-cli#create-an-event-hub) etc. , e.g. on the CLI: | ||
|
|
||
| ```shell | ||
| az eventhubs eventhub create --name debezium-test-hub --resource-group eventhubstest --namespace-name debezium-test | ||
| ``` | ||
|
|
||
| ### Build the module | ||
|
|
||
| [Get the Connection string](https://docs.microsoft.com/azure/event-hubs/event-hubs-get-connection-string) required to communicate with Event Hubs. The format is: `Endpoint=sb://<NAMESPACE>/;SharedAccessKeyName=<ACCESS_KEY_NAME>;SharedAccessKey=<ACCESS_KEY_VALUE>`. | ||
| E.g. on the CLI: | ||
|
|
||
| ```shell | ||
| az eventhubs namespace authorization-rule keys list --resource-group eventhubstest --namespace-name debezium-test --name RootManageSharedAccessKey | ||
| ``` | ||
|
|
||
| Set environment variables required for tests: | ||
|
|
||
| ```shell | ||
| export EVENTHUBS_CONNECTION_STRING=<Event Hubs connection string> | ||
| export EVENTHUBS_NAME=<name of the Event hub created in previous step> | ||
| ``` | ||
|
|
||
| Execute the tests: | ||
|
|
||
| ```shell | ||
| mvn clean install -DskipITs=false -Deventhubs.connection.string=$EVENTHUBS_CONNECTION_STRING -Deventhubs.hub.name=$EVENTHUBS_NAME -am -pl :debezium-server-eventhubs | ||
| ``` | ||
|
|
||
| ### Examine Events in the Event Hub | ||
|
|
||
| E.g. using kafkacat. Create _kafkacat.conf_: | ||
|
|
||
| ```shell | ||
| metadata.broker.list=debezium-test.servicebus.windows.net:9093 | ||
| security.protocol=SASL_SSL | ||
| sasl.mechanisms=PLAIN | ||
| sasl.username=$ConnectionString | ||
| sasl.password=Endpoint=sb://debezium-test.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<access key> | ||
| ``` | ||
|
|
||
| Start consuming events: | ||
|
|
||
| export KAFKACAT_CONFIG=<path to kafkacat.conf> | ||
| kafkacat -b debezium-test.servicebus.windows.net:9093 -t debezium-test-hub | ||
|
|
||
| ### Clean up | ||
|
|
||
| Delete the Event Hubs namespace and log out, e.g. on the CLI: | ||
|
|
||
| ```shell | ||
| az group delete -n eventhubstest | ||
| az logout | ||
| ``` | ||
| Debezium Server has been moved to a [separate repository](https://github.com/debezium/debezium-server/). |
Oops, something went wrong.