Skip to content

Commit

Permalink
Merge pull request #586 from Altinity/2.2.0
Browse files Browse the repository at this point in the history 
Release 2.2.0
  • Loading branch information
@subkanthi
subkanthi authored Jul 15, 2024
2 parents abe1154 + 75f2ee3 commit 12d050f
Show file tree
Hide file tree
Showing 139 changed files with 2,513 additions and 542 deletions.
12 changes: 12 additions & 0 deletions .github/workflows/testflows-sink-connector-kafka.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,18 @@ jobs:
working-directory: sink-connector/tests/integration
run: echo "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$(hostname -I | cut -d ' ' -f 1)"

- name: Create a virtual environment
run: |
echo "Install Python modules..."
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y python3.12-venv
echo "Create and activate Python virtual environment..."
python3 -m venv venv
source venv/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
- name: Install all dependencies
working-directory: sink-connector/tests/integration
run: pip3 install -r requirements.txt
Expand Down
24 changes: 24 additions & 0 deletions .github/workflows/testflows-sink-connector-lightweight.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,18 @@ jobs:
working-directory: sink-connector/tests/integration
run: echo "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$(hostname -I | cut -d ' ' -f 1)"

- name: Create a virtual environment
run: |
echo "Install Python modules..."
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y python3.12-venv
echo "Create and activate Python virtual environment..."
python3 -m venv venv
source venv/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
- name: Install all dependencies
working-directory: sink-connector-lightweight/tests/integration
run: pip3 install -r requirements.txt
Expand Down Expand Up @@ -130,6 +142,18 @@ jobs:
working-directory: sink-connector/tests/integration
run: echo "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root@$(hostname -I | cut -d ' ' -f 1)"

- name: Create a virtual environment
run: |
echo "Install Python modules..."
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y python3.12-venv
echo "Create and activate Python virtual environment..."
python3 -m venv venv
source venv/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
- name: Install all dependencies
working-directory: sink-connector-lightweight/tests/integration
run: pip3 install -r requirements.txt
Expand Down
3 changes: 3 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,9 @@ First two are good tutorials on MySQL and PostgreSQL respectively.
* [Mutable Data Handling](doc/mutable_data.md)
* [ClickHouse Table Engine Types](doc/clickhouse_engines.md)
* [Troubleshooting](doc/Troubleshooting.md)
* [TimeZone and DATETIME/TIMESTAMP](doc/timezone.md)
* [Logging](doc/Logging.md)
* [Production Setup](doc/production_setup.md)

### Operations

Expand Down
2 changes: 0 additions & 2 deletions doc/OLD_README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,6 @@ GLOBAL OPTIONS:
| clickhouse.server.user | ClickHouse username |
| clickhouse.server.password | ClickHouse password |
| clickhouse.server.port | ClickHouse port, For TLS(use the correct port `8443` or `443` |
| clickhouse.server.database | ClickHouse destination database |
| snapshot.mode | "initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.\<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ <br/>PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode <br/> MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html |
| connector.class | MySQL -> "io.debezium.connector.mysql.MySqlConnector" <br/> PostgreSQL -> <br/> Mongo -> <br/> |
| offset.storage.file.filename | Offset storage file(This stores the offsets of the source database) MySQL: mysql binlog file and position, gtid set. Make sure this file is durable and its not persisted in temp directories. |
Expand Down Expand Up @@ -298,7 +297,6 @@ mvn install -DskipTests=true
| clickhouse.server.url | | ClickHouse Server URL |
| clickhouse.server.user | | ClickHouse Server username |
| clickhouse.server.password | | ClickHouse Server password |
| clickhouse.server.database | | ClickHouse Database name |
| clickhouse.server.port | 8123 | ClickHouse Server port |
| clickhouse.topic2table.map | No | Map of Kafka topics to table names, <topic_name1>:<table_name1>,<topic_name2>:<table_name2> This variable will override the default mapping of topics to table names. |
| store.kafka.metadata | false | If set to true, kafka metadata columns will be added to Clickhouse |
Expand Down
1 change: 0 additions & 1 deletion doc/configuration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@
| clickhouse.server.user | ClickHouse username |
| clickhouse.server.password | ClickHouse password |
| clickhouse.server.port | ClickHouse port, For TLS(use the correct port `8443` or `443` |
| clickhouse.server.database | ClickHouse destination database |
| snapshot.mode | "initial" -> Data that already exists in source database will be replicated. "schema_only" -> Replicate data that is added/modified after the connector is started.\<br/> MySQL: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-snapshot-mode \ <br/>PostgreSQL: https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-snapshot-mode <br/> MongoDB: initial, never. https://debezium.io/documentation/reference/stable/connectors/mongodb.html |
| connector.class | MySQL -> "io.debezium.connector.mysql.MySqlConnector" <br/> PostgreSQL -> <br/> Mongo -> <br/> |
| offset.storage.file.filename | Offset storage file(This stores the offsets of the source database) MySQL: mysql binlog file and position, gtid set. Make sure this file is durable and its not persisted in temp directories. |
Expand Down
1 change: 0 additions & 1 deletion doc/configuration_kafka.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ for both **Debezium** and **Sink**
| clickhouse.server.url | | ClickHouse Server URL |
| clickhouse.server.user | | ClickHouse Server username |
| clickhouse.server.password | | ClickHouse Server password |
| clickhouse.server.database | | ClickHouse Database name |
| clickhouse.server.port | 8123 | ClickHouse Server port |
| clickhouse.topic2table.map | No | Map of Kafka topics to table names, <topic_name1>:<table_name1>,<topic_name2>:<table_name2> This variable will override the default mapping of topics to table names. |
| store.kafka.metadata | false | If set to true, kafka metadata columns will be added to Clickhouse |
Expand Down
Loading

0 comments on commit 12d050f

Please sign in to comment.