Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][doc][Connector-V2][SqlServer] Add SqlServer connector documentation #5498

Merged
merged 12 commits into from
Nov 8, 2023
Merged
Prev Previous commit
Next Next commit
fix doc error
  • Loading branch information
zhilinli123 committed Sep 25, 2023
commit 900f07e7d93864667d31d54d8a5a568bab1d02a0
47 changes: 6 additions & 41 deletions docs/en/connector-v2/source/SqlServer-CDC.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# SqlServer CDC
# SQL Server CDC
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An important content has been missed. How can I start SQL Server's CDC?


> SqlServer CDC source connector

## Support Mysql Version
## Support SQL Server Version

- server:2019 (Or later version for information only)

Expand All @@ -27,17 +27,17 @@ describes how to setup the SqlServer CDC connector to run SQL queries against Sq

## Supported DataSource Info

| Datasource | Supported versions | Driver | Url | Maven |
|------------|-------------------------------------------------------------------------------------------------------|----------------------------------------------|---------------------------------|-----------------------------------------------------------------------|
| SqlServer | <li> [SqlServer](https://dev.mysql.com/doc): server:2019 (Or later version for information only)</li> | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433 | https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc |
| Datasource | Supported versions | Driver | Url | Maven |
|------------|---------------------------------------------------------------|----------------------------------------------|---------------------------------------------------------------|-----------------------------------------------------------------------|
| SqlServer | <li> server:2019 (Or later version for information only)</li> | com.microsoft.sqlserver.jdbc.SQLServerDriver | jdbc:sqlserver://localhost:1433;databaseName=column_type_test | https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc |

### Install Jdbc Driver

Please download and put SqlServer driver in `${SEATUNNEL_HOME}/lib/` dir. For example: cp mssql-jdbc-xxx.jar `$SEATNUNNEL_HOME/lib/`

## Data Type Mapping

| Mysql Data type | SeaTunnel Data type |
| SQLserver Data type | SeaTunnel Data type |
|---------------------------------------------------------------------------------------------------|----------------------------------------------------|
| CHAR<br/>VARCHAR<br/>NCHAR<br/>NVARCHAR<br/>STRUCT<br/>CLOB<br/>LONGVARCHAR<br/>LONGNVARCHAR<br/> | STRING |
| BLOB | BYTES |
Expand Down Expand Up @@ -91,41 +91,6 @@ Please download and put SqlServer driver in `${SEATUNNEL_HOME}/lib/` dir. For ex

> This is a stream mode cdc initializes read table data will be read incrementally after successful read The following sql DDL is for reference only

```
CREATE DATABASE column_type_test;

USE column_type_test;
EXEC sys.sp_cdc_enable_db;

CREATE TABLE full_types (
id int NOT NULL,
val_char char(3),
val_varchar varchar(1000),
val_text text,
val_nchar nchar(3),
val_nvarchar nvarchar(1000),
val_ntext ntext,
val_decimal decimal(6,3),
val_numeric numeric,
val_float float,
val_real real,
val_smallmoney smallmoney,
val_money money,
val_bit bit,
val_tinyint tinyint,
val_smallint smallint,
val_int int,
val_bigint bigint,
val_date date,
val_time time,
val_datetime2 datetime2,
val_datetime datetime,
val_smalldatetime smalldatetime,
val_xml xml
PRIMARY KEY (id)
);
```

```
env {
# You can set engine configuration here
Expand Down
92 changes: 58 additions & 34 deletions docs/en/connector-v2/source/SqlServer.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,46 +80,71 @@ Read external data source data through JDBC.

### Simple:

> Read the table directly and print to the console DDL is for reference only
> Simple single task to read the data table

```
CREATE DATABASE column_type_test;

USE column_type_test;
EXEC sys.sp_cdc_enable_db;

CREATE TABLE full_types_jdbc (
id int NOT NULL,
val_char char(3),
val_varchar varchar(1000),
val_text text,
val_nchar nchar(3),
val_nvarchar nvarchar(1000),
val_ntext ntext,
val_decimal decimal(6,3),
val_numeric numeric,
val_float float,
val_real real,
val_smallmoney smallmoney,
val_money money,
val_bit bit,
val_tinyint tinyint,
val_smallint smallint,
val_int int,
val_bigint bigint,
val_date date,
val_time time,
val_datetime2 datetime2,
val_datetime datetime,
val_smalldatetime smalldatetime
PRIMARY KEY (id)
);
# Defining the runtime environment
env {
# You can set flink configuration here
execution.parallelism = 1
job.mode = "BATCH"
}
source{
Jdbc {
driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
url = "jdbc:sqlserver://localhost:1433;databaseName=column_type_test"
user = SA
password = "Y.sa123456"
query = "select * from full_types_jdbc"
}
}

transform {
# If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
# please go to https://seatunnel.apache.org/docs/transform-v2/sql
}

sink {
Console {}
}
```

### Parallel:

> Read your query table in parallel with the shard field you configured and the shard data You can do this if you want to read the whole table
> Read your query table in parallel with the shard field you configured and the shard data You can do this if you want to read the whole table

```
env {
# You can set flink configuration here
execution.parallelism = 10
job.mode = "BATCH"
}

source {
Jdbc {
driver = com.microsoft.sqlserver.jdbc.SQLServerDriver
url = "jdbc:sqlserver://localhost:1433;databaseName=column_type_test"
user = SA
password = "Y.sa123456"
# Define query logic as required
query = "select * from full_types_jdbc"
# Parallel sharding reads fields
partition_column = "id"
# Number of fragments
partition_num = 10
}
}

transform {
# If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
# please go to https://seatunnel.apache.org/docs/transform-v2/sql
}

sink {
Console {}
}

```

### Fragmented Parallel Read Simple:

Expand Down Expand Up @@ -151,7 +176,6 @@ source {


transform {

# If you would like to get more information about how to configure seatunnel and see full list of transform plugins,
# please go to https://seatunnel.apache.org/docs/transform-v2/sql
}
Expand Down