Description
Is your feature request related to a problem? Please describe.
Hi, Team,
Recently I want to try Pulsar CDC Source Connector and JDBC Sink Connector for PGSQL to sink multiple table data between PGSQLs.
But I find that the JDBC sink connector for PGSQL is configured per-target-table basis; it seems that I can not sink several different target tables in a config file.
Although I can set up multiple configuration files for each target tables (the same target PGSQL database), this might cause the target side PGSQL pool performance due to many occupied connections.
In PGSQL, it is always a good practice to utilize connect pool when many client-side connections are needed.; and this might also holds for other databases (e.g. Cassandra).
I think Pulsar can provide JDBC connection pool for all supported JDBC sink connectors and other sink connectors if possible.
https://jdbc.postgresql.org/documentation/head/datasource.html
https://docs.datastax.com/en/developer/java-driver/4.14/manual/core/pooling/
https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-usagenotes-j2ee-concepts-connection-pooling.html
https://mariadb.com/docs/connect/programming-languages/java/connection-pools/
Describe the solution you'd like
If my understanding is correct, Pulsar IO connectors are controlled by pulsar-admin tool, it would be great to create connection pool for specified connector with some option of pulsar-admin sinks command.
- Providing a subcommand
pulsar-admin sinks poolto create a connection pool resource pulsar-admin sinks poolmay provide--nameand--pool-config, etc, just likepulsar-admin sinks createsuboption.- Providing a new JDBC sink configuration property
connpoolto link with pre-configured connection pool.
Additional context
As far as I know, in kafka connect there is no such feature available.
If Pulsar can provide such database performance-friendly feature, this feature can be attractive for new users.
Best Regards.