[TASK][EASY] JDBC Engine supports StarRocks

[banmoy]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before creating

• I have searched in the task list and found no similar tasks.

Mentor

• I have sufficient knowledge and experience of this task, and I volunteer to be the mentor of this task to guide contributors to complete the task.

Skill requirements

• Basic knowledge of Scala programming language
• Basic knowledge of JDBC concepts
• Basic knowledge of Docker and Testcontianers

Background and Goals

Currently, Kyuubi supports JDBC engines with limited dialects, and we can extend the dialects to support StarRocks, which is an open-source and high-performance analytical database.

Reference https://www.bilibili.com/video/BV1UT411m7LR

Implementation steps

• Implement a StarRocks dialect in the JDBC engine
• Leverage the Testcontainers to add integration tests
• Update document correspondingly

Additional context

No response

[github-actions]

Hello @banmoy,
Thanks for finding the time to report the issue!
We really appreciate the community's efforts to improve Apache Kyuubi.

[banmoy]

@pan3793 could you help to evaluate whether this issue can be added to the task list of Code Contribution Program? Thanks

[pan3793]

@banmoy thanks, task was added to the list

[banmoy]

Thanks a lot 😃

[pan3793]

@banmoy would you like to complete it by yourself, or leave it to other contributors?

[banmoy]

@pan3793 could first leave it to other contributors if they are interested in it

[bowenliang123]

Thanks for raising this task. We could split the task into several smaller subtasks to accomplish.
You could be assigned to this task first, and submit the relevant PR to it. @banmoy
Will try to help if there's somewhere you need a hand or review.

[bowenliang123]

Btw, consider using the official docker image from StarRocks for testing, which could be helpful and easy to maintain.
refer : https://docs.starrocks.io/en-us/latest/quick_start/deploy_with_docker

[phionex0571]

@pan3793 Hello, I have strong interests on it and during my daily work, Apache Kyuubi and StarRocks are two most useful tools. I think it can be finished in 14 days! please aisgn it to me~thx

[pan3793]

@phionex0571 Emm... you have been assigned 2 tasks, do you have sufficient free time to take the third and complete them in parallel?

[bowenliang123]

@banmoy Can I assign this task to you , and you can guide and cooperate with @phionex0571 ? In order to maximize the common knowledge from individuals and professionalism from SR team.

[phionex0571]

@phionex0571 Emm... you have been assigned 2 tasks, do you have sufficient free time to take the third and complete them in parallel?

@pan3793 Yeeep, in the previous year, I'd finished a StarRocks proxy in Apache Kyuubi 1.4, but that was not elegant and that was imitated like Apache Doris Engine implementation in Apache Kyuubi. And it's applied by 3 applications in production environment of my company. I hold strong belief that I'm able to finish this issue in time.

[banmoy]

I'm not sure I could start the work ASAP. I think the task can be assigned to other contributors first if they have big interests in it. I'm also glad to provide the knowledge of StarRocks if needed in discussion, review or test. WDYT? @bowenliang123

[bowenliang123]

Let's assign this task to both @phionex0571 @banmoy first.
Please raise the subtasks for this task. Small subtasks and improvements are always welcomed.
@phionex0571 please feel free to ask @banmoy about details of StartRocks, since he is from StarRocks team and has professionalism in Spark/Flink connector as well.

Also cc @zhaomin1423, the original contributor of the JDBC engine. They may need some guidance for this task.

Thanks to all.

[zhaomin1423]

Let's assign this task to both @phionex0571 @banmoy first. Please raise the subtasks for this task. Small subtasks and improvements are always welcomed. @phionex0571 please feel free to ask @banmoy about details of StartRocks, since he is from StarRocks team and has professionalism in Spark/Flink connector as well.

Also cc @zhaomin1423, the original contributor of the JDBC engine. They may need some guidance for this task.

Thanks to all.

Thanks you all, StarRocks is excellent and used widely, look forward.

[phionex0571]

@bowenliang123 @banmoy Hi, I've finished almost all the coding work and I have a question about StarRocks version in test cases. There're
several versions of StarRocks and which one should I choose to design unit tests, is it reasonable to use StarRocks 2.5.13? or 3.x?

[bowenliang123]

The latest stable released version of StarRocks is 3.1.x, which is reasonable to be tested with.
And feel free to raise the PR, even not unfinished. Just remember to mark the PR in the draft status.
By running CI tests, the failing tests and fixes bring it to more mutual.

[banmoy]

@phionex0571 Is it possible to make the unit tests run against the latest several StarRocks versions, such as 2.5, 3.0, and 3.1? So that we can tell users in the doc which version of StarRocks is supported, just like Spark/Flink/Trino? I think test cases should be same among different versions, but just use the specific docker image for each version.

[bowenliang123]

@banmoy It's not suggested to tested on multiple versions of StarRocks. As for other engines for Spark/Flink/Trino, we tested on different matrix combinations as Kyuubi use different version of API on these engines and has dedicated engine implementation and adaptations.
But as StarRocks introduced no major FE feature or breaking changes in MySQL protocol or dialect between 2.x and 3.x, and the JDBC engine uses the same version of Mysql Driver for connection, it's reasonable to test only on the latest stable version.

[banmoy]

@phionex0571 Hi, is there any progress on this task?

[bowenliang123]

@banmoy StarRocks support of dialects/connection/tests in the JDBC engine has been introduced in PR 5882 (#5882), if you may be concerned. Have check or validation on it.