Enable Phoenix product test

[virajjasani]

Description

Enable Phoenix product tests.

HBase 2.5.10 has a fix required to stabilize Phoenix tests in the docker environment. Since Phoenix 5.2.1 with HBase 2.5.10-hadoop3 are already used by Trino, the fix is available for trino to consume.

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

@wendigo could you please take a look?

[ebyhr]

Could you run stress tests likes #21480? Removing unnecessary jobs and duplicating a Phoenix job.

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

Could you run stress tests likes #21480?

Could you please help me understand how I can run it? So far, I confirmed the changes with
testing/trino-product-tests-launcher/bin/run-launcher test run --config config-default --environment multinode-phoenix5 -- -g configured_features,phoenix

[ebyhr]

@virajjasani I mean removing unnecessary jobs and duplicating a Phoenix job (10 lines) in ci.yml.

[ebyhr]

Is #21569 a client side issue? Our testing target version is still 5.2.0. https://github.com/trinodb/docker-images/blob/a8bd3d297cbf23ea815856cb43901632f23ed677/testing/phoenix5/Dockerfile#L18

[virajjasani]

Is #21569 a client side issue?

No, it was server side issue, which is fixed with HBase 2.5.9 (Jira: HBASE-28567)

[virajjasani]

Our testing target version is still 5.2.0.

Thanks for pointing this! Can i update it in addition to this PR?

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

Upgrade Phoenix version in docker image to 5.2.1: trinodb/docker-images#220

[virajjasani]

@ebyhr I guess I might have removed even some necessary job, all the trino-phoenix5 jobs are failing. Probably I can restore everything back and still repeat trino-phoenix5 at least 20 times for stress testing?

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

@ebyhr @lhofhansl The build result of stress test looks good, let me revert the changes and make PR ready for review.

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[ebyhr]

@virajjasani You should run product tests pt, not integration tests test because you updated testing/trino-product-tests/src/main/java/io/trino/tests/product/phoenix/TestPhoenix.java

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

@virajjasani You should run product tests pt, not integration tests test because you updated testing/trino-product-tests/src/main/java/io/trino/tests/product/phoenix/TestPhoenix.java

I am bit confused. It seems Trino does not have SuitePhoenix* or suite-phoenix* in ci.yaml. Also, #21569 build results were also focused on trino-phoenix5 unless I am mistaken. I might be missing something fundamental here.

[ebyhr]

@virajjasani You can check Suite6NonGeneric.

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

@ebyhr Thanks for the pointers!
I have checked the output of Suite6NonGeneric and Phoenix test was included. Could you please also help verify if the stress test looks good to you? I can increase the num of suite runs, they were 10 this time and each one seems to have taken close to 34 min.

[ebyhr]

@virajjasani Thank you for your work! The test result looks good to me. Could you squash commits into one?

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[virajjasani]

Done, squashed into single commit.

[virajjasani]

Btw I submitted ICLA yesterday just while creating this PR.

[ebyhr]

Run Phoenix product test after 5.2.1 upgrade

You can change the commit title to "Enable Phoenix product test".

[hashhar]

@virajjasani You renamed the PR title only, please also update the commit message. Looks good to merge.

[virajjasani]

Done. Thank you @ebyhr @hashhar!!

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[lhofhansl]

Looks good.

Thanks for thinking about this @virajjasani, I was thinking about pining you on this and saw that you already did it. :)

We should move to HBase 2.6.1 now (which also has this fix), but that's for a different PR.
And when Phoenix and Hadoop 3.4.1 are working together reliably (ATM they do not) we should move to that as well, also for a different PR.

[virajjasani]

We should move to HBase 2.6.1 now (which also has this fix), but that's for a different PR.
And when Phoenix and Hadoop 3.4.1 are working together reliably (ATM they do not) we should move to that as well, also for a different PR.

Ack, these are the next targets.
On a separate note, Phoenix 5.3 is expected to be ready in a couple of months and it comes with many new features. For Phoenix 5.3, using default HBase 2.6 profile would also be nice.

Also, now that we have Phoenix 5.2 already, I am thinking about enabling pagination with very low page size by default. I saw that hbase-site and phoenix properties are defined for multinode test setup, need to figure out how we can enable some properties by default.

[hashhar]

@cla-bot check

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

The cla-bot has been summoned, and re-checked this pull request!