*: support bazel

[hawkingrei]

What problem does this PR solve?

Issue Number: close #34083

Problem Summary:

What is changed and how it works?

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• bb7133
• tisonkun

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/c277dbcc9edbc53c5c7f147c1f23e2236e06e473

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[zz-jason]

could you explain more about why adding this support?

[hawkingrei]

could you explain more about why adding this support?

We run thousands of builds and tests in a day. But I find that we have not any local cache and remote cache to speed up ci. so I import the bazel into tidb to solve this problem. I believe that it can boost performance 10x.

[hawkingrei]

could you explain more about why adding this support?

if I meet an unstable test in the CI, I will just rerun this test case by the bazel.
if I have a test case that it just tests util package and util package have not any changes, we can use the remote cache to skip this test.

image.png

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[sre-bot]

Please follow PR Title Format:

• pkg [, pkg2, pkg3]: what is changed

Or if the count of mainly changed packages are more than 3, use

• *: what is changed

After you have format title, you can leave a comment /run-check_title to recheck it

[breezewish]

Just curious, if these BAZEL files are automatically generated, can they be git-ignored? It's usually not a good practice to include generated files in the git repository.

[hawkingrei]

Just curious, if these BAZEL files are automatically generated, can they be git-ignored? It's usually not a good practice to include generated files in the git repository.

In fact, Some bazel files are automatically generated. for example flaky test (It isn't recommended) and some other config have to be controlled by development such as visibility and test size that controls the test timeout. so it must be with the repo.

BTW, the open resource repos using the bazel like abseil-cpp, cockroach, tensorflow have many BAZEl.build files in their repo.

[breezewish]

Just curious, if these BAZEL files are automatically generated, can they be git-ignored? It's usually not a good practice to include generated files in the git repository.

In fact, Some bazel files are automatically generated. for example flaky test (It isn't recommended) and some other config have to be controlled by development such as visibility and test size that controls the test timeout. so it must be with the repo.

BTW, the open resource repos using the bazel like abseil-cpp, cockroach, tensorflow have many BAZEl.build files in their repo.

@hawkingrei Does this mean that we need to manually maintain these BAZEL.build files instead of using auto-generated ones, as the auto-generated ones may not be desired? I would like to know, for example, when I introduced a new package in the TiDB repository (or simply, added a new file), what steps should I take in order to make sure everything works? How these BAZEL files need to be generated? What kind of modification should I make? etc. If these steps are not property done, will my tests just being ignored in CI?

[breezewish]

Maybe you also need to introduce a CI step to ensure that people doesn't forget to generate necessary BAZEL files? I'm not sure however.

[hawkingrei]

Maybe you also need to introduce a CI step to ensure that people doesn't forget to generate necessary BAZEL files? I'm not sure however.

In fact, Generate bazel config is a task that takes some time in the CI. Althought it take less time than that tests without cache. so I will remove it in the future to speed ci again in the future.

However, Gezalle just can manage the dependency in the bazel config. it cannot manage the other thing. if we need to manage the config such as flasky test and so on. Gazalle just copy the old config to the new config. it is the main reason why we cannot remove bazel.build.

[hawkingrei]

Just curious, if these BAZEL files are automatically generated, can they be git-ignored? It's usually not a good practice to include generated files in the git repository.

In fact, Some bazel files are automatically generated. for example flaky test (It isn't recommended) and some other config have to be controlled by development such as visibility and test size that controls the test timeout. so it must be with the repo.
BTW, the open resource repos using the bazel like abseil-cpp, cockroach, tensorflow have many BAZEl.build files in their repo.

@hawkingrei Does this mean that we need to manually maintain these BAZEL.build files instead of using auto-generated ones, as the auto-generated ones may not be desired? I would like to know, for example, when I introduced a new package in the TiDB repository (or simply, added a new file), what steps should I take in order to make sure everything works? How these BAZEL files need to be generated? What kind of modification should I make? etc. If these steps are not property done, will my tests just being ignored in CI?

For the common developers, It will be fine that they use gazelle to generate the config. For the repo admin, it needs to control the other thing by changing the config in the bazel. in these cases, we need to change the config and save it, so that gazelle can get this changement and keep them in the new config if the dependency has been updated.

[bb7133]

LGTM

I had a few questions which are the same from @tisonkun , and I agree with the feedback, at least for now.

[bb7133]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 971ee85

[ti-chi-bot]

@hawkingrei: Your PR was out of date, I have automatically updated it for you.

At the same time I will also trigger all tests for you:

/run-all-tests

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.