Remove ansible import functionality #2565

dveeden · 2025-07-26T07:31:23Z

What problem does this PR solve?

What is changed and how it works?

This removes the functionality to import cluster and dm setups that were setup with Ansible (so pre-tiup). I don't think the ansible import is used much or at all now and it also doesn't have much testing.

This gets rid of two dependencies (which make packaging easier)

github.com/relex/aini (direct)
github.com/google/shlex (indirect)

It also helps a bit with #2554

It also makes maintenance of tiup easier.

Ansible related issues: https://github.com/pingcap/tiup/issues?q=is%3Aissue%20state%3Aopen%20ansible%20label%3Aansible

Note that https://github.com/pingcap/tidb-ansible has been archived on Jun 24, 2021.

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

Need to cherry-pick to the release branch
Need to update the documentation

Release notes:

Ansible import functionality has been removed

ti-chi-bot · 2025-07-26T07:31:30Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign xhebox for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

dveeden · 2025-07-26T07:31:47Z

/cc @xhebox

codecov-commenter · 2025-07-26T07:32:45Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.03%. Comparing base (97a3aa7) to head (fffcd37).

⚠️ Current head fffcd37 differs from pull request most recent head 4590f59

Please upload reports for the commit 4590f59 to get more accurate results.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2565      +/-   ##
==========================================
+ Coverage   42.15%   49.03%   +6.88%     
==========================================
  Files         336      336              
  Lines       36363    36358       -5     
==========================================
+ Hits        15327    17827    +2500     
+ Misses      19124    16355    -2769     
- Partials     1912     2176     +264

Flag	Coverage Δ
cluster	`36.97% <ø> (+11.58%)`	⬆️
dm	`26.83% <ø> (+7.48%)`	⬆️
tiup	`33.95% <ø> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

xhebox · 2025-07-27T15:52:27Z

Hmm, this is a big change. I would like to leave it for the next major version.

@wuhuizuo

* pkg/set: convert check to require (#2557) * meta: replace pingcap/check with testing (#2559) * embed: replace pingcap/check with testing (#2555) * base52: replace pingcap/check with testing (#2556) * localdata: replace pingcap/check with testing (#2561) * utils: replace pingcap/check with testing (#2562) * telemetry: remove (#2564) * Telemetry removal * Remove left over teleCommand * repo: exclude prerelease version from version list if possible (#2550) Signed-off-by: xhe <xw897002528@gmail.com> * repo: abstract v1repoapi as interface (#2542) * env: clean up old code Signed-off-by: xhe <xw897002528@gmail.com> * repo: leverage all apis into v1_repository Signed-off-by: xhe <xw897002528@gmail.com> * repo: abstract it as an interface Signed-off-by: xhe <xw897002528@gmail.com> * clean the remained part Signed-off-by: xhe <xw897002528@gmail.com> * rename to LocalComponentInstalled Signed-off-by: xhe <xw897002528@gmail.com> * update Signed-off-by: xhe <xw897002528@gmail.com> * try fix Signed-off-by: xhe <xw897002528@gmail.com> * g Signed-off-by: xhe <xw897002528@gmail.com> * try fix Signed-off-by: xhe <xw897002528@gmail.com> * g Signed-off-by: xhe <xw897002528@gmail.com> * g Signed-off-by: xhe <xw897002528@gmail.com> * g Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * cluster: also check exporter ports (#2563) Signed-off-by: xhe <xw897002528@gmail.com> * cluster: improve SELinux compatibility (#2501) * cluster: improve SELinux compatibility * fixup * cluster: update --ssh desc (#2380) * Update --ssh desc * Update components/cluster/command/root.go --------- Co-authored-by: xhe <xw897002528@gmail.com> * cluster: add stop/start timeout for systemd service (#2568) Signed-off-by: xhe <xw897002528@gmail.com> * *: move insight to tiup (#2553) * Move insight to tiup * Fix linting issues * Fix unused/gocritic/staticcheck issues * *: check_to_require for cluster (#2566) * check_to_require: cluster * Restore TestCountDirMultiPath --------- Co-authored-by: xhe <xw897002528@gmail.com> * *: update dmpb (#2569) * Update dmpb * Remove dm grpc gw * *: replace otiai10/copy with os.CopyFS (#2571) * ci: add staging mirror option to release workflow (#2576) * ci: fix STAGING version competition (#2579) * playground: fix config typo (#2578) Signed-off-by: Ryan Leung <rleungx@gmail.com> * ci: add ci for feature branch (#2577) Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * ci: fix staging version string (#2582) * ci: fix STAGING version competition * ci: replace slashes with dots in branch name for STAGING_VER * Apply suggestion from @wuhuizuo * playground: fix tiflash metric (#2587) Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * *: Replace github.com/alecthomas/assert with github.com/stretchr/testify/require (#2570) * Switch from yaml.v2 to yaml.v3 (#2558) * Switch from yaml.v2 to yaml.v3 * Use KnownFields * Use KnowFields for a few more places that previously had UnmarshalStrict * Fix issues found by make check * Formatting differences in yaml * update * fix CI Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> Co-authored-by: xhe <xw897002528@gmail.com> * cluster: support new ready api for PD (#2592) * cluster: support new ready api for PD Signed-off-by: xhe <xw897002528@gmail.com> * try fix Signed-off-by: xhe <xw897002528@gmail.com> * try fix Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * cluster: clear leader_priority for pd leader if exist (#2591) * cluster: clear leader_priority for pd leader if exist Signed-off-by: xhe <xw897002528@gmail.com> * fix Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * cluster: prompt for instance restart (#2513) * cluster: prompt for instance restart * Add comment * fixup * fixup * Split precheck from Upgrade * bump version to 1.16.3 (#2593) Signed-off-by: xhe <xw897002528@gmail.com> * *: Modernize (#2574) * cluster: prevent panic when the operation is not upgrade (#2598) Signed-off-by: xhe <xw897002528@gmail.com> * *: fix unclosed mirror (#2607) * *: fix unclosed mirror Signed-off-by: xhe <xw897002528@gmail.com> * fix Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * playground: prepare code for nextgen support (#2608) * playground: prepare code for nextgen support Signed-off-by: xhe <xw897002528@gmail.com> * fix tikv Signed-off-by: xhe <xw897002528@gmail.com> * fix CI Signed-off-by: xhe <xw897002528@gmail.com> * fix CI Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * playground: add support for nextgen (#2619) * playground: add support for nextgen Signed-off-by: xhe <xw897002528@gmail.com> * s Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * playground: hack around nextgen version resolve (#2621) Signed-off-by: xhe <xw897002528@gmail.com> * playground: fix monitor version for nextgen (#2622) Signed-off-by: xhe <xw897002528@gmail.com> * Remove ansible import functionality (#2565) * Remove ansible import functionality * update go.sum * playground: rename tidb-nextgen to tidb-x (#2623) * playground: rename tidb-nextgen to tidb-x Signed-off-by: xhe <xw897002528@gmail.com> * fix lint Signed-off-by: xhe <xw897002528@gmail.com> * fix lint Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * repository: delete temporary dir for downloading (#2626) Signed-off-by: xhe <xw897002528@gmail.com> * utils: Make UserHome() respect the HOME env variable (#2624) * Fix minor typos (#2601) * bump to v1.16.4 (#2627) Signed-off-by: xhe <xw897002528@gmail.com> * *: do not trigger release for release-x branch (#2629) * playground,cluster: respect HOME env variable (#2628) * playground,cluster: respect HOME env variable * fix import * Update golang.org/x/net (#2631) * tiup: fix recording of execution history (#2635) * Fix recording of execution history * Improve filtering of sensitive info and add tests * Add test for dumpling style password * hide sensitive dmctl info from execution history * Add redaction of sensitive info in URLs * Fix function name * Add test to increase coverage * Also strip access-key from S3 URIs * fix test Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> Co-authored-by: xhe <xw897002528@gmail.com> * *: use full env only when TIUP_CLUSTER_DEBUG is true (#2637) * logger: use TIUP_CLUSTER_DEBUG to set zap level (default INFO) * env printing: whitelist in non-debug; full env in DebugMode; centralize EnvList in environment; refactor callers * docs: clarify EnvList reference to pkg/environment/env.go for env visibility control * Switch to codecov-action (#2639) * playground: refine nextgen version resolve (#2640) * playground: refine nextgen version resolve Signed-off-by: xhe <xw897002528@gmail.com> * g Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * CI: update and fix integrate-dm (#2643) * playground: fix metafile location (#2638) * playground: fix metafile location * g Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> Co-authored-by: xhe <xw897002528@gmail.com> * Upgrade usql (#2595) * cluster: fix instance level timeout stop (#2644) Signed-off-by: xhe <xw897002528@gmail.com> * playground: dont request mirror when binpath specified (#2645) * playground: dont request mirror when binpath specified Signed-off-by: xhe <xw897002528@gmail.com> * add tikv worker Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * playgroud: support tiflash in nextgen (#2641) * playground: refine nextgen version resolve Signed-off-by: xhe <xw897002528@gmail.com> * playgroud: support tiflash Signed-off-by: xhe <xw897002528@gmail.com> * fix version Signed-off-by: xhe <xw897002528@gmail.com> * fix Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> * cluster: cleanup ansible import related things (#2630) * cluster: cleanup ansible import related things * revert back missing code Signed-off-by: xhe <xw897002528@gmail.com> * fix test Signed-off-by: xhe <xw897002528@gmail.com> * remove ansible test Signed-off-by: xhe <xw897002528@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> Co-authored-by: xhe <xw897002528@gmail.com> * *: cleanup unit test generated data (#2647) * *: add option to force redownload binaries (#2648) Signed-off-by: xhe <xw897002528@gmail.com> * playground: fix nextgen install (#2649) * fix: merge master and fix build errors * fix: resolve merge conflicts and fix build errors * fix: resolve merge conflicts and fix build and linting errors * fix * fix * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> * fix Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> --------- Signed-off-by: xhe <xw897002528@gmail.com> Signed-off-by: Ryan Leung <rleungx@gmail.com> Signed-off-by: Lloyd-Pottiger <yan1579196623@gmail.com> Co-authored-by: Daniël van Eeden <git@myname.nl> Co-authored-by: xhe <xw897002528@gmail.com> Co-authored-by: glkappe <qihang.li@pingcap.com> Co-authored-by: wuhuizuo <wuhuizuo@126.com> Co-authored-by: Ryan Leung <rleungx@gmail.com> Co-authored-by: Otto Kekäläinen <otto@kekalainen.net> Co-authored-by: Michael Zhang <90968483+mzhang77@users.noreply.github.com> Co-authored-by: Allen Zhong <pdev@zhoal.pw>

Remove ansible import functionality

4590f59

ti-chi-bot bot requested a review from srstack July 26, 2025 07:31

ti-chi-bot bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jul 26, 2025

ti-chi-bot bot requested a review from xhebox July 26, 2025 07:31

dveeden added 3 commits August 2, 2025 10:24

Merge remote-tracking branch 'upstream/master' into ansible_removal

abf2e75

update go.sum

b227d69

Merge remote-tracking branch 'upstream/master' into ansible_removal

283a963

xhebox merged commit 2b87bad into pingcap:master Oct 27, 2025
15 of 21 checks passed

dveeden mentioned this pull request Oct 30, 2025

cluster: cleanup ansible import related things #2630

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove ansible import functionality #2565

Remove ansible import functionality #2565

Uh oh!

dveeden commented Jul 26, 2025 •

edited

Loading

Uh oh!

ti-chi-bot bot commented Jul 26, 2025

Uh oh!

dveeden commented Jul 26, 2025

Uh oh!

codecov-commenter commented Jul 26, 2025 •

edited

Loading

Uh oh!

xhebox commented Jul 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Remove ansible import functionality #2565

Remove ansible import functionality #2565

Uh oh!

Conversation

dveeden commented Jul 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how it works?

Check List

Uh oh!

ti-chi-bot bot commented Jul 26, 2025

Uh oh!

dveeden commented Jul 26, 2025

Uh oh!

codecov-commenter commented Jul 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

xhebox commented Jul 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dveeden commented Jul 26, 2025 •

edited

Loading

codecov-commenter commented Jul 26, 2025 •

edited

Loading