Skip to content

ci: Introduce Nx agents#6863

Merged
lachlancollins merged 15 commits intoTanStack:mainfrom
ZackDeRose:nx-agents
Feb 10, 2024
Merged

ci: Introduce Nx agents#6863
lachlancollins merged 15 commits intoTanStack:mainfrom
ZackDeRose:nx-agents

Conversation

@ZackDeRose
Copy link
Contributor

@ZackDeRose ZackDeRose commented Feb 9, 2024

Playing with Nx agents here.

Here's what I'm seeing:

  • We can do a bit better by bumping up the agent count from 3 => 12 (free and not as restrictive as github seems to be when asking for a bunch of machine), and reducing the parallel option from 5 => 3 to not overload any one machine. Looks like now we're getting 3.5 minutes for a full run. We might try for more agents here, it's worth exploring!

  • A downside from the legacy DTE setup to Nx agents is in legacy, we could setup workers and main machines all in parallel, now we have to setup the main job and then setup all other agents. We're losing about 20s here with cached node_modules - looks like this is made up for by bumping up agent counts though. When we launch Nx Workflows, we should be able to get that 20s back.

  • There's some more optimizations we could try later, adding in small/med/large agent loads could help here for getting the full swarm when we need to run everything, and not get overly excited for smaller PRs

  • Potentially we can add in the format and knip tasks to the swarm. This is probably trivial for perf, but it could help in the sense that all tasks can be run the same way (instead of setting up specific machines for the format and knip tasks), and we could see it in the nx cloud task list instead of spelunking github action logs!

  • Nx Agents can be used for the main job as well - looks like those workflows are growing up to ~10 minutes now. I think now it's solid enough that we can use it reliably for the releases, while using the --skip-cache option to ensure we're running all checks before publishing!

@lachlancollins / @TkDodo let me know thoughts here, I'll clean this up a bit so we can merge if you like!

@vercel
Copy link

vercel bot commented Feb 9, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
query ⬜️ Ignored (Inspect) Visit Preview Feb 10, 2024 11:22pm

@nx-cloud
Copy link

nx-cloud bot commented Feb 9, 2024

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 1fdf144. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution


✅ Successfully ran 1 target

Sent with 💌 from NxCloud.

@codesandbox-ci
Copy link

codesandbox-ci bot commented Feb 9, 2024

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 1fdf144:

Sandbox Source
@tanstack/query-example-angular-basic Configuration
@tanstack/query-example-react-basic-typescript Configuration
@tanstack/query-example-solid-basic-typescript Configuration
@tanstack/query-example-svelte-basic Configuration
@tanstack/query-example-vue-basic Configuration

@codecov-commenter
Copy link

codecov-commenter commented Feb 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7a51c2a) 41.82% compared to head (899075b) 41.82%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6863   +/-   ##
=======================================
  Coverage   41.82%   41.82%           
=======================================
  Files         178      178           
  Lines        7015     7015           
  Branches     1416     1416           
=======================================
  Hits         2934     2934           
  Misses       3718     3718           
  Partials      363      363           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ZackDeRose
Copy link
Contributor Author

@lachlancollins / @TkDodo we're in a merge-able state now

@lachlancollins lachlancollins changed the title introducing nx agents ci: Introducing Nx agents Feb 10, 2024
@lachlancollins lachlancollins changed the title ci: Introducing Nx agents ci: Introduce Nx agents Feb 10, 2024
@lachlancollins lachlancollins merged commit 59c0d53 into TanStack:main Feb 10, 2024
@lachlancollins
Copy link
Member

lachlancollins commented Feb 13, 2024

@ZackDeRose regarding the optimisations you mentioned above, how difficult would it be to implement agents for the ci.yml workflow? As you mentioned, CI times are now 9+ mins as we're testing examples too. I don't think it would be necessary to skip cache if it's already been tested in the exact same setup in a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants