Not all sandboxes in `ci.json` are built during PR reviewing

[joshuaellis]

🐛 bug report

Preflight Checklist

• I have read the Contributing Guidelines for this project.
• I agree to follow the Code of Conduct that this project
  adheres to.
• I have searched the issue tracker for an issue that matches the one I want
  to file, without success.

Description of the problem

Our complete list of sandboxes in react-spring is about 20. But only 10 seem to be built from the configuration, as seen in a recent PR but sometimes when there are more sandboxes in the issue or PR they are still built as can be seen in this PR.

Maybe i've done something wrong? Or maybe there's a maximum and it needs documenting?

How has this issue affected you? What are you trying to accomplish?

I'm unable to actually test against all the demo sandboxes we have.

To Reproduce

Make a PR in react-spring and you'll see.

Link to sandbox: link (optional)

Your Environment

Software	Name/Version
Сodesandbox
Browser
Operating System

[DeMoorJasper]

Thanks for the bug report, @lbogdan do you know why this is happening?

[joshuaellis]

Hey, Is there any update on this? It's a slight hinderance having to check each PRs sandboxes manually by pulling their branch 😞

[lbogdan]

Hey @joshuaellis ,

Sorry for the delay, I'll take a look at it this week!

[lbogdan]

@joshuaellis Indeed, we have a limit of 10 forked sandboxes per build, mostly as an anti-abuse measure.

I went over this with @CompuIves and decided to increase it to 25. I'll let you know when it's deployed.

[joshuaellis]

Ah I didn't know there was a limit, makes sense! Are the docs website OS? I don't mind making a PR to include this (or maybe it's already there and I missed it?)

[lbogdan]

It doesn't seem to be documented, no. The CodeSandbox CI documentation source is part of this repository, you can find it at https://github.com/codesandbox/codesandbox-client/blob/master/packages/homepage/content/docs/8-ci.md .

[lbogdan]

We just deployed the new limit of 25, I triggered a build on an open PR, you can check the result here: https://ci.codesandbox.io/status/pmndrs/react-spring/pr/1597/builds/149193 .

I also noticed what seems to be a typo here: https://github.com/pmndrs/react-spring/blob/aac4adaf2e948d165566bdae283b540135ec133e/.codesandbox/ci.json#L18 - looking at https://github.com/pmndrs/react-spring/tree/master/demo/src/sandboxes it seems like that should be trail instead of train.

[joshuaellis]

Ah great catch. Thank you for that. I'm submitting a PR for the doc updates this evening. Thanks for applying that update! ⭐

[joshuaellis]

Sorry to res this again, but this issue has cropped up again, barely any of the sandboxes are being made from the PR, see:

• https://ci.codesandbox.io/status/pmndrs/react-spring/pr/1886
• feat: add useReducedMotion hook & export useIsomorphicLayout hook pmndrs/react-spring#1886

[lbogdan]

Hey @joshuaellis ,

Just saw your tweet. Sorry, we completely missed your latest comment (in hindsight, we should've closed this back when we solved it).

Right now the pmndrs/react-spring triggers a repository size limit in our GitHub importer, most probably caused by the growth of the repository.

We're looking at ways to increase this limit and will get back to you shortly!

[joshuaellis]

Hey @joshuaellis ,

Just saw your tweet. Sorry, we completely missed your latest comment (in hindsight, we should've closed this back when we solved it).

Right now the pmndrs/react-spring triggers a repository size limit in our GitHub importer, most probably caused by the growth of the repository.

We're looking at ways to increase this limit and will get back to you shortly!

Thanks for getting back to me! No worries on the delay, it happens.

When you talk about size limit, is this around the size of the repo of the amount of sandboxes?

I'm just wondering if perhaps there's anything I can do in the meantime as a workaround??

If I'm honest, it's mainly when it builds the sandbox referenced in the issue that's most helpful for me. If it is about limitations on the amount of sandboxes I have added, I could reduce that.