chore: modernize workspaces setup using yarn

[JamesHenry]

Hi Folks 👋

I'm James, you might possibly know me already from having created https://github.com/typescript-eslint/typescript-eslint or my work on some other popular open-source tools over the years such as ESLint, Prettier and Babel. Right now I've got my Lerna maintainer hat on 🐉

I am here after reviewing @maribethb issue on the lerna repo.

I spent a significant amount of time getting to know your repo and how it is currently set up and I have applied the changes I think best set you up for future success.

Please see the video explanation below for full details.

Explanation of changes

This walkthrough ended up being a bit of a beast! But hopefully you can watch it on a greater than 1x speed 😄

Please let me know if anything is unclear!

google_blockly-samples.mp4

---

Additional FYIs

There are a few issues with examples on latest master right now (which are therefore still present in this PR):

• examples/blockly-requirejs only works if you patch the define() calls within the blockly package to remove trailing .js references that exist on some values (but not all)
• examples/blockly-rtc generates examples/blockly-rtc/eventsdb.sqlite which is not ignored by git, shouldn't it be?
• examples/blockly-vue3, just throws at runtime Uncaught TypeError: Cannot set properties of undefined (setting 'stock_buy_simple')

Multiple examples do not work with Node 18, which is already the LTS version of node. They likely just require dependency updates but that wasn't in scope for me here. The error you will see in each case is Error: error:0308010C:digital envelope routines::unsupported

[maribethb]

I'm working on a more coherent response, but for now let me say thank you!! This is incredible, my whole team is currently watching the video with our jaws on the floor that you put this much work into our repo.

I have known about workspaces for a while but not invested time into learning more or facing the daunting task of updating this multi-level monorepo to use them. This is an incredible starting point and I'm excited to do more research. Thanks again!

[JamesHenry]

@maribethb You are very welcome! 😄

Best of luck and feel free to reach out again if anything comes up, particularly if related to lerna of course.

One thing we do for OSS repos is offer 100% free usage of Nx Cloud, which acts as a remote distributed cache for any and all tasks you want to set up as cacheable with the lerna task runner.

Additionally it is also able to act as a coordinator for all processes using what we call Distributed Task Execution (DTE). I'm sure you'll be very used to these features from Blaze!

More info here: https://nx.dev/nx-cloud/intro/what-is-nx-cloud

[maribethb]

Hi @JamesHenry,

Apologies for the long delay; I know it's been a while since you looked at our repo. I had to do a lot of research to make sure I understand all the pieces of this well enough to maintain it going forward, plus all the stuff that was already on my roadmap for the quarter :) We are going to merge this, or a version of this, but I had a few questions I was hoping you could answer because I'm still not sure on:

• Is there an easy way you generated the yarn lock file? Now that we've done a bunch of dependabot updates it'll sadly be out of date.
• I'm not 100% sure of why exactly we need the hoisting limits. I thought that if two different workspaces needed incompatible versions of something, then one would be hoisted, but the other packages that need a different one would still get their own copy of it. I don't think this is a problem for us, I'm just trying to understand why this is needed. Any additional insight you have is appreciated; I've read various articles about the hoisting limits but still unsure of this point.

Thank you!

---

More complicated question/remark about hoisting limits / notes to future me: I am not sure if we will get to take advantage of hoisting dependencies of sibling plugins that depend on each other. Hoisting the blockly devDependency while working on a plugin would be super helpful to us and avoid problems where dev-tools has its own copy of blockly separate from the plugin's. I am not sure whether blockly would actually be usefully hoisted though.

• plugins/dev-tools
  • depends on blockly
• plugins/foo
  • depends on plugins/dev-tools
    • depends on blockly
  • depends on blockly

So blockly should be hosted up to the plugins/foo workspace-level, where it satisfies both dev-tools and foo. At least, that is what I hope will happen. I guess I am not confident about that, because I am not sure how it works when one of the dependencies is a sibling workspace. If all we do is symlink over to the plugins/dev-tools workspace (which is what currently happens using legacy lerna bootstrap) then we would not be able to take advantage of hoisting of blockly, because that symlinked directory would have its own copy of blockly and the node_modules of the two are not in parent directories so hoisting wouldn't work unless it occurred at the level of the plugins/ directory or higher.

I'm hoping my question is understandable there. I think since nested workspaces are supported now in yarn, we could potentially set things up so that the top level there are two workspaces, examples and plugins, then each of those directories also specifies all of its children as workspaces. That way we could let plugins hoist up to the plugins level, but keep examples as isolated. I can't find a lot about using nested workspaces though. I also found this stack overflow post which implies that we could perhaps set a configuration file in the examples/ directory that limits hoisting just for the examples directory, but leave it enabled for the plugins directory, which would be preferable.

This is a more niche topic so if you don't have any experience with this part, that's totally fine too, and something we can continue to figure out on our own. But I figured I'd ask while I have you here as monorepos in particular are more likely to run into this kind of setup.

[maribethb]

We are planning to modernize our workspaces setup in the nearish future but it will take a bit different form than this PR. Thank you again for all of this work, it was really useful to us as we explored what our next steps should be.