[POC] refactor(monorepo): Monorepo implementation

[karelhala]

Fixes #201, #218

This is really specific PR because it reshapes entirely this repository, so be aware that after merging this several things might get broken.

Used technologies

Because monorepo is somewhat different than usual repository with just one NPM package we hove to somehow maintain build process and release process, after some research it looks like lerna is the key for this. It's used for instance in babel package.

Lerna basically allows us to run different scripts in different subdirectories (packages).

I introduced new package called @patternfly/react-core it's not released and either someone from PF team or me can do the registering on https://www.npmjs.com/org/create. This way we can use scoped packages for instance @patternflu/react-console, @patternfly/react-graphs, @patternfly/react-whatever

Steps to finish

• Use lerna to controll scripts in packages
• Running build wil create dist in each package folder
• Running test will test each package
• Running storybook will result in all packages exposing their stories in storybook
• Change commitizen to use new packages
• Change release to release with lerna (I'll probably need help on this one)
• Use correct commits - right now they are just to track which steps are done
• Update contributing guide

Questions

• Which release mode do we want? Fixed vs Independent
• Do we want to allow running test/build/storybook from each package?
• Where to generate codecov? Root directory or for each package?
• Where to store node_modules? I suggest storing them in root folder (so using .npmrc for each new package), but this might lead in a problem when one package might need different version of same package.

[karelhala]

@serenamarie125 @priley86 @jeff-phillips-18 I have couple of questions regardless of monorepo, it's mostly how to set things up for this repo.

One thing to note, this might break a lot of things in code so I would adwise merging this into different branch, releasing new package under different name (preferably use scoped package) and let all projects slowly use this new package while we catch all the bugs. In some point (preferably right after merging this) we have to stop developping new features in patternfly and release new features in new package only. In current one we have to release only fix PRs.

[coveralls]

Pull Request Test Coverage Report for Build 814

• 0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 70.404%

---

Totals
Change from base Build 804:	0.0%
Covered Lines:	1084
Relevant Lines:	1389

---

💛 - Coveralls

[coveralls]

Pull Request Test Coverage Report for Build 1072

• 0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 71.663%

---

Totals
Change from base Build 1043:	0.0%
Covered Lines:	1256
Relevant Lines:	1585

---

💛 - Coveralls

[karelhala]

Btw I was thinking about how to split this repo and what about this:

• @patternfly/react-core - core items, inputs, buttons, tables, tooltip, toolbar, navigation, labels etc
• @patternfly/react-form - form items, react-select (with some styles), date picker, modals
• @patternfly/react-filter - filter component, tagging once @PanSpagetka has it done
• @patternfly/react-terminal - serial console
• @patternfly/react-charts - basic and advanced charts, they might use http://www.reactd3.org/ and other libraries

This division is not set in stone, it's just and idea if you disagree and have another opinion, please say it so I can divide this repo as needed.

[priley86]

just trying to summarize on the conversation from yesterday (for others that were not at the meeting), and based on my current understanding...

• i think monorepo is still the long term solution that everyone wants. It scales much better for large repos supporting large products.
• @jeff-phillips-18 is following up on organization/IA of the new hierarchal structure. It does seem that splitting the dependencies by their individual dependency trees is good for reduced installs and prevents more potential cross dependency conflicts, however we still need to determine how that directory structure maps to PF Next and to our own design (ideally we can support good dependency structuring and good design structuring). Another question quickly becomes - is there a case for additional monorepos / repos for shared redux container components like your original issue mentions (but I think we have to defer this right now and focus on UI components and UI patterns here for now).
• we still do not have a solution for automated releases w/ Lerna. From reading #193 further, it looks like semantic-release-monorepo would be a short term option, however it would not support cases where you have cross dependencies in the same monorepo (package B depends on package A...so we need the automation to publish B then publish A). semantic-release will solve this problem formally later on...

do you want to explore Lerna automated publishing solutions in a playground and report back to us @karelhala ? I think this is vital before it gets introduced (b/c personally i don't want to do manual publishes here). I'm also in favor of maintaining deploy tokens as a security mechanism if possible...let me know if you need any help w/ this...

another question too here - should we convert to a yarn workspace as part of this PR?

[karelhala]

@priley86 sure I started to dig trough the settings of semantic releases at https://github.com/karelhala/lerna-example

another question too here - should we convert to a yarn workspace as part of this PR?

I don't think yarn workspaces are required for this. I'll try to set things up without them

About semantic releas - looks like we jst have to switch to lerna-semantic-release with cz-lerna-changelog - slightly different, it adds which packages are effect when committing.

About linking repo between one and each other, that's quite simple, that's what lerna lerna#bootstrap is for.

I'll test some more settings, like automated releasing, linking packages and basically settings of lerna.

[priley86]

@karelhala i believe a yarn workspace ensures that node_modules in a monorepo are shared at the top level (not sure if NPM would do this yet). Can you check on that? NPM5 aggregates node_modules for us at the repo level, but I'm not sure about the monorepo...

yes - please keep us posted on your research around lerna-semantic-release \ cz-lerna-changelog... I think continuing our commit based auto release is very important! 😸

maybe you can setup a playground repo that we can test some simple PRs with first?

[karelhala]

maybe you can setup a playground repo that we can test some simple PRs with first?

Sure thing, you can look at https://github.com/karelhala/lerna-example, feel free to open any new PR, I am currently figting with travis (he's refusing to publish to git)

yarn workspaces

I naively thought that .npmrc would solve this, looks like not. So we might have to use yarn workspaces after all.. It's not perfect as well tho. If we install package which depends on package-a@1.0.0 in one of our packages and in other we install different package which depends on package-a@2.0.0 we will have node_modules inside each ot these packages, with their corresponding versions of package-a. It's however better than before, but not perfect.

Commitizen

This is how cz-lerna-changelog looks like, it has different type of commit (some extra characters) and you have to pick which package is changed (which is to be expected).
Some extra characters in first step*

Pick which package is efffected

Final summarize

Release notes

Also since GH wouldn't know how to show release notes (or lerna semantic release is messing with them) there are no direct release notes which could be found right next to tag. But rather are stored in package folder.

[priley86]

@karelhala just wanted to provide an update here...

We have been reviewing both your documentation and your lerna sandbox further, and discussed it some this morning. The tentative plan is currently this:

• @dmiller9911 is looking to use lerna / lerna-semantic-release mostly as you have described. He is planning to create a second sandbox that would map a few of the components existing here in pf-react today directly into the new structure. The idea being, this would become the exact model once finished and merged into the repo.
• A few minor changes to your proposal would be:
  1. We would like to keep patternfly-react as an “export all” package to ensure we do not break existing consumers (even after repo is converted to new structure we can still provide it as a separate package), phasing the approach.
  2. Once converted to the new structure, we will begin exporting each component individually as a package (like you have done). The means consumer would need to split import {Button, Filter} from "patternfly-react" into two imports downstream: import {Button} from "@patternfly-react/button” , “import {Filter} from @patternfly-react/filter” but we plan to write a React codemod to make the upgrade seamless for those who choose to do this. This will have major performance implications and help us prepare for future components in the repo.
  3. Additionally - I think we can begin investigating generators for scaffolding new packages in the monorepo.

@dmiller9911 is planning to demo his progress at the upcoming PF React Community Meeting for anyone with questions/ideas/suggestions.

[priley86]

more updates... for anyone that was not able to attend today's Patternfly-React Contributors Monthly Exchange, some more discussion notes were posted here:
https://docs.google.com/document/d/1VJRcGmdICVYpX-osVdMmqPkvFp_oAn5RrarEroRIYfU/edit#
and here:
https://docs.google.com/document/d/1zWYa1KKv7rlC_vdb9Vza_iOP4vGJdVgX_jKlBV3Tp8k/edit

[jeff-phillips-18]

I believe this can be closed with the merge of #332