Carbon Monorepo #5
Conversation
|
This is great! (1) Should the RFC mention conventional commits and standardized changelog generation? (2) If we're using GitHub teams and CODEOWNERS for merge access control, wouldn't it be easier if all React components were in a (3) Notifications - with GitHub teams and CODEOWNERS, could we auto-assign teams for PR review based on touched files in PR, which would trigger notifications for the correct people? (4) Backlog - should we mention how we can configure issue templates to auto-apply labels? Anything we can do with notifications on new issues? (5) With community-contributed components living alongside core components... let's say somebody creates |
|
(6) Could we configure Circle CI conditionally run test suites based on which files have been touched? E.g., if doing a PR for React components, to only run relevant tests or would our CI need to run every test suite in the monorepo? (7) Random thought... all core and community-contributed components living in one list feels very "app store", where for a new component to be accepted it needs to meet our high bar of component acceptance criteria. Being in one list, component names need to be unique. Back to my above "hero" component example... if somebody wants to contribute a hero component and grab the name... better contribute now before somebody beats you to contributing their "hero". I like that. |
|
This is an tremendous amount of work and a great insight into the problem, thanks so much for putting it together. I'm gonna put first impressions then refine and add to them. The more I let the ideas sync in the more excited I get. It's really ambitious but I think executed well has the opportunity to be a huge leap forward in component libraries.
|
|
Thanks @joshblack for writing this up - Basically my biggest thoughts on this topic is around backward-compatibility with Wrt the toolstack, if we want to go with Storybook across the board, we need to see how to ensure the fast style development with it. Storybook's underlying WebPack is known as slow for building styles (refs: webpack-contrib/sass-loader#307 and webpack-contrib/sass-loader#296 (comment)). Another approach there would be custom dev env for style development only. |
|
Overall, this sounds pretty good - my main concerns are with noise from everything else happening in the repository and the level-of-effort and logistics from an add-ons perspective for transitioning to a newly-defined development process and distribution method. |
|
@vpicone on DefinitelyTyped issues... they seem to be pulling it off with 2.7k open and 2.8k closed. What part of their issues do you like? Their issue naming convention? E.g. |
|
Following up on DefinitelyTyped, thought it was insanely cool how they generate |
|
Super pumped about this idea!!! From an addon's owner perspective here are a couple of thoughts.
|
I believe
Yup!
Just added: https://github.com/carbon-design-system/rfcs/blob/monorepo-rfc/text/0000-monorepo.md#github to start to address this
I think on major version bumps every 6 months we could look across our packages and note the following:
With these questions answered, we could make a determination based on the follow to deprecate or not:
There are a couple of options:
|
What kind of pain are you expecting with this? The colocation is more to try and influence every button variant to be similar. As soon as they are no longer siblings, things can tend to drift just because they aren't next to each other.
I think the goal will be to offer web components as a common base for anyone, including frameworks, but the decision to use them will lie exclusively with the framework. Benefits of using them would mostly include:
Drawbacks would include lack of access to framework-specific features, at least in the case of React for things like Suspense. Ideally, our spec would be the source of truth that everything falls under but until then we'll definitely be transitioning over to React from vanilla like you said 👍
100% agreed. I think their issue strategy and CODEOWNERS tool are things we absolutely should try and do. |
Well those issues are for every single third party type in javascript so that number is reasonable. Being able to find/create issues for a specific package when there's dozens/hundreds in the repo is a tough problem. Like I said, it's not perfect but it's something for us to consider. |
|
@joshblack I suppose once all of the imports are moved from relative to package/workspace based it wouldn't really matter. I was thinking from a practical standpoint where a lot of times you'll be jumping between components for compound components and the directory being cluttered with other libraries could make it tough to navigate (especially for new contributors). If we have to choose grouping by library, or grouping by component, I'd say the more common dev experience is going to be working in a single library, not alternating between libraries. We should aim to make that experience as smooth as possible while still enabling the x-library dev. |
|
It looks like we'd have a package for our global styles (reset, helpers, etc... non-component styles that don't come from a Let's say we have two groupings of components: core and community. Components have their own component-specific Sass source, which imports helper Sass mixins from the styles package. Let's say we need to do a Sass breaking change, or we want to deprecate a helper mixin. That'd be easy to update all uses of that mixin for core components. But what about community mixins? For components that the core team doesn't maintain... how would we remove something without breaking them? Would we effectively have 3 groupings of components: core, community, graveyard (probably a better name) where community components would be sunset if they don't migrate breaking changes during the ~6 month deprecation windows between deprecating something and removing it the following major release? |
|
Another topic: CLI! @kuehndaniel says we can use Developer Certificate of Origin (DCO) for external contributors, if that's easier. GitHub has a bot that enforces DCO on pull requests by requiring all commit messages to contain the You can configure DCO bot to skip sign-off or organization members. Source There's no way to skip all commit messages that include With monorepo, we'll be adding every maintainer to the But what happens when we want an external maintainer for some component? At that point they would have to be part of the org, thus bypassing the commit signoff requirement... Is that a hard blocker for how we could use DCO? Or would we require all committers to sign their commits, IBMers included? UPDATE Could we add a Git hook that automatically adds the sign-off message? We would just need to inform contributors to make that file executable prior to committing? E.g.: https://github.com/zalando/zalando.github.io/blob/9d7cbdf1e5e5b27838899ef685838460cdb69b6a/_docs/releasing/contributions.md#automatic-git-sign-off |
There was a problem hiding this comment.
@joshblack this is a fantastic concept. It seems like it would be a massive repo, but could ease a lot of the burden of keeping track of so many dependencies and CI functionality.
I've added some (lengthy as usual) comments below related to component directory structure and contents. Hope this helps!
cheers, scott
text/0000-monorepo.md
Outdated
|
|
||
| - The Vanilla.js implementation of the component, this is to facilitate sharing with frameworks | ||
| - The style representation of the component, initially this would be Sass but could in theory be multiple formats that are offered | ||
| - A specification for the component (where applicable) |
text/0000-monorepo.md
Outdated
| - The Vanilla.js implementation of the component, this is to facilitate sharing with frameworks | ||
| - The style representation of the component, initially this would be Sass but could in theory be multiple formats that are offered | ||
| - A specification for the component (where applicable) | ||
| - Metadata around the component (could be a generated artifact) to be consumed by websites to aid in search result ranking |
text/0000-monorepo.md
Outdated
| In theory, we could support component packages for every framework implementation. As a result, the following packages would be siblings of each other: | ||
|
|
||
| ``` | ||
| packages |
There was a problem hiding this comment.
Would it be possible to do something like this:
button/
├── button-angular
│ └── package.json
└── tests (framework-specific)
├── button-react
│ └── package.json
├── button-vue
│ └── package.json
├── button-handlebars (implementable templates)
│ └── package.json
│ └── component-vanilla.js ?
│ └── component.hbs
└── button (common items + vanilla)
└── package.json
└── sass files
└── component-demo1-static.html
└── component-demo2-static.html
└── component-vanilla.js ?
└── tests (common)
text/0000-monorepo.md
Outdated
| - The style representation of the component, initially this would be Sass but could in theory be multiple formats that are offered | ||
| - A specification for the component (where applicable) | ||
| - Metadata around the component (could be a generated artifact) to be consumed by websites to aid in search result ranking | ||
| - Tests for the component |
There was a problem hiding this comment.
Vanilla/common could have tests that would be used by all frameworks:
- aat
- requirements
- a11y
- actions like keypress, click, etc
- checks for correct existence of
ariaattributes
- HTML
- w3c
- conforms to spec (consisten HTML structure)
- general
- checks for correct existence of
dataattributes - checks for correct content of attributes
- common expectations (is there a button(s) to open a dropdown, does the tooltip open in one of four possible directions (top/bottom/right/left)
selectorsimplementation
- checks for correct existence of
- a11y
note: these tests would all need to be in an exportable format so they could be imported by the frameworks for testing as well as used by apps ingesting the component to use to test their implementations of the component
text/0000-monorepo.md
Outdated
| - A specification for the component (where applicable) | ||
| - Metadata around the component (could be a generated artifact) to be consumed by websites to aid in search result ranking | ||
| - Tests for the component | ||
| - A development "story" for storybook |
There was a problem hiding this comment.
Could this instead be a variants or demos object in the vanilla/common package? The individual frameworks would be writing framework-specific stories for storybook, but they could create those stories from a common set of demos. For instance button, common demos would be the set of variants (primary, secondary, etc) and modifiers (small), etc. But then each demo would have a set of content which is framework agnostic:
- Small Ghost button
- content: Button
- Small disabled Ghost button
- content: Button
- disabled: true
- Small Ghost button with icon
- content: with icon
- icon: add
- Small disabled Ghost button
- content: with icon
- disabled: true
- icon: add
With simple common configs like this, each framework could produce an identical set of demos which should produce the same component-output from an aat/visual-styling standpoint and the HTML would be pseudo-consistent (ibm-button, CvButton, etc). It would also give something to test for (does framework x have demo y and is it consistently implemented when compared to the "some static version to test against")
An added benefit with something like this means there could be a partial common api to each framework's implementation - ie, each framework could accept a disabled boolean for button which would result in adding the disabled attribute to their button, or a content attribute which the framework use to add the button's content. note: this wouldn't be the only way a framework could make a button disabled or change it's content, just a common way they could all use in addition to how the individual framework would accept a config that creates the component output.
text/0000-monorepo.md
Outdated
| - The framework implementation of the component, using Vanilla as needed | ||
| - A specification for the component | ||
| - Metadata for the component | ||
| - Tests |
There was a problem hiding this comment.
If vanilla/common has the tests listed above, the framework ones would be specific to compilation within the framework, so:
- tests
- compilation
- configuration
- framework-specific demo/requirements tests
text/0000-monorepo.md
Outdated
|
|
||
| - The framework implementation of the component, using Vanilla as needed | ||
| - A specification for the component | ||
| - Metadata for the component |
There was a problem hiding this comment.
What would be in specification and metadata vs common ones above please?
text/0000-monorepo.md
Outdated
|
|
||
| As it would contain everything needed to use the component, while still relying on the core package and receiving updates. | ||
|
|
||
| In addition, each framework would have an entrypoint for core components. For React, this package would be found under: `packages/react` and would be published to `@carbon/react`. |
There was a problem hiding this comment.
what does this look like in implementation? Does that mean that one using npm i @carbon/react would get the entire suite, but would implement like so:
import { Button } from '@carbon/react';
vs
npm i @carbon/button-react
which is then done like so?
import Button from '@carbon/button-react';
There was a problem hiding this comment.
Yup! Exactly. Essentially @carbon/react re-exports all core component packages by doing:
export { Accordion, AccordionItem } from '@carbon/button-accordion';
export { default as Button } from '@carbon/button-react';
// ...
There was a problem hiding this comment.
This also enables add-ons to implement the same contract as core where they only bring in the components they need instead of having to subscribe to all of core (although this is just an idea at this stage).
text/0000-monorepo.md
Outdated
| - npm files | ||
| - `package.json` will have deterministic ordering and appropriate fields for versions, keywords, and publish config | ||
| - `.npmignore` will have appropriate defaults | ||
| - A `docs` folder for generated SassDoc and JSDoc information |
There was a problem hiding this comment.
In theory, there would be a common SassDoc/JSDoc, but also each framework could have their own set of these (like if a framework had a unique demo which required Sass). Would these be combined somehow or live within both framework and common/vanilla?
There was a problem hiding this comment.
@scottnath great point, one nice thing is that each package/implementation can create and ship their own sub-site if they need to. An example could be storybooks for vanilla/react/etc. What I'm hoping to do is use GitHub Actions alongside Netlify to help orchestrate publishing here to sub-sites, fingers crossed it goes well 😅
text/0000-monorepo.md
Outdated
|
|
||
| There are ways to mitigate this, including issue templates, but ultimately scaling issues and pull requests in a monorepo with multiple teams/collaboratos is still an unknown. | ||
|
|
||
| Finally, there is a concern around making everything under the Carbon umbrella public. The main use-case being: what if a team wants to use Carbon to build a component but is unable to make this component open-source? One solution could be the mirror our project to an internal GitHub Enterprise repo, but this could cause fragmentation in the ecosystem. |
There was a problem hiding this comment.
another option vs mirror the project would be that the component development environment is ingestible for use external the the carbon monorepo. So an add-on repo would pull in carbon as a dependency and the add-on development environment would share a directory structure from carbon when creating components:
new-addon-component/
├── new-addon-component-angular
├── new-addon-component-react
├── new-addon-component-vue
├── new-addon-component-handlebars
└── new-addon-component
but then if the add-on was creating a change to an existing component, the add-on's development environment would act the same way as developing within Carbon, except any new versions of an existing component would supersede carbon's verion:
add-on repo
├── button (carbons, pulled from node_modules as tho local)
├── tooltip (carbons)
├── overflow-menu (addon has different version)
├── overflow-menu-angular (carbons, pulled from node_modules as tho local)
├── overflow-menu-react (exists within repo)
├── overflow-menu-vue (carbons, pulled from node_modules as tho local)
├── overflow-menu-handlebars (carbons, pulled from node_modules as tho local)
└── overflow-menu (exists within repo)
For the above, the add-on team's common overflow menu content (like the demos or the selectors object) would differ from Carbon's thus when running tests in the non-react version those implementations would possibly fail some of the tests which would be local to this add-on.
The concept of using this structure instead of an actual mirror would be the ease in updating to the latest carbon. Also, the local overwrite could happen automatically just by adding a new directory with the same naming structure as carbon - ie: adding a button directory would locally defer to using the local button component vs carbon's.
|
Just wanted to say thanks @scottnath for the input here! 🎉 |
text/0000-monorepo.md
Outdated
|
|
||
| Alongside colocating packages inside of this project, this RFC also proposes that all packages intended to work with each other in the Carbon ecosystem should have the same major version. This would mean that component packages, alongside framework packages, would all use their major version to match whatever version of Carbon they support. If Carbon is at v11, then this would mean the packages would have some version number like `11.x.y`. | ||
|
|
||
| _Note: a big drawback here is around forcing semantic versioning. What if a package doesn't have to have a major bump in order to work?_ |
There was a problem hiding this comment.
In this case I think the breaking change can be justified as "there is a design breaking change" ... since the design language/styles will be the primary major version driver there will be some form of breakage even if it isn't API related.
Case in point, carbon-components-angular had no API breaking changes going to v3, but staying on v2 would have implied continued carbon v9 support.
Additionally major versions are the only real time to update peerDependencies
text/0000-monorepo.md
Outdated
| └── button-vue | ||
| ``` | ||
|
|
||
| However, we could limit these packages to only core packages, if needed. We could also move components to their own sub-packages folder, for example `packages/react` or a top-level `react` set of packages. |
There was a problem hiding this comment.
scoping things like packages/react or a top level react set, would definitely make importing the projects easier, and easier to work of a single frameworks packages ... less good from the perspective of discrete components though
text/0000-monorepo.md
Outdated
| - A development "story" for storybook | ||
| - Other exports that could be useful for sites | ||
| - Usage content for the component (mdx) | ||
| - Component docs (mdx) |
There was a problem hiding this comment.
For carbon-components-angular we keep our functional documentation as JSDoc style (TSDoc) comments in the code, and export a single docs site through compodoc ... this is definitely something we're looking to change up (for a more integrated demos + docs experience, something beyond storybook) but we really want to keep comments as the single source of truth.
text/0000-monorepo.md
Outdated
|
|
||
| We would structure the project so that component development happened using storybook. Given our usage of Yarn workspaces and lerna, we could subsequently build packages for each storybook development environment for each implementation. In addition, we could have specific frameworks have their own development environment, where applicable. | ||
|
|
||
| Given the potential size of the component library, it may also be necessary to invest in development tooling to help make working on individual components fast and simple. One idea could be an internal CLI tool that would enable folks to target specific frameworks or components for development in a storybook environment. This could look like: |
There was a problem hiding this comment.
Yeah this seems pretty key ... storybook definitely chokes on large projects, so being able to scope down to a handful for a given session would be amazing.
text/0000-monorepo.md
Outdated
|
|
||
| Push access to the project would be given to all members of the GitHub teams included in the project. | ||
|
|
||
| Publish access would ideally be automated through CI, but manual access can be granted to packages that belong to a specific GitHub team / workstream. |
There was a problem hiding this comment.
I may have missed this earlier, but automated versioning is ideal ... if we can guarantee that fix is always a patch, and a feat is always a minor - and we don't have to think about it - we can increase the confidence our consumers have in our work. It's also a nice bonus for a new contributor to have a PR merged and see the version bump.
The "version spam" is slightly annoying, but we definitely need to make sure we don't let releases become an emotional thing.
text/0000-monorepo.md
Outdated
|
|
||
| # Alternatives | ||
|
|
||
| The main alternative would be for **only** Carbon core to have a monorepo design. Given a desire to want to ship per-component packages, it is hard to see how we could scale doing repo-per-compnoent efforts. Particularly with respect to consistency between projects, keeping things up-to-date, and the experience of working with interlinked components, aka compound components. |
There was a problem hiding this comment.
Another alternative would be for each implementation to also be a mono-repo ... the primary benefits here would be cutting down on the notifications, less tooling, fewer total files, while keeping all the remainder of the implementation benifits.
The primary downside would be the same slow fragmentation of the ecosystem as each monorepo diverged.
text/0000-monorepo.md
Outdated
|
|
||
| Alongside these stages, we would need to define several items including: | ||
|
|
||
| - What is the process for adding a new component package? Is it okay if this package lives only for a particular framework? |
There was a problem hiding this comment.
In the context of non-core packages I would go with a strong yes ... the burden of support would be too high, especially in the case of teams/BUs that are exclusive to one framework (WSC is an example - we are 100% Angular)
For core packages I would go with a fairly strong no ... Eventually each framework will want a version of a component.
text/0000-monorepo.md
Outdated
| Alongside these stages, we would need to define several items including: | ||
|
|
||
| - What is the process for adding a new component package? Is it okay if this package lives only for a particular framework? | ||
| - What is the release cadence like for packages? |
There was a problem hiding this comment.
My suggestion:
Major - TBD, but our current cadence is once every 6 months?
Minor, Patch - automatic
text/0000-monorepo.md
Outdated
|
|
||
| - What is the process for adding a new component package? Is it okay if this package lives only for a particular framework? | ||
| - What is the release cadence like for packages? | ||
| - How do we communicate when components are core-supported versus community-supported? In their README? On the website? |
There was a problem hiding this comment.
Yes 😉 The more sources the better! Perhaps the website could pull some meta info from the monorepo?
text/0000-monorepo.md
Outdated
| - What is the process for adding a new component package? Is it okay if this package lives only for a particular framework? | ||
| - What is the release cadence like for packages? | ||
| - How do we communicate when components are core-supported versus community-supported? In their README? On the website? | ||
| - How does one suggest another framework implementation? For example, if a team wanted preact or svelte. |
There was a problem hiding this comment.
Or WebComponents!
Perhaps a proposal issue, followed by a period looking for/establishing a team to own the initial development? This ties pretty closely to the governance model I think.
There was a problem hiding this comment.
I see carbon-components-angular and -vue aren't currently using GitHub milestones:
https://github.com/ibm/carbon-components-angular/milestones
https://github.com/carbon-design-system/carbon-components-vue/milestones
In a later phase of this monorepo effort, if we are to include the angular and vue components, how would they project manage in GitHub if they wanted to? Same goes for any community component where a contributor would want to track progress if it's a bigger effort?
We've discussed GitHub issues and labels in this RFC, but not so much project management.
Maybe we let angular and vue core component efforts have their own GitHub projects so they can prioritize their issues and track what's in review? And use our existing milestones to include those efforts in our tracked major and minor releases?
|
Totally open to other project management strategies, but we've found the simple Todo -> In progress -> Review -> Closed flow to be pretty effective. Since Projects are so flexible we've also been able to track larger (multi-issue) projects as a separate column.
This seems like the best solution ... It doesn't have to be limited to vue/angular though, projects are amazing for taking out and working on a slice of the total issues - could be useful to track community component related work as well. |
This RFC proposes a new approach to Carbon's repo structure and ecosystem organization with respect to frameworks and add-ons.
View rendered text