-
Notifications
You must be signed in to change notification settings - Fork 12.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Infer project references from common monorepo patterns / tools #25376
Comments
All the tools you've listed use normal commonjs module resolution (and consequently use the normal One of the things I was running into in trying to setup an
I would assume package root, and if we don't find one there, we could read a
The usecase is a (near) zero-config start for common monorepo setups, so opt-out, IMO. Could add something like a |
In https://github.com/strongloop/loopback-next, we are using monorepo to develop a bunch of modules. Right now, we have one IMO, it's important to allow each package to have its own tsconfig configuration, because this configuration often involves a list of files to include/exclude from compilation. Assuming the Initially, I was reading the proposal as to assume a single
If we can find an elegant way how to support most of commonly-used monorepo layout (tools) that are setting up cross-package dependency tree using the information from standard npm/package.json I think the tricky question we need to answer first: how to distinguish between dependencies that are considered as monorepo-local and should be configured as TypeScript references; and external dependencies that should be consumed as read-only? The The first solution that comes to my mind:
I find this solution a bit too complex and involved, but don't have any better alternative right now :( (*) Handling of symlinks outside of monorepo is actually another interesting case to discuss. If we treat all symlinked dependencies as project references, then people using multiple single-repo projects with manual For example, loopback depends on strong-remoting, where both modules are maintained by the same team. Sometimes I need to make changes both in |
I was thinking we could update the -b flag to accept a list of globs (whereas today it takes a single folder or list thereof), so it works like the packages field in lerna or workspace fields in npm/yarn, so you can easily pass |
how would that work for tsserver? |
@mhegazy AFAIK Nothing in tsserver yet cares about project references (or the dependency graph thereof), just the presence of declaration maps. 😉 The only one that requires it is probably a cross-project compile on save - which is a matter of integrating the entire {
"references": [{ path: "packages/*" }]
} to setup the context. (Which, hopefully, would also allow |
I mean, jamming it into a |
Or alternatively, we can just actually read a |
I think cross repo reference + yarn workspace worked before TypeScript 2.9 Say you want to import something from your monorepo folder However, since 2.9, I believe it goes through symlinks or something, VSCode now suggests something like |
Also bear in mind that the various tools can treat the workspace globs differently. I tried The downside to trying to support specific monorepo types is that these are just the ones we're using this week. Next week we'll probably be using something completely different 😆 |
I am using a monorepo setup based on lerna. The problem I have in this scenario is, that the root folder contain all dependencies of all packages. |
@Cryrivers actually, that bug was introduced in 2.9.2. There's an issue open for it, but I can't seem to find it now. 2.9.1 works as expected. |
Just to share: for Theia I've added a script that converts yarn workspaces to ts project references: https://github.com/theia-ide/theia/blob/b7471470214533912174fa1c0b07301346026939/scripts/configure-references#L19 It can be executed as |
I think this setup is a bit backwards, why do we need to define project references in Imagine following Lerna monorepo setup:
Each has it's own
Adminapp has the project reference to the components package in the
I propose that instead of having project references, we would have package directories, e.g. In short, instead of this in
One could define just the package root directories in
TS Could look in to that packages directory and search for Running a It seems a bit odd that monorepo setup can't jump to declaration without a declaration maps. The source code of each project is lying in the |
[1] lerna/lerna#1616 Edit: We're back; see below. |
@RyanCavanaugh I think it's premature to close this issue. There must be ways of reducing the duplication of dependency info that aren't tightly coupled to any particular monorepo tool. We only really need two things, neither of which depend on external monorepo tools:
|
@RyanCavanaugh You can lock to an older version of lerna for now. The license change isn't retroactive. |
I'm not interested in getting sued (or getting my employer sued) because I forgot to put the right characters in a Regardless, any feature work we do relating to lerna would need to be tested on its latest version. People will log bugs and point us at their repos that point to a We can re-open this if a maintained ecosystem that is legal for us to test with opens up. |
Well the way I see it, if The argument here sounds like "we can't consider fixing or improving aspects of |
This issue specifically is basically "make tsbuild work from the same fields lerna does" even though that's not the title. If lerna and some tool X (let's call it free lerna, or flerna) standardize on behavior so that we can test flerna without getting sued, then that's all well and good. If not, then some other part of the TS ecosystem written by someone not under a bill of attainder will fill that gap. My current take on the monorepo ecosystem is that there isn't something as popular as lerna yet, and we don't usually want to invest too much in places where we wouldn't be supporting the most-popular variant as a first-class citizen. Tool popularity will almost certainly change in the future and, like all other issues here, we'll reconsider it at that time even if this particular currently has the open/close bit set to zero. There is plenty in the pipe to fix and improve Also I am trolling a tiny bit because come on |
Okaaay. Well can we still look at making project dependency declarations less repetitive? Does it need a new issue? I've already had colleages get the Also:
this is gonna happen anyway, it's not specific to this issue. Hopefully that bit was trolling 🤞. |
npm has announced that they'll support the same |
yarn has
|
I've created a cli tool based on information from |
I solved this using a custom shell script. I have similar monorepo project to @thi.ng/* called @ctx-core/*. I tend to use a git submodule to include the packages in active development. @Bnaya, I'm excited about @thi.ng. I'd love to find a way to mix in other large scale monorepo libraries together so we can utilize each others work. I'm sure patterns will emerge. A vision that I see is every developer & organization could cultivate their own monorepo, with a la carte libraries that are under their own control by utilizing forking. #!/bin/sh
ROOT_DIR=$(dirname $(dirname "$0"))
tsc -b \
$(ls $ROOT_DIR/packages/ctx-core/packages/*/tsconfig.json | xargs dirname) \
$(ls $ROOT_DIR/packages/*/tsconfig.json | grep -v ctx-core | xargs dirname) \
$@ |
|
For my personal needs I wrote a tool for pnpm which syncs package.json dependencies to tsconfig.json. Basic idea: look at the workspace and dependencies and if the dependency is a link to one of the workspace module, then put relative path in tsconfig.json. It has an alpha quality, but works great for small/middle sized monorepo: https://github.com/Bessonov/set-project-references |
📝 Its note from my experience of creating @monorepo-utils/workspaces-to-typescript-project-references. We can use the almost same sync logic to following package managers.
There is a package manager/monorepo tool that uses another logic. 📝 lerna use It is difficult to support All monorepo tools because currently does not exists workspaces specification. However, If TypeScript supports |
What is the opinion on this issue at the moment? Is the view that IMO this is still a major pain point for TS - it all works fine when its a single package, but almost any project nowadays involves some kind of monorepo setup... The methods people seem to be using to solve this
|
I'm looking into converting a few existing repositories into a pnpm workspace monorepo as well (+ possibly add rush to the mix if I get pnpm to work) and I've been hitting my head against this for the past few days. There's various articles (and issues on SO) to be found covering monorepo setups with TypeScript and lerna or yarn. However, most of these are outdated, contain conflicting information or do things in different ways, in addition to having various DX shortcomings (i.e. no proper @scope auto-import, no definition lookups, manual building of libs, and so on) that honestly make the whole monorepo setup a no-go for me. There doesn't really appear to be any official documentation anywhere merging all of these concepts together, never mind adding something like My conclusion after these past few days is that the level of in-depth TS knowledge you seem to need to get a setup like this at least semi-functional just feels too high (read as: I couldn't figure it out and I'm rather frustrated by it). This is not a critique to the TS team, I'm just wondering if what I've been trying to set up is simply not feasible at the moment. I'm also not really sure where this belongs. Workspaces are currently implemented by package managers, but TypeScript seems to be moving more and more into a "holistic" role, adding various capabilities to link different (nested) projects together, so there's definitely quite a bit of overlap. Since it's seemingly going to take a while before package managers converge on a workspace approach, making native workspace support inside TS rather difficult, could it perhaps be an option to gather & document various approaches in use today, with their benefits and (DX) shortcomings? Or document a section with best practices somewhere? |
Causing auto import to fail: vercel/turborepo#331 |
Probably not as good as a single source of truth, but fwiw meta-updater can be used to keep them in sync. See the config that |
Is it possible to implement something like this? tsconfig.json{
"references": [
{
"path": "workspace:@namespace/package-name[/tsconfig.json]"
// or "workspace": "@namespace/package-name"
}
]
} package.json of "@namespace/package-name"{
"exports": {
"tsconfig.json": "path/to/tsconfig.json"
}
} Advantages:
|
I was thinking lets add a new compilerOptions called workspace. It defaults to none but supports npm, yarn, pnpm. All it does is add to the references based on the rules of that workspace tool. I am just now starting to read though the Typescript compiler and code so not sure what that would entail but it seems easy to me. The advantage here is it can look at the projects deps and dev deps and then see if those projects are workspaces in the current project. If so add them. If not don't. That way you don't have globs which will not work if you have circler references. Will fork and give it a try but what is the process to get this as a feature request and into the code? Does Microsoft except pull requests? |
Genesis: see section "Repetitive configuration" in #3469 (comment)
Search Terms
monorepo infer project references automatically yarn lerna workspace package.json
Suggestion
For common monorepo managers, we should natively understand cross-project references declared in
package.json
as if they were declared intsconfig.json
Open questions:
tsconfig.json
file? This data is actually not present in the current (non-tsconfig) dependency graph. We could assume it to be in the package root; what if it's elsewhere?Use Cases
Examples
https://github.com/RyanCavanaugh/learn-a
This repo has a fair bit of duplication where projects need to write down their dependencies in package.json and as references (with different syntax) in
tsconfig.json
.Checklist
My suggestion meets these guidelines:
The text was updated successfully, but these errors were encountered: