Add support for monorepos such as DefinitelyTyped

[styfle]

Some projects are published to npm as multiple packages but use a single github repository.

I didn't dig into details to see npm has a way to detect this, but I can tell you how it works for DefinitelyTyped 😄

Basically, if the package @types/node is bumped, I would expect to see a link to commits pointing to /commits/master/types/node

Currently, it links to all commits in the entire repo which is huge as you can see here: /commits/master

Is this possible?

[greysteil]

This has been sitting in my inbox as a "good question, needs thought" issue for the last week. Finally getting to it now!

[greysteil]

So, digging in, this looks like a bit of a nightmare. the registry doesn't have any knowledge that @types/node is a monorepo, and the repo itself doesn't provide any information to suggest its structure (i.e., it doesn't have a lerna.json or use Yarn workspaces).

There is an unstructured hint in the README that has been pushed to npm, however. I might be able to make use of that...

[styfle]

Thanks for the response!

I see now how tricky this is to make a pattern that will work for any arbitrary monorepo since they all handle publishing differently.

DefinitelyTyped is kind of the exception to most npm packages because they don't deal with JS source code so they don't bother maintaining a bunch of package.json files in a packages directory.

Maybe you can look at how often Dependabot is updating @types/* packages because if it is often, then its probably worth it to add a special case to check the README.md for the magic string "Files were exported from".

[greysteil]

Yeah, this one is definitely tricky, but I'm not giving up on it yet.

I think there's a bigger problem here that there's currently no way for monorepos to specify the directory that their code lives in in their package.json. That affects react, angular, etc., and is a bit sad. I'm chatting with the npm folks to make sure I'm not missing anything, and will make a proposal to fix it if need be.

[styfle]

Yeah I could see that being part of the repository field. Something like:

"repository" : {
  "type": "git",
  "url": "https://github.com/DefinitelyTyped/DefinitelyTyped.git",
  "path": "/types/node"
}

I found the npm team is pretty good at entertaining ideas.
Did you create an idea on their community page? https://npm.community/c/ideas
Or just tweet at them?

[greysteil]

Asked in the discord app (here). I'll create an idea on the community page if I don't hear back.

[greysteil]

FYI, I've created an RFC for npm on this - it's here.

[styfle]

@greysteil The RFC has been accepted! Great job! 🎉 🎉 🎉

Are you able to move forward with implementing this feature in Dependabot?

[greysteil]

Yes! Just implemented in this commit. Will need to get each monorepo to convert to the new format, but that should be doable. 🎉