Utilize the @express npm scope

[wesleytodd]

Opening this to get feedback on the idea of using the @express scope as a way to organize modules and as a continuation of this conversation. All of this is assuming we can get the scope, the process for which I have started with npm.

1. What value does using a scope provide?

• Increase discoverability for new users looking for "safe", "compatible", "best practice" modules
• Name packages more strategically by not have to compete for names in the global namespace
• Provides guarantees to users for modules under the scope (security best practices, maintenance guarantees, etc.)
• Provide a place for "batteries included" style additions for Express which are "blessed" but still community maintained

2. What negative effects would it have?

• Does adding modules under the scope negatively effect the existing non-scoped modules? For example, would it make them look "less official"? Would it cause confusion?
• It could exclude other/better packages by the very nature of "promoting" the scoped ones

Any other downsides?

3. If splitting was an issue, would we consider migrating those packages to the scope?

Migrating would cause churn, currently it would loose the version history, and is extra work. My only experience with this is as a user of Babel, and that was not a great user experience. Personally I think we should avoid migrating them unless npm can help out with this as a feature. I tweeted at them, maybe this is something on their roadmap.

4. Would it be "squatting" to sit on it and not use it?

I think @dougwilson brought up some good points in the other thread around the many ways users could try to piggy back off the express name to "do harm", but I still think it would protect the most important part by claiming the namespace even if we did not use it.

5. What type of modules would we publish here?

For this I think that we would want to set some requirements. For example, it should "express specific". This would mean coupling to express api's or providing some feature which only makes sense in an express context. Also, we would want to have a standard set of code requirements like tests, travis ci and standard readme stuff.

I think we would also want to set some criteria around "sufficient expected user base". I don't think we would want every niche usage type middleware module being eligible. Examples of modules I would see being published here would be session, serve-static, or helmet. Examples of modules not to publish here would be modules in the "Other Middleware" section of the website like sriracha-admin.

If the module was coming from outside the current express contributors, I think it might also be good to require some "core team"/CTC members be added with publish rights in case of security or other important issues.

Sorry for the long winded post, but I wanted to get as much out of my head as I could on this to start the discussion.

[dougwilson]

From the discussion, I also grabbed the following scopes: expressjs, pillarjs, jshttp.

[wesleytodd]

Hey, so to bring up this conversation again, I have seen a small uptick in folks at work using the OpenAPI middleware package I built. I documented it with the intent to move it to @expressjs as we agreed it would be a good idea to start using. I merged the most important remaining feature (nested routers) which I knew was missing. Since we did agree that for new packages we liked this idea, are we good with me moving that repo into the express org and giving it a 1.0.0 publish?

https://github.com/wesleytodd/express-openapi

[wesleytodd]

Also, if we wanted I would be happy to set it up with the permissions as we discussed to copy from the pkgjs org to illustrate it on something new and low impact. I can even setup automated publishing and credential management as we have discussed in the past to illustrate the workflow.

[dougwilson]

I think from our discussion the main issue was going to be consistency, and getting us to that point. IMO there is no reason not to move that into the org, but right now, repos in the org are supposed to be published as the same name in npm as their repo name. I believe from when we discussed this, we are supposed to actually come up with a full migration plan for all the modules in the org to get them to move all at once, looking at how the hapijs project did it, potentionally, or determine what we need to do in order to perhaps support both a global name and a scoped name? I'm a bit fuzzy on the details from the TC meeting about it, but what I do remember is that we need an actual plan to get everything into the npm org as part of using it.

[dougwilson]

And to note in here, I believe we said that @express would be for the core express parts and @expressjs would be for the middlewares -- not saying you're not saying that, as your wrote @expressjs, just wanted to try to get things summed up in this thread. Of course, we probably should of actually summed up those meetings into this discussion thread at the time we had them, but I can volunteer to re-watch our recorded meeting and sum them up here unless someone wants to beat me to it.

[wesleytodd]

I think from our discussion the main issue was going to be consistency, and getting us to that point.
we are supposed to actually come up with a full migration plan for all the modules in the org

Yep! I was hoping this would be the next step on that plan. We need a path to follow, this would blaze the trail at a lower interruption cost than moving something like session.

determine what we need to do in order to perhaps support both a global name and a scoped name

My understanding of the plan was that we would figure out how to reliably publish there via CI so that we could automate publishing to both. My proposal is that this package be used to figure out how to do that. I would publish as @expressjs/openapi and @wesleytodd/openapi.

I believe we said that @express would be for the core express parts and @expressjs would be for the middlewares

👍

---

So the plan would be I move the repo, then integrate an automated release flow which dual publishes. If we all agree it looks good, then it would be the template by which we roll out to the other projects.

[wesleytodd]

I was hoping we could use Release Please via their new github action, which is what Restify just moved to and is getting great support from @bcoe and folks at google.

[dougwilson]

I think from our discussion the main issue was going to be consistency, and getting us to that point.
we are supposed to actually come up with a full migration plan for all the modules in the org

Yep! I was hoping this would be the next step on that plan. We need a path to follow, this would blaze the trail at a lower interruption cost than moving something like session.

Yea. And to extra clarify myself: I'm not saying that moving everything is a pre-requisite :) Ideally just some semblance of a plan, haha. Actually moving everything is going to be difficult, as parts of what we discussed and know within ourselves.

I was hoping we could use Release Please via their new github action, which is what Restify just moved to and is getting great support from @bcoe and folks at google.

The biggest thing is that we have to publish with OTP on, so the module is not flagged in the npm Security Insights API as having been published without OTP. Now, I'm not saying that does not do OTP, just that is my main concern about the automated releases, as in the past it has just been ignored / forgotten about and we don't want to end up burned by that if we begin planning only to realize that down the road :)

[wesleytodd]

The biggest thing is that we have to publish with OTP on

For sure, my plan was to figure out setting up wombat dressing room or the tool electron is using for this. Again, as an experiment to ensure it will work for us long term.

Actually moving everything is going to be difficult, as parts of what we discussed and know within ourselves.

I thought we had settled on double publishing, then eventually deprecating the global scope ones. And by eventually I imagine many years from now. But if we have it all working with automation then dual publishing should not be an issue, right?

[wesleytodd]

And another thought I have to mitigate the risk. I can set it all up where it is on my repo and ensure it works before we move it over. That way we can ensure we don't make a mistake in the process.

[dougwilson]

The biggest thing is that we have to publish with OTP on

For sure, my plan was to figure out setting up wombat dressing room or the tool electron is using for this. Again, as an experiment to ensure it will work for us long term.

Yep, definately 👍 Of course I suppose the exact same experiment can be done anywhere, even if it was a personal module 😛 Not to say don't, just to me, it's not strictly an argument to moving into the org generically unless I'm missing something that the express org gives vs somewhere else. (I am not saying not to do it, to be clear, just a general statement in case... I am missing something!).

I thought we had settled on double publishing, then eventually deprecating the global scope ones. And by eventually I imagine many years from now. But if we have it all working with automation then dual publishing should not be an issue, right?

Yea, double-publishing works. I haven't listened back on our recordings just yet. But of course the devil's in the details as they say -- for example the package.json that is checked into the repo, which module name is in there? And since npm includes the commit has when the publish is done, does it mean the secondary publish(s) no longer actually would match up to the contents of a hash (and I'm not talking about including untracked files, I mean including modified tracked files like pacakge.json)? Not sure if there are folks that care about these things or not, but in the past there have been issues opened (due to my own mistake) when the hash of the published version didn't match a commit hash in the github repo.

And another thought I have to mitigate the risk. I can set it all up where it is on my repo and ensure it works before we move it over. That way we can ensure we don't make a mistake in the process.

I guess that answers part of my first point, haha. And I do want to clarify that I'm not saying you have to do it that way :) Sorry I keep clarifying, just want to ensure my POV comes through in text correctly :)

[wesleytodd]

it's not strictly an argument to moving into the org generically

Nope, I just dont want to send emails to too many people while getting it setup. Remember the unexpected release emails from the statusboard build? 😆

for example the package.json that is checked into the repo

The way I am doing this now is I have a parallel release branch on which I merge each release. That branch has the package name changed as the first commit. You can see it in action on the current repo: https://github.com/wesleytodd/express-openapi/tree/wesleytodd

no longer actually would match up to the contents of a hash

Yes, but it cannot have the same hash, it has different content. That is just part of dual publishing which we cannot get around.

[dougwilson]

no longer actually would match up to the contents of a hash

Yes, but it cannot have the same hash, it has different content. That is just part of dual publishing which we cannot get around.

For this one, maybe I can clarify (your answer may still be they don't match, but just wanted to be on the same page): the two releases don't need to have the same commit hash, just a commit hash that appears in the repository, I think. From when folks opened issues before, that was their main concern ("version X.Y.Z on npm says commit hash of 1234abcd but there is no 1234abcd in the repo!").

[wesleytodd]

Ah, commit hash, not content hash 😄, I miss understood. So in the repo I have it now, the commit exists just not on the main branch (it is on wesleytodd where I release from to my scope). So the version released to the global namespace would have one commit hash which would live on a branch (maybe named global-namespace-release) and the one published under the scope would be exist on main. Make sense?

[jonchurch]

@wesleytodd Assuming from this history that there was no traction from npm on getting the express org?

[wesleytodd]

We do have it! I just joined it and was going to add everyone asap. Just on the traige team call now and the TC call next, so haven't been able to do it yet.