Scripts: Transform scripts directory to type:module #24698

valentinpalkovic · 2023-11-03T09:59:31Z

Closes N/A

What I did

Set "type": "module" in scripts/package.json
Replaced ts-node in scripts by esbuild toolchain
Removed workaround for loading pure esm modules (e.g. execa)
Typescriptified scripts/check-package.js
Replaced require.main === module expression by esm equivalent
Set up scripts/jest.config.cjs to tell Jest how to properly run tests in esm environment
Removed shebangs, since they are not respected in a Windows environment
Adjust scripts/tsconfig.json to tell Typescript, that we are using ESM, but a bundler (moduleResolution setting) is used in the end. With that, we are not forced to use extensions for relative imports.

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

stories
unit tests
integration tests
end-to-end tests

Manual testing

This section is mandatory for all contributions. If you believe no manual test is necessary, please state so explicitly. Thanks!

Documentation

Add or update documentation reflecting your changes
If you are deprecating/removing a feature, make sure to update
MIGRATION.MD

Checklist for Maintainers

When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli/src/sandbox-templates.ts
Make sure this PR contains one of the labels below:
Available labels
- bug: Internal changes that fixes incorrect behavior.
- maintenance: User-facing maintenance tasks.
- dependencies: Upgrading (sometimes downgrading) dependencies.
- build: Internal-facing build tooling & test updates. Will not show up in release changelog.
- cleanup: Minor cleanup style change. Will not show up in release changelog.
- documentation: Documentation only changes. Will not show up in release changelog.
- feature request: Introducing a new feature.
- BREAKING CHANGE: Changes that break compatibility in some way with current major version.
- other: Changes that don't fit in the above categories.

🦋 Canary release

This pull request has been released as version 0.0.0-pr-24698-sha-758c0de1. Install it by pinning all your Storybook dependencies to that version.

More information


Published version	`0.0.0-pr-24698-sha-758c0de1`
Triggered by	@valentinpalkovic
Repository	storybookjs/storybook
Branch	`valentin/use-type-module-in-scripts`
Commit	`758c0de1`
Datetime	Wed Nov 8 09:01:56 UTC 2023 (`1699434116`)
Workflow run	6795945840

To request a new release of this pull request, mention the @storybookjs/core team.

core team members can create a new canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=24698

socket-security · 2023-11-03T10:07:45Z

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Packages	Version	New capabilities	Transitives	Size	Publisher
@types/window-size	1.1.4	None	`+0`	5.1 kB	types
jest_workaround	0.79.19	None	`+0`	1.02 MB	magic-akari

🚮 Removed packages: [email protected]

…ule-in-scripts

…-scripts

combine-compodoc.ts

code/addons/a11y/package.json

JReinhold · 2023-11-08T08:25:47Z

What do you mean by this?

Replaced ts-up in scripts by esbuild toolchain

Our bundle scripts still use tsup, I don't see any changes to that?

https://github.com/storybookjs/storybook/pull/24698/files#diff-f1388de49bfa818ab9841e27336be434f4b2174789389f7cdec7a3743f88c239

https://github.com/storybookjs/storybook/pull/24698/files#diff-5548a29db85d623884df9f9582126025d3375e858931d16c4943002e231537d8

(Which I'm happy about, because tsup has many advantages over rawdogging esbuild for us, mainly that it supports generating .d.ts files, which esbuild doesn't)

scripts/utils/esmain.ts

scripts/release/publish.ts

storybook-bot · 2023-11-08T08:42:55Z

Failed to publish canary version of this pull request, triggered by @valentinpalkovic. See the failed workflow run at: https://github.com/storybookjs/storybook/actions/runs/6795743645

code/addons/actions/package.json

IanVS

Awesome cleanup here!

scripts/tasks/bench.ts

IanVS · 2023-11-08T12:54:42Z

scripts/check-package.ts

      };
    })
    .reduce((acc, next) => {
      acc[next.name] = next;
      return acc;
-    }, {});
+    }, {} as Record<string, { name: string; defaultValue: boolean; suffix: string; helpText: string }>);


Instead of the cast, you might be able to provide a generic to reduce instead, which is a bit safer normally.

code/addons/a11y/package.json

valentinpalkovic · 2023-11-08T15:06:10Z

@ndelangen, I will wait for your approval before this gets merged to get in your opinion.

JReinhold

💪🧹

package.json

scripts/tasks/sandbox.ts

valentinpalkovic · 2023-11-12T07:12:00Z

@ndelangen friendly reminder

scripts/utils/esmain.ts

…ule-in-scripts

socket-security · 2023-11-14T12:56:06Z

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

esbuild-register from scripts/node_modules

IanVS · 2023-11-14T15:05:41Z

code/ui/manager/package.json

-    "check": "node --loader esbuild-register/loader -r esbuild-register ../../../scripts/prepare/check.ts",
-    "prep": "node --loader esbuild-register/loader -r esbuild-register ../../../scripts/prepare/esm-bundle.ts"
+    "check": "node --loader ../../../scripts/node_modules/esbuild-register/loader.js -r ../../../scripts/node_modules/esbuild-register/register.js ../../../scripts/prepare/check.ts",
+    "prep": "node --loader ../../../scripts/node_modules/esbuild-register/loader.js -r ../../../scripts/node_modules/esbuild-register/register.js ../../../scripts/prepare/esm-bundle.ts"


This is pretty dang ugly. Are you sure this isn't a good enough reason to explore using tsx instead? ;)

https://www.npmjs.com/package/tsx

@valentinpalkovic let's try this tomorrow.

tsx will be installed in scripts. It would then look like this:

"check": "../../../scripts/node_modules/tsx/index.js ../../../scripts/prepare/check.ts"

This is still ugly. A bit less ugly than the current solution, but still ugly. I don't see strong reasons for now to invest even more time into this PR.

Why should we introduce a tool, which builds on top of esbuild, if we can use esbuild directly and have no issues with it? I haven't seen strong reasons and arguments for introducing it so far.

oh yeah that's not much better. Seems like something that can be cleaned up later if we find a nice way. Using NODE_PATH could also be an option, though I feel like it's discouraged / deprecated these days.

Hey @IanVS

I have spiked a bit a solution using tsx. I haven't invested much time, but the following occurred:

The strange thing is, that named exports are not allowed to be imported with named imports.

e.g. fs-extra
Currently, we do named imports like this:

import { readJson } from 'fs-extra';

Node then complains and says:

import { readFile } from 'fs-extra'; ^ SyntaxError: The requested module 'fs-extra' does not provide an export named 'readFile' at ModuleJob._instantiate (node:internal/modules/esm/module_job:124:21) at async ModuleJob.run (node:internal/modules/esm/module_job:190:5)

Indeed, if I go to the Readme of fs-extra the following is written:

ESM
There is also an fs-extra/esm import, that supports both default and named exports. However, note that fs methods are not included in fs-extra/esm; you still need to import fs and/or fs/promises separately:

import { readFileSync } from 'fs' import { readFile } from 'fs/promises' import { outputFile, outputFileSync } from 'fs-extra/esm'

Default exports are supported:

import fs from 'fs' import fse from 'fs-extra/esm' // fse.readFileSync is not a function; must use fs.readFileSync

So, I have to change all named imports to default imports.

The same error then occurs on relative file imports, like this one:

import { allTemplates } from '../code/lib/cli/src/sandbox-templates';

/Users/valentinpalkovic/Projects/storybook-next/scripts/task.ts:33 allTemplates as TEMPLATES, ^ SyntaxError: The requested module '../code/lib/cli/src/sandbox-templates' does not provide an export named 'allTemplates' at ModuleJob._instantiate (node:internal/modules/esm/module_job:124:21) at async ModuleJob.run (node:internal/modules/esm/module_job:190:5)

I am not sure what’s going on here or why the error occurs with tsx, but not with the esbuild-loader

…was not patched back

Convert build-package to Typescript

8286a8d

valentinpalkovic added the build Internal-facing build tooling & test updates label Nov 3, 2023

valentinpalkovic self-assigned this Nov 3, 2023

valentinpalkovic force-pushed the valentin/use-type-module-in-scripts branch from f428363 to 760f2b8 Compare November 3, 2023 10:06

valentinpalkovic added the ci:daily Run the CI jobs that normally run in the daily job. label Nov 3, 2023

valentinpalkovic changed the base branch from next to valentin/remove-lerna November 3, 2023 10:07

valentinpalkovic force-pushed the valentin/use-type-module-in-scripts branch from 760f2b8 to 6bbedf1 Compare November 3, 2023 10:33

Use esbuild as esm loader for node

b7ef8c5

valentinpalkovic force-pushed the valentin/use-type-module-in-scripts branch from 6bbedf1 to b7ef8c5 Compare November 3, 2023 10:42

valentinpalkovic added 7 commits November 3, 2023 12:01

Convert require commonjs expressions to esm equivalents

c436354

Fix babel and eslint

946d863

Fix tests

3c806e0

Fix generate-sandbox script

e22f3b9

Remove ts-node in scripts

baa57e0

Fix check script

50a7d47

Fix run-registry script

c74b562

valentinpalkovic force-pushed the valentin/remove-lerna branch from 8286a8d to 85fe759 Compare November 6, 2023 12:23

valentinpalkovic added 10 commits November 6, 2023 14:58

Fix Typescript issues

122e94f

Replace tsup with esbuild

103794e

Use node directly to check angular framework

b9f0e4e

Fix sandbox generation

29f5388

Merge remote-tracking branch 'origin/next' into valentin/use-type-mod…

2b32377

…ule-in-scripts

Merge branch 'valentin/remove-lerna' into valentin/use-type-module-in…

1f17431

…-scripts

Fix execaCommand in build-package

cf06a9c

Use esMain helper to check for execution origin

55dc4f2

Add import for execa package in

40e627f

combine-compodoc.ts

Fix esMain helper usage

25b6907

Base automatically changed from valentin/remove-lerna to next November 7, 2023 14:35

valentinpalkovic commented Nov 8, 2023

View reviewed changes

code/addons/a11y/package.json Outdated Show resolved Hide resolved

valentinpalkovic requested review from kasperpeulen, shilman, yannbf, IanVS and ShaunEvening as code owners November 8, 2023 08:02

JReinhold reviewed Nov 8, 2023

View reviewed changes

scripts/utils/esmain.ts Show resolved Hide resolved

JReinhold reviewed Nov 8, 2023

View reviewed changes

scripts/release/publish.ts Show resolved Hide resolved

Remove custom execa implementation

758c0de

JReinhold reviewed Nov 8, 2023

View reviewed changes

code/addons/actions/package.json Outdated Show resolved Hide resolved

IanVS approved these changes Nov 8, 2023

View reviewed changes

Fix tests

6d5080e

valentinpalkovic added ci:daily Run the CI jobs that normally run in the daily job. and removed ci:daily Run the CI jobs that normally run in the daily job. labels Nov 9, 2023

JReinhold approved these changes Nov 9, 2023

View reviewed changes

package.json Show resolved Hide resolved

scripts/tasks/sandbox.ts Show resolved Hide resolved

shilman reviewed Nov 13, 2023

View reviewed changes

scripts/utils/esmain.ts Outdated Show resolved Hide resolved

ndelangen approved these changes Nov 14, 2023

View reviewed changes

Merge remote-tracking branch 'origin/next' into valentin/use-type-mod…

3c41176

…ule-in-scripts

Update package.json scripts to use

e792e6f

esbuild-register from scripts/node_modules

IanVS reviewed Nov 14, 2023

View reviewed changes

valentinpalkovic added 2 commits November 14, 2023 17:15

Replace default export of esMain by named export

e8f8a65

Merge branch 'next' into valentin/use-type-module-in-scripts

46cb2fb

ndelangen merged commit e559a1e into next Nov 15, 2023
106 of 107 checks passed

ndelangen deleted the valentin/use-type-module-in-scripts branch November 15, 2023 08:48

github-actions bot mentioned this pull request Nov 15, 2023

Release: Prerelease 7.6.0-alpha.7 #24853

Merged

36 tasks

ndelangen added a commit that referenced this pull request Nov 23, 2023

fix check-dependencies script by converting it to cjs, because #24698 …

e372504

…was not patched back

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scripts: Transform scripts directory to type:module #24698

Scripts: Transform scripts directory to type:module #24698

valentinpalkovic commented Nov 3, 2023 •

edited by storybook-bot

Loading

socket-security bot commented Nov 3, 2023 •

edited

Loading

JReinhold commented Nov 8, 2023 •

edited

Loading

storybook-bot commented Nov 8, 2023

IanVS left a comment

IanVS Nov 8, 2023

valentinpalkovic commented Nov 8, 2023

JReinhold left a comment

valentinpalkovic commented Nov 12, 2023

socket-security bot commented Nov 14, 2023 •

edited

Loading

IanVS Nov 14, 2023

ndelangen Nov 14, 2023

valentinpalkovic Nov 14, 2023 •

edited

Loading

IanVS Nov 14, 2023 •

edited

Loading

valentinpalkovic Nov 15, 2023

Scripts: Transform scripts directory to type:module #24698

Scripts: Transform scripts directory to type:module #24698

Conversation

valentinpalkovic commented Nov 3, 2023 • edited by storybook-bot Loading

What I did

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

Manual testing

Documentation

Checklist for Maintainers

🦋 Canary release

socket-security bot commented Nov 3, 2023 • edited Loading

JReinhold commented Nov 8, 2023 • edited Loading

storybook-bot commented Nov 8, 2023

IanVS left a comment

Choose a reason for hiding this comment

IanVS Nov 8, 2023

Choose a reason for hiding this comment

valentinpalkovic commented Nov 8, 2023

JReinhold left a comment

Choose a reason for hiding this comment

valentinpalkovic commented Nov 12, 2023

socket-security bot commented Nov 14, 2023 • edited Loading

IanVS Nov 14, 2023

Choose a reason for hiding this comment

ndelangen Nov 14, 2023

Choose a reason for hiding this comment

valentinpalkovic Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

IanVS Nov 14, 2023 • edited Loading

Choose a reason for hiding this comment

valentinpalkovic Nov 15, 2023

Choose a reason for hiding this comment

valentinpalkovic commented Nov 3, 2023 •

edited by storybook-bot

Loading

socket-security bot commented Nov 3, 2023 •

edited

Loading

JReinhold commented Nov 8, 2023 •

edited

Loading

socket-security bot commented Nov 14, 2023 •

edited

Loading

valentinpalkovic Nov 14, 2023 •

edited

Loading

IanVS Nov 14, 2023 •

edited

Loading