Bundling server with Rollup + esbuild #1714

infomiho · 2024-02-01T15:48:08Z

We wanted to tackle the issue of requiring our users to always use .js extension in the Wasp external imports.

One of the directions we can take is to bundle the server with e.g. Rollup and delegate the module resolution to the bundler.

Note: I've removed tsc in the server since we are using esbuild to compile TS. We can return tsc --noEmit if needed to also perform type checks.

Signed-off-by: Mihovil Ilakovac <[email protected]>

infomiho · 2024-02-01T15:57:00Z

waspc/data/Generator/templates/sdk/server/utils.ts

@@ -40,33 +40,3 @@ async (req: RequestWithExtraFields, res: Response, next: NextFunction) => {
 }

 export const sleep = (ms: number): Promise<unknown> => new Promise((r) => setTimeout(r, ms))
-
-export function getDirPathFromFileUrl(fileUrl: string): string {


Since this code relied on importing from a specific directory e.g. config and bundling gets rid of the folder structure and bundles everything in one file that we can't use this method anymore.

Aha, so we lose dynamic imports, right?

Yes, we used the dynamic imports based on folder structure, but now there is not folder structure we can rely on. All of the code is bundled together. Not a big deal, we used it only for auth provider imports.

infomiho · 2024-02-01T15:57:26Z

waspc/data/Generator/templates/server/rollup.config.js

@@ -0,0 +1,37 @@
+import esbuild from 'rollup-plugin-esbuild'


Uses Rollup and the esbuild plugin (which is fast and that's want we want in dev)

infomiho · 2024-02-01T15:57:37Z

waspc/data/Generator/templates/server/src/auth/providers/index.ts

 ];
-const providers = await importProviders(whitelistedProviderConfigFileNames);


Same reason for removing as https://github.com/wasp-lang/wasp/pull/1714/files#r1474701889

infomiho · 2024-02-01T15:57:58Z

waspc/examples/todo-typescript/main.wasp

@@ -41,7 +41,7 @@ app TodoTypescript {
    system: PostgreSQL
  },
  webSocket: {
-    fn: import { webSocketFn } from "@src/websocket/index.js",


Removing extensions and it keeps working 🎉

We have an issue with a different behaviour in our LSP and what our code will tolerate.

LSP doesn't recognise imports from folders, BUT this will run fine though

Adding index helps (this will also run fine)

Yeah this is tricky. DO you mind opening an issue for it? You can mark it with restructuring. I think we can handle this one after 0.12 is out.

I've created an issue for us to tackle this later #1717

infomiho · 2024-02-01T15:58:33Z

waspc/src/Wasp/Generator/ServerGenerator/AuthG.hs


-    enabledProviderIds =
+    providers =


Quick and dirty change to get the whole server working. Maybe this code will need refactoring later.

Any ideas how you would refactor it, or what is not great now? Might help docuemting this now while it is fresh.

I'll take another look now at the code, so I'm okay with what we merge in this PR :)

Refactored it a bit in d2d1b4d

Now it feels fine 👍

infomiho · 2024-02-02T09:40:12Z

waspc/data/Generator/templates/server/package.json

-    "build-and-start": "npm run build && npm run start",
-    "watch": "nodemon --exec 'npm run build-and-start || exit 1'",
+    "bundle": "rollup --config --silent",
+    "start": "npm run validate-env && NODE_PATH=dist node --enable-source-maps -r dotenv/config bundle/server.js",


Using source maps for better error reporting 👍

infomiho · 2024-02-02T09:42:18Z

waspc/data/Generator/templates/server/rollup.config.js

+        sourcemap: true,
+      },
+    ],
+    external: (id) => !/^[./]/.test(id),


I saw this pattern in the wild, it basically means: anything that's not a relative import, it's a dependency and don't try to bundle it together. https://rollupjs.org/configuration-options/#external

Interesting! We should certainly provide this comment inside this file, since it won't be obvious.

Martinsos · 2024-02-02T12:45:29Z

We wanted to tackle the issue of requiring our users to always use .js extension in the Wasp external imports.

One of the directions we can take is to bundle the server with e.g. Rollup and delegate the module resolution to the bundler.

Note: I've removed tsc in the server since we are using esbuild to compile TS. We can return tsc --noEmit if needed to also perform type checks.

So it doesn't do typechecking in the terminal any more? That sucks a bit. I know Vite does same thing, but I also think that sucks. I know IDE does the typechecking, but I love (and some other people in Discord said the same, or at least one person) when the actual tool that is running my development also let's me know about errors, so it works even if I don't open it in editor and search for type errors. I want to know in terminal all is fine, from the "credible" source. But you said we can return tsc --noEmit then?

Martinsos

@infomiho left a bit of comments, but I will let @sodic do the real review and approve this, since he has more knowledge than me. But I like the change!

Martinsos · 2024-02-02T12:37:21Z

waspc/src/Wasp/Generator/ServerGenerator/AuthG.hs


-    enabledProviderIds =
+    providers =


Any ideas how you would refactor it, or what is not great now? Might help docuemting this now while it is fresh.

Martinsos · 2024-02-02T12:38:03Z

waspc/data/Generator/templates/sdk/server/utils.ts

@@ -40,33 +40,3 @@ async (req: RequestWithExtraFields, res: Response, next: NextFunction) => {
 }

 export const sleep = (ms: number): Promise<unknown> => new Promise((r) => setTimeout(r, ms))
-
-export function getDirPathFromFileUrl(fileUrl: string): string {


Aha, so we lose dynamic imports, right?

Martinsos · 2024-02-06T12:04:03Z

waspc/examples/todo-typescript/main.wasp

@@ -41,7 +41,7 @@ app TodoTypescript {
    system: PostgreSQL
  },
  webSocket: {
-    fn: import { webSocketFn } from "@src/websocket/index.js",


Yeah this is tricky. DO you mind opening an issue for it? You can mark it with restructuring. I think we can handle this one after 0.12 is out.

infomiho · 2024-02-06T12:51:55Z

Here's an issue to introduce type checking to the dev setup: #1718

Prototype of server bundling

0bb0256

Signed-off-by: Mihovil Ilakovac <[email protected]>

infomiho commented Feb 1, 2024

View reviewed changes

Silence rollup output. Use source maps for Node

822f4ba

infomiho commented Feb 2, 2024

View reviewed changes

infomiho added 2 commits February 2, 2024 10:51

Rewrite the rollup.config.js

de10d5a

Cleanup

9eaea5c

infomiho added 3 commits February 5, 2024 17:14

Production build

7494739

Cleanup

7c97390

Tests

8564222

infomiho changed the title ~~Prototype of server bundling~~ Bundling server with Rollup + esbuild Feb 5, 2024

Revert email sender change

3a224e0

Martinsos reviewed Feb 6, 2024

View reviewed changes

infomiho mentioned this pull request Feb 6, 2024

Adjust the way LSP finds user's files #1717

Open

Refactor

d2d1b4d

infomiho mentioned this pull request Feb 6, 2024

Explicitly type-check user code during wasp compile #1718

Open

infomiho merged commit b23e4ce into main Feb 7, 2024
1 of 4 checks passed

infomiho deleted the miho-bundle-server branch February 7, 2024 15:13

sodic mentioned this pull request Aug 2, 2024

Make Ext import extensions consitent #2222

Open

sodic mentioned this pull request Jan 17, 2025

Type-check server code when building #2458

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bundling server with Rollup + esbuild #1714

Bundling server with Rollup + esbuild #1714

infomiho commented Feb 1, 2024

infomiho Feb 1, 2024

Martinsos Feb 2, 2024

infomiho Feb 6, 2024

infomiho Feb 1, 2024

infomiho Feb 1, 2024

infomiho Feb 1, 2024

infomiho Feb 5, 2024 •

edited

Loading

Martinsos Feb 6, 2024

infomiho Feb 6, 2024

infomiho Feb 1, 2024

Martinsos Feb 2, 2024

infomiho Feb 6, 2024

infomiho Feb 6, 2024

infomiho Feb 2, 2024

infomiho Feb 2, 2024

Martinsos Feb 2, 2024

Martinsos commented Feb 2, 2024 •

edited

Loading

Martinsos left a comment

Martinsos Feb 2, 2024

Martinsos Feb 2, 2024

Martinsos Feb 6, 2024

infomiho commented Feb 6, 2024

		];
		const providers = await importProviders(whitelistedProviderConfigFileNames);

Bundling server with Rollup + esbuild #1714

Bundling server with Rollup + esbuild #1714

Conversation

infomiho commented Feb 1, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

infomiho Feb 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Martinsos commented Feb 2, 2024 • edited Loading

Martinsos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

infomiho commented Feb 6, 2024

infomiho Feb 5, 2024 •

edited

Loading

Martinsos commented Feb 2, 2024 •

edited

Loading