Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: bundle packages to be Ecmascript modules (T-186) #64

Merged
merged 8 commits into from
Jan 12, 2023

Conversation

desfero
Copy link
Collaborator

@desfero desfero commented Jan 5, 2023

The current support of ESM modules seems to be inconsistent and some of the libraries still don’t provide proper support.
All major bundlers support ESM but also on top of that provide good patching when a module is not proper ESM modules.
The most strict implementation has the Node.js which is the cause reason why next.js example fails.
(For .e.g when no package.json#exports is present it will switch back and use package.json#module which is the case with apollo client (apollographql/apollo-feature-requests#287 and apollographql/apollo-client#9890) but node.js doesn't support package.json#modules and therefore converts module to CommonJS which is different than what bundlers doo (see https://nodejs.org/api/esm.html#commonjs-namespaces))

I had also a look if we can get rid of folder packages (the one with just package.json files) but unfortunately, we have at least 2 blockers:

I tried to get rid also of mocks folders as they are importing devDependencies which is something tsup does bundle by default (and we want to still keep import and use the single version of jest)
I have considered using project references or importing directly src files but without any quick success:

  • Project references are not working good with ts-jest (see Tests fail when using TypeScript project references kulshekhar/ts-jest#1648)
  • Direct src imports won’t work cause ts-jest is using package.json.exports where mocks that point directly to src should not be present as we omit src folder when publishing package
    So for now we just make sure tsup doesn't bundle devDependencies by overriding external config.

The good news is that we can drop CommonJS support as all bundlers seem to properly handle ES imports/exports.

So to summarise:

  • We only now ship ESM Modules
  • Until the @apollo/client fixes the ESM modules support Next.js example won’t work without either
    • transpiling @lens-protocol package (the easiest and recommended)
    • making sure that any of the @lens-protocol imports don’t get imported by node.js (which basically means no @lens-protocol imports inside files that are imported during SSR

@desfero desfero self-assigned this Jan 5, 2023
@height
Copy link

height bot commented Jan 5, 2023

Link Height tasks by mentioning a task ID in the pull request title or commit messages, or description and comments with the keyword link (e.g. "Link T-123").

💡Tip: You can also use "Close T-X" to automatically close a task when the pull request is merged.

@desfero desfero changed the title feat: bundle packages to be Ecmascript modules compliant feat: bundle packages to be Ecmascript modules Jan 6, 2023
@desfero desfero force-pushed the feat/T-186_Support_Next.js branch from d8007d4 to 8fd1dad Compare January 6, 2023 10:41
@desfero desfero changed the title feat: bundle packages to be Ecmascript modules feat: bundle packages to be Ecmascript modules (T-186) Jan 6, 2023
@desfero desfero marked this pull request as ready for review January 6, 2023 11:13
@desfero
Copy link
Collaborator Author

desfero commented Jan 6, 2023

Link T-186

<Link to={`/profile/handle/${profile.handle}`}>
<ProfilePicture picture={profile.picture} />
</Link>
<ProfilePicture picture={profile.picture} />
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no support, for now, to provide route param with the handle so it was pointing to a nonexisting route.

@cesarenaldi
Copy link
Member

Thank you for the exhaustive explanation! I'll provide a proper code review shortly.

@desfero
Copy link
Collaborator Author

desfero commented Jan 9, 2023

There is one thing I forgot to include, the generation of d.ts.map files, which means DX gonna be bad as typescript won't nicely navigate to implementation between packages. Let's review that PR and will do the followup with DX improvements this week.

@@ -3,7 +3,7 @@
"target": "ESNext",
"useDefineForClassFields": true,
"lib": ["DOM", "DOM.Iterable", "ESNext"],
"allowJs": false,
"allowJs": true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intentional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All changes to the package.json in src/packages are intentional, this one is just making everything in sync across all tsconfig.

@cesarenaldi
Copy link
Member

Not a blocker.

Do we still need the separation between tsconfig.json and tsconfig.base.json in each (most) of the packages?

Or is that left as precursor for a packages/tsconfig shared TS config package?

@desfero
Copy link
Collaborator Author

desfero commented Jan 10, 2023

@cesarenaldi the tsconfig.base.json is left intentionally until we get a better understanding of how to solve local development issues that we have right now (a need to rebuild every time, bad auto imports).

testEnvironment: 'node',
testPathIgnorePatterns: ['/node_modules/', '/dist/'],
transformIgnorePatterns: [`/node_modules/(?!@lens-protocol/*)`],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this was previously working cause it was loading the CJS files.

Jest support for ESM confuses me all the times I look into it. What would happen if we were to keep ts-jest preset and add NODE_OPTIONS=--experimental-vm-modules to the Jest script?

@@ -12,7 +12,7 @@ import {
} from '@lens-protocol/domain/use-cases/transactions';
import { assertNever, invariant } from '@lens-protocol/shared-kernel';
import { IStorage } from '@lens-protocol/storage';
import differenceBy from 'lodash/differenceBy';
import differenceBy from 'lodash/differenceBy.js';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, are you aware of any plan for Lodash to have proper exports configuration?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is lodash-es that's under the hood using es exports, maybe it's worth to migrate at some point (or completely remove the use of lodash given it's used in just a couple of places)

@cesarenaldi
Copy link
Member

Great work! Thank you for looking into this annoying issue.

pawellula added 2 commits January 12, 2023 11:09
# Conflicts:
#	examples/web-wagmi/src/publications/components/PublicationCard.tsx
@desfero desfero merged commit cc453cc into main Jan 12, 2023
@desfero desfero deleted the feat/T-186_Support_Next.js branch January 12, 2023 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Types are broken with compilerOptions.module set to nodenext due to imports missing file extensions
2 participants