Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import specifiers in --experimental-strip-types #214

Closed
GeoffreyBooth opened this issue Jul 13, 2024 · 45 comments
Closed

Import specifiers in --experimental-strip-types #214

GeoffreyBooth opened this issue Jul 13, 2024 · 45 comments
Labels
typescript Discussions related to TypeScript typescript-agenda Issue or PRs to be discussed during TypeScript team meeting

Comments

@GeoffreyBooth
Copy link
Member

This is a dedicated thread for the issue of file extensions in import specifiers in TypeScript code. Refs: nodejs/node#53725 (comment), #208 (comment), nodejs/node#53725 (comment), nodejs/node#53725 (comment), among others.

Current proposal: Users write full filenames with extensions in their specifiers, similar to ES module JavaScript: import './file.ts'. Also, .ts extensions are required in require calls, similar to the .cjs extension in CommonJS JavaScript: require('./file.ts').

  • This can be type-checked by tsc via the allowImportingTsExtensions tsconfig.json option.
  • This aligns with ES module JavaScript and Deno.
  • tsc cannot compile such TypeScript files into JavaScript (today) but many other build tools can. However, there’s not necessarily a need to compile TypeScript files into JavaScript when they can be run directly. The only benefit that a build step would provide would be for startup time performance (it would be slightly faster to evaluate JavaScript directly rather than stripping types from TypeScript and then evaluating it) and users concerned about this should arguably be using a full-featured bundler such as Vite rather than a one-to-one transpiler such as tsc.

Extension searching alternative: Users would leave off extensions, similar to CommonJS: import './file'. This is a very common way for TypeScript code to be written today, because most TypeScript code is transpiled down to CommonJS JavaScript where such specifiers are permitted.

  • This can’t be supported in require statements, as it would be a semver-major breaking change.
  • It would mean that we can’t add new file formats in the future without them also being breaking changes.
  • If this is implemented by the same pass that replaces types with whitespace, where the extension is added before the source is evaluated, then we have the same problems introduced by supporting transforms: line and column numbers would shift and we would need to add support for source maps, slowing down execution.
  • If this is implemented in Node’s resolution step, then resolution rules for ES module JavaScript would differ based on whether the importing module was a JavaScript file or a TypeScript file. This would be hard to document and explain, and would be a difficult edge case for userland customization hooks to need to support.

Replace .js alternative: Users would write import '/file.js' to refer to file.ts. This is what tsc recommends for users using tsc as their build tool, as it compiles .ts files to .js files one-to-one and never rewrites import specifiers.

  • This is counterintuitive, as there are no .js files generated as part of running files via strip-types.
  • What if both file.js and file.ts exist? None of the various options (error, load one file or the other) are obviously correct or consistent.
  • Rewriting file.ts to file.ts is arguably something that’s logically handled by the build tool or bundler, as that’s the tool generating the file.js file to which the new import specifier would refer. It’s hard to see how it’s Node’s responsibility to make up for a shortcoming in a TypeScript build tool.
  • If this is implemented in Node’s resolution step, then resolution rules for ES module JavaScript would differ based on whether the importing module was a JavaScript file or a TypeScript file. This would be hard to document and explain, and would be a difficult edge case for userland customization hooks to need to support.
@GeoffreyBooth GeoffreyBooth added the typescript Discussions related to TypeScript label Jul 13, 2024
@cspotcode
Copy link
Contributor

In cases where some .js code is never compiled at all, and imports code that may or may not be compiled (it shouldn't have to know or care), the non-compiled code uses .js import specifiers to refer to .ts files. In production when these .ts files are compiled to .d.ts and .js, the code works. And in development using a loader, the .js resolves to .ts and the code works.

then resolution rules for ES module JavaScript would differ based on whether the importing module was a JavaScript file or a TypeScript file

Would not be true, because non-compiled .js can import .ts using a .js import specifier.

This is not as confusing as people think. Why should ./foo point to ./foo.js? There is no file foo on the filesystem. But ./foo logically maps to ./foo.js.

IIRC I implemented this behavior in ts-node: https://typestrong.org/ts-node/docs/options#experimentalresolver

@targos
Copy link
Member

targos commented Jul 14, 2024

However, there’s not necessarily a need to compile TypeScript files into JavaScript when they can be run directly

What about package consumers who don't care about TypeScript? If packages are published without a build step, they will be left on the side or they'll have to run their application with the strip-types flag only because of their dependencies.

Also there's something that really needs a deep investigation: is the TypeScript type checker able to work with packages in node_modules that haven't gone through a build step that converts the .ts files to .d.ts? If the answer is no, we should definitely not recommend people to publish their packages without a build step.

users concerned about this should arguably be using a full-featured bundler such as Vite rather than a one-to-one transpiler such as tsc.

We are in deep disagreement here. I personally hate packages that are published as bundles.

@targos
Copy link
Member

targos commented Jul 14, 2024

Also there's something that really needs a deep investigation: is the TypeScript type checker able to work with packages in node_modules that haven't gone through a build step that converts the .ts files to .d.ts? If the answer is no, we should definitely not recommend people to publish their packages without a build step.

I made some local tests and it looks like tsc and esbuild are able to consume packages with .ts files in the "exports" field, so that's good.

@kibertoad
Copy link

Other than .js => .ts extension that requires the flag allowImportingTsExtensions, and I'm strongly against supporting .js for ts imports,

@marco-ippolito Can you elaborate on this? Isn't supporting ".js" for TS imports consistent with how tsc works? Why introduce new semantics?

@marco-ippolito
Copy link
Member

marco-ippolito commented Jul 14, 2024

Other than .js => .ts extension that requires the flag allowImportingTsExtensions, and I'm strongly against supporting .js for ts imports,

@marco-ippolito Can you elaborate on this? Isn't supporting ".js" for TS imports consistent with how tsc works? Why introduce new semantics?

There is a typescript flag that allows .ts imports allowImportingTsExtensions, so it is supported. Its not the default but it is consistent

@targos
Copy link
Member

targos commented Jul 14, 2024

There is a typescript flag that allows .ts imports allowImportingTsExtensions, so it is supported. Its not the default but it is consistent.

TypeScript does not allow to emit (transpile) code when this flag is enabled, so there's some level of inconsistency.
If I want to use --experimental-strip-types in development and publish my package on the npm registry, I will have to install two tools:

  • typescript for type checking
  • Something else that supports rewriting .ts specifiers to .js, for transpilation

@marco-ippolito
Copy link
Member

There is a typescript flag that allows .ts imports allowImportingTsExtensions, so it is supported. Its not the default but it is consistent.

TypeScript does not allow to emit (transpile) code when this flag is enabled, so there's some level of inconsistency. If I want to use --experimental-strip-types in development and publish my package on the npm registry, I will have to install two tools:

  • typescript for type checking
  • Something else that supports rewriting .ts specifiers to .js, for transpilation

I believe this is something we can discuss with the TypeScript team as its a problem also shared by other runtimes

@GeoffreyBooth
Copy link
Member Author

@cspotcode The point of type stripping is to not need to use a build tool or bundler. Deno and Bun run TypeScript files directly, including in production, and our type stripping is intended to do the same. Yes this means that startup time is slightly impacted, but that’s not very concerning for long-lived servers, which are the most common use case for Node apps. And if you’re coding for a use case where startup time does matter, like a CLI tool or a serverless/lambda function, then you shouldn’t simply transpile files one-to-one, you should bundle them like you would for the browser.

We are in deep disagreement here. I personally hate packages that are published as bundles.

@targos When I wrote that I was thinking of applications, not packages. But to your point, I think what we hate about packages published as bundles is when they bundle their dependencies, inlining them rather than treating them as external dependencies that can be deduped. I don’t think it’s terribly offensive when package authors use a build tool to optimize their library while externalizing dependencies, so that my app loads node_modules/foo/dist/lib.js and everything is in one minified file, but that file contains nothing more than foo’s code; any imports or requires to foo’s dependencies are still resolved to other packages under node_modules just like if a bundler hadn’t been used.

Maybe you’re also offended by package authors who minify or otherwise make their packages hard to debug at runtime, but ultimately you’re the one who’s choosing whose open source packages to use, and you can evaluate them based on whatever criteria you want. The Svelte team writes apps in TypeScript and libraries in JavaScript with JSDoc types, so that their libraries don’t need a build step yet can be consumed by anyone while still getting type checking, and that feels like the best model to me. If we're going to recommend anything, that’s what I would recommend.

What about package consumers who don’t care about TypeScript? If packages are published without a build step, they will be left on the side or they’ll have to run their application with the strip-types flag only because of their dependencies.

This is the same argument as any new or nonstandard syntax. We had this same discussion around --experimental-specifier-resolution=node: what if people publish packages that tell users they need to enable this flag in order to use their package? We could be having it today around --experimental-require-module: what if people publish packages that require(esm) and tell their users that they need to enable that flag in order to use their package? You could even make the argument today specifically about TypeScript: what if people publish packages in TypeScript and tell users they need to pass --import=tsx and run a particular tsconfig.json to run their package. And so on.

Generally public package authors don’t want their libraries to be hard to use, or they won’t attract many users. And private package authors, like corporate users writing packages for specific teams whom they know and can coordinate with, might choose to opt into experimental features since they know their users can handle it and want to. It’s not really our place to intrude in the relationship between package authors and consumers.

While I agree in principle that we should do what we can to prevent users from shooting themselves in the foot, where I draw the line is that I don’t think we should intentionally limit Node, blocking perfectly valid and unobjectionable use cases, just because some package authors might misuse a feature. By that logic we shouldn’t have shipped require(esm) because package authors might use it and publish packages expecting it to be enabled and some of their consumers don’t have it enabled. Limiting Node just isn’t the right solution for encouraging package authors to publish maximally compatible packages, and I don’t think it’s really necessary: public package authors generally want their packages to be used, so they have every incentive to publish with as broad a target audience as is feasible.

@GeoffreyBooth
Copy link
Member Author

Also, the cat is already out of the bag: Bun evaluates TypeScript within node_modules:

mkdir temp && cd temp
mkdir -p ./node_modules/pkg
echo 'export const message: string = "Hello!"' > ./node_modules/pkg/index.ts
echo 'import { message } from "pkg"; console.log(message)' > test.ts
bun test.ts
Hello!

@mcollina
Copy link
Member

tsc cannot compile such TypeScript files into JavaScript (today) but many other build tools can. However, there’s not necessarily a need to compile TypeScript files into JavaScript when they can be run directly. The only benefit that a build step would provide would be for startup time performance (it would be slightly faster to evaluate JavaScript directly rather than stripping types from TypeScript and then evaluating it) and users concerned about this should arguably be using a full-featured bundler such as Vite rather than a one-to-one transpiler such as tsc.

This is the biggest problem for me. TypeScript does not have a spec, but it has a reference implementation. If we evaluate code as .ts, a user would expect that code to be compilable by tsc.

@AlttiRi
Copy link

AlttiRi commented Jul 26, 2024

From #217 issue:

I strongly believe we should not support .js extension for .ts files.
The reason is that the compiler/bundler should be responsible to resolve the correct extension at compile time.

For distribution, if a developer cares about performance, he should provide already compiled .ts to .js.

Running .ts file is very useful (I already use this feature in Beta version of WebStorm), but mostly for development. One extra millisecond for file resolving is acceptable.


It would be fine, if @microsoft TypeScript team will change extend the behaviour of allowImportingTsExtensions.

Currently, it has the follow description:

Allow imports to include TypeScript file extensions. Requires '--moduleResolution bundler' and either '--noEmit' or '--emitDeclarationOnly' to be set.

That requirement is unacceptable for library projects that use tsc --build for compiling with emitting .js/.d.ts/.map.js files.

I think, if the allowImportingTsExtensions will work without any "Requires", then importing of .js can be abandoned.


Here is a library config to compile .ts to .js with .d.ts files with tsc --build command.

    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "noEmit": false,
    "declaration": true,

It works, but it requires to use .js in imports everywhere.

Adding "allowImportingTsExtensions": true, "breaks" it:

error TS5096: Option 'allowImportingTsExtensions' can only be used when either 'noEmit' or 'emitDeclarationOnly' is set.

I just say it again:

I think, if the allowImportingTsExtensions will work without any "Requires", then importing of .js can be abandoned in TS projects.

Why do these limitations exist?

This flag is only allowed when --noEmit or --emitDeclarationOnly is enabled, since these import paths would not be resolvable at runtime in JavaScript output files. The expectation here is that your resolver (e.g. your bundler, a runtime, or some other tool) is going to make these imports between .ts files work. *

Can't TS just replace ".ts" string with ".js" string in imports while compiling?

@Tobbe
Copy link

Tobbe commented Jul 26, 2024

Can't TS just replace ".ts" string with ".js" string in imports while compiling?

No, because it might be a package named foo.ts 🙂

See also nodejs/node#53725 (comment)

@mitschabaude
Copy link

mitschabaude commented Jul 28, 2024

I'm very excited about node --experimental-strip-types, it promises to simplify and unify the experience of running your code (tests etc) during development of a TS library.

However, the Current proposal (which seems to be strongly supported by @GeoffreyBooth and @marco-ippolito) is the single reason that makes --experimental-strip-types useless for me at the moment.

For me to use type stripping in practice, the behavior described in "Replace .js alternative" would be needed: Imports specifying a .js file would need to be resolved to their .ts source.

The reason is that my libraries all use ESM and are all written to be compiled by tsc. Since tsc requires .js extensions in imports when compiling ESM projects, all my libraries are written with .js imports. Thus, in the current implementation and proposal, --experimental-strip-types would not be able to run any of my code, because of being incompatible with tsc.

I really hope that type stripping can support .js imports in the future, and in the remaining post want to address the various points made against that support!

This is counterintuitive, as there are no .js files generated as part of running files via strip-types.

Being usable is way more important than being intuitive.

What if both file.js and file.ts exist? None of the various options (error, load one file or the other) are obviously correct or consistent.

This seems like a non-issue in practice. Projects will either have .ts or .js files. Also, there is an obvious and simple resolution option: .js files get preference if a .js import specifier is used. This solution will offend or confuse absolutely nobody.

It has also been argued in the original PR that extra checks for file existence implies a performance hit.
I don't buy this argument, because trying to read the .js file is already what the "current proposal" does on a .js import. Only when no .js import is found would you also check for a .ts file and read it. So, the performance hit only affects a case that is currently not supported.

I'm perfectly willing to accept the cost of an extra check for a .js file before my .ts file gets loaded. After all, I would only use strip-types during development to run my files and tests. For release, I would strip them of types using tsc anyway.

@GeoffreyBooth seems to have a more expansive vision: Everyone ships TS files directly. I don't think this vision is realistic to achieve in the medium term for reasons I won't elaborate on here. I just want to note that in my advocated solution there is no performance hit when loading .ts files by specifying a .ts import. (In that case, you would obviously look for a .ts file first, and it's fine if no other extension is supported.)

@cspotcode The point of type stripping is to not need to use a build tool or bundler.

Any serious library will always use tsc to check types. Or at least, that's what I would absolutely always do. Otherwise it's too easy to ship broken code.

For me, the point of type stripping is to not have to use any build tool other than tsc. The use case for strip-types would be quickly running my TS files directly. Currently I need some script or other installed tool like esbuild or tsx for that, which has downsides.

But using no other build tool than tsc is precisely what the current behavior makes impossible, as @targos explained here: #214 (comment)

We are in deep disagreement here. I personally hate packages that are published as bundles.

@targos When I wrote that I was thinking of applications, not packages.
...
Maybe you’re also offended by package authors who minify or otherwise make their packages hard to debug at runtime, but ultimately you’re the one who’s choosing whose open source packages to use, and you can evaluate them based on whatever criteria you want.

I agree with @targos here, I also hate packages published as bundles. And yes, I am indeed also offended by package authors who minify or otherwise make their packages hard to debug at runtime. And I personally want to publish libraries that are not minified or bundled. That's why I like to only use tsc for compiling!

The Svelte team writes apps in TypeScript and libraries in JavaScript with JSDoc types, so that their libraries don’t need a build step yet can be consumed by anyone while still getting type checking, and that feels like the best model to me. If we're going to recommend anything, that’s what I would recommend.

I don't think it's the Node.js TSC's job to recommend any particular opinionated way of writing libraries. I fully respect Svelte's style of doing things, but it should be out of question that writing TS source files is also a valid approach.

Using TS for libraries is the style that I and apparently most people prefer, so it should clearly be considered in a discussion of how to support TS in node!

where I draw the line is that I don’t think we should intentionally limit Node, blocking perfectly valid and unobjectionable use cases

I couldn't agree more! That's why I recommend resolving .js imports to .ts files if no .js file exists. Not doing so would block perfectly valid and unobjectionable use cases (including all of my own use cases for type stripping), and we surely don't want to intentionally limit Node 🙂

@marco-ippolito marco-ippolito added the typescript-agenda Issue or PRs to be discussed during TypeScript team meeting label Aug 7, 2024
@alshdavid
Copy link

@mitschabaude it appears the TypeScript team are considering options to resolve this in tsc

@mitschabaude
Copy link

mitschabaude commented Aug 21, 2024

@JakobJingleheimer I'm moving our discussion here from #217 (comment)

In general I get the argument that "guessing an extension" is a step backwards for Node.js. Nonetheless it's a much lighter form of guessing than we used to have in CJS.
I also get the argument from the TS team that rewriting import paths is complicated.

Also, thanks @JakobJingleheimer for pointing out that rewriting the imports can also be accomplished using a Node.js resolve hook. That might become my method of choice for now.

There's just one point where I think you and other Node.js team members are a bit off: with the recommendation to "just use esbuild instead of tsc". This is not meant in an unfriendly way, we all have our blind spots 😅 But I want to try to change your perspective on that!

  • tsc is one of the most popular ways of transpiling TS. According to the 2023 State of JS survey, it's used by 52% of all JS developers. (slightly ahead of esbuild in that survey, swc is probably used more via webpack and vite)
    • not all of those 52% are using a node/ESM setup, but it should be clear that a solution which rules out tsc compilation leaves a LOT of people unsatisfied
  • esbuild is a bundler, and can't map a TS code base to a JS code base with the same file layout. For some use cases, this makes it just not the right tool.
    • Libraries. Personally, I'm mainly a library author, not an app developer. As a library author, I want my TS code base compiled into JS files that look as closely as possible to the source. Bundling everything into one file harms debugging of the code I ship. tsc is just perfect for that job.
    • Relative path preservation. Some dev patterns rely on referring to a relative position in the file system in places other than imports. For example, loading a Wasm file or starting a Node.js Worker with source file in a relative path. Bundlers are focused on resolving imports and are bad with handling those other ways of referring to a relative file. I use both Wasm and workers a lot and off-the-shelf tools (that rely on esbuild) like npx tsx are often not able to run my TS code for exactly this reason. (A complicated esbuild setup can probably do it, but then again -- it's much simpler with just tsc + Node)
  • tsc is literally the TS spec. Famously, TS doesn't have a spec and tsc's behaviour is basically the language definition. For better or worse. TS features are "released" when they become supported by tsc. Esbuild and others are necessarily lagging behind in supporting those features. They have no concept of TS versions so they can't be relied on for handling TS code that's written for one specific version of TS. The recommendation "just use esbuild" harms adoption of the newest or advanced features, and might (therefore) even slow down evolution of TS as a whole.
  • tsc is the only TS type-checker, so it's needed anyway. Even assuming none of the above matters: Any sufficiently advanced TS project will eventually want to run tsc's type-checker on the source code. So, if we want to both compile the source to JS, and type-check it, then not being able to compile with tsc means we need two different build tools. For some of us including me, the whole benefit of strip-types is that we don't need a third tool for TS + Node development apart from tsc and Node (which we will need anyway). The proposal to use esbuild instead of tsc to build therefore crushes the value proposition of strip-types: We do need a third tool after all. (If I have to set up esbuild in my project, then I'm not far from crafting a little npm script that runs any source file through esbuild and executes it on the spot. In other words, I don't need strip-types.) (A similar script using tsc is much worse because of speed, so if I can use only tsc + Node, then the addition of strip-types provides significant value)

@aduh95
Copy link
Contributor

aduh95 commented Aug 21, 2024

I also get the argument from the TS team that rewriting import paths is complicated.

It might not be as complicated as originally thought: microsoft/TypeScript#59597 (comment)

@mitschabaude
Copy link

I also get the argument from the TS team that rewriting import paths is complicated.

It might not be as complicated as originally thought: microsoft/TypeScript#59597 (comment)

that comment was written before the meeting that I linked to

@kyubisation
Copy link

I would also like to thank the Node.js maintainers/community for the work in this area!

However, I would like to second the opinion of @mitschabaude in regards to the .ts/.js imports.
My perspective is primarily from a library developer as well.

From my understanding, could this not be partially achieved with an ESM hook? Maybe only conditionally active when --experimental-strip-types or --experimental-transform-types is used?

This could of course be implemented in user space, but then I feel like the usability of the Node.js TypeScript integration is unnecessarily limited.
If performance is a concern, then the recommendation for users would very well be to compile the code to JavaScript before running the code anyway.

Thank you for your consideration.

Naive implementation:

Code
import { existsSync } from "node:fs";
import { basename, dirname, extname } from "node:path";

const typeScriptExtensions = [".ts", ".mts", ".tsx"];

/**
 * Example resolve for resolving .js imports from TypeScript files
 * that point to other TypeScript files, instead of .js files.
 * If resolving fails, we check whether there is a corresponding
 * TypeScript file.
 */
export const resolve = async (specifier, context, nextResolve) => {
  try {
    return await nextResolve(specifier, context);
  } catch (err) {
    if (
      !err.url ||
      !specifier.endsWith(".js") ||
      !context.parentURL ||
      !typeScriptExtensions.includes(extname(context.parentURL))
    ) {
      throw err;
    }

    const extensionlessUrl =
      dirname(err.url) + "/" + basename(err.url, extname(err.url));
    for (const typeScriptExtension of typeScriptExtensions) {
      const maybeTypeScriptFile = extensionlessUrl + typeScriptExtension;
      if (existsSync(new URL(maybeTypeScriptFile))) {
        return {
          format: "module",
          shortCircuit: true,
          url: maybeTypeScriptFile,
        };
      }
    }

    throw err;
  }
};

Playground (You might need to switch to the zsh Terminal and run npm test)

@JakobJingleheimer
Copy link
Member

From my understanding, could this not be partially achieved with an ESM hook?

It absolutely can and has been ;)

An argument about how difficult it would be for tsc to handle this properly is nonsense: it is ALREADY doing the resolution—just not outputting it.

@kyubisation
Copy link

As previously stated, this is complicated.
The argument is that it is very easy to do inbuilt in Node.js, whereas it appears to require shimming in the TypeScript output, which is not great.

@aduh95
Copy link
Contributor

aduh95 commented Aug 25, 2024

@kyubisation I took the liberty of reviewing your proposed loader:

  • You could do import { posix as path } from 'node:path' so your code still works on Windows.
  • You could use format: "module-typescript" so Node.js strips the types for you (assuming you passed the --experimental-strip-types flag).
  • You could check that err.url is a file: URL (otherwise your users would get a weird error when you call existsSync).

As is, it would still have some inconsistencies with tsc (on the top of my head: no support for .cts, a .mjs import should be able to resolve to a .mts file, you assume .ts would be ESM and never CJS), but maybe those are not important for your use-case.

The argument is that it is very easy to do inbuilt in Node.js, whereas it appears to require shimming in the TypeScript output, which is not great.

If I had to fork one of the project to "make it work somehow", I would pick Node.js, but that's because I'm more familiar with the codebase (i.e. I can't really speak for how easy/hard it would be to implement in tsc codebase). However:

  • My personal opinion would be that it would be much better to have specifier-rewrites in tsc, it's one of the pain points of working with TS files. Why wouldn't we push for it when we can?
  • ESM implementation of Node.js got rid of extension guessing for good reasons, reintroducing it will force us to face the same rough edge cases, and/or to allow inconsistencies in the resolve algorithm.
  • No one is working on it AFAIK, and nothing is going to happen until someone submits a PR (and note that such PR would be unlikely to get consensus).
  • It's an experimental feature, such incompatibilities should not be surprising IMO.

I'm hopeful the tsc discussion will resolve soon-ish, AFAICT there are no deal breakers, only a few challenges. The ball is on their court, I'm rooting for them!

@kyubisation
Copy link

kyubisation commented Aug 25, 2024

As mentioned:

Naive implementation:

But nevertheless, thank you.

I would be open to contribute, but as you mention, there is little point until consensus is reached.
Your (and others) arguments are also a valid assessment, but as it is, it is not useable in the projects I'm involved in.

From my understanding/viewpoint in following the discussion around the strip and transform types features, it did not seem as though there was much consumer research and the opinions of larger players (e.g. TypeScript or Vite) were, in my eyes, either downplayed or ignored. (Of course, my perspective is limited and I might have missed things.)
That is of course the prerogative of the Node.js maintainers, as they/you are the ones putting in the effort and providing support.

So, my comment is just meant as another feedback from a potential and interested consumer.

@kyubisation
Copy link

My comment might have been a bit harsh. The Node.js is doing the best they can.
I can wait.

@damianobarbati
Copy link

Guys I'm not sure if this is the right discussion, but I have 2 questions about the flag.

  1. The flag works good for snippets but once you import something:
import foe from "./foe";

it fails, since I'm required to provide an extension and I'm reluctant to migrate my codebase fully to ".ts" extensions everywhere: if anything happen I can't switch easily back to tsx.

  1. There's no way we'll have the relative paths support? Since the flag is rewriting the original code stripping types, could it also rewrite the import strings to full absolute paths?

@JakobJingleheimer
Copy link
Member

JakobJingleheimer commented Aug 30, 2024

There's no way we'll have the relative paths support? Since the flag is rewriting the original code stripping types, could it also rewrite the import strings to full absolute paths?

Stripping types is far simpler than what you're asking.

Please read the rest of this thread, such as the posts immediately above yours, for information about your question 🙂

@aduh95
Copy link
Contributor

aduh95 commented Aug 30, 2024

Since the flag is rewriting the original code stripping types, could it also rewrite the import strings to full absolute paths?

When Node.js is run with --experimental-strip-types and loads a .ts file, it uses SWC to replace all type declarations with white spaces, that way error stack traces still points correctly to the correct line/column, without needing to use a source map (for performance reasons). Say it another way, it only removes things from the original code, it's a bit of a stretch to call that "rewriting the original code", although that's technically not incorrect.
As you can imagine, rewriting import specifiers would be quite a dramatic change from our current policy, so unlikely to get consensus. Also, I'm curious why you think using absolute paths (or rather URLs) would be helpful, maybe you didn't mean absolute/relative paths, but rather you meant adding the extension to an extension-less specifier? In any case, it wouldn't help resolve the issue, we would still need to maintain an extension guessing algorithm, and that's best solved by a user-land hook as e.g. the one suggested in #214 (comment).

@marco-ippolito
Copy link
Member

marco-ippolito commented Sep 11, 2024

In today's TypeScript team meeting emerged that TypeScript will create an emit flag that rewrites the .ts => .js and supports Node module resolution. Consensus is that Node should not perform extension guessing or support extensionless imports.

@acutmore
Copy link

@marco-ippolito what do you think is the best way to handle package imports?

// package.json
{
  "imports": {
    "#dep": "./path/to/dep.ts"
  },

Would there be a way to re-write these paths as part of npm publish?

@JakobJingleheimer
Copy link
Member

Sorry, what's the problem with subimports? I just added a case for them in my codemod, and they seem to work fine out of the box.

@jakebailey
Copy link

jakebailey commented Sep 11, 2024

I think the most straightforward thing here is to make a condition specific to your project such that when it's enabled, it points to the source, but otherwise points to the emitted JS. (e.g. in the same vein as https://colinhacks.com/essays/live-types-typescript-monorepo?q=1 or tshy)

https://github.com/arethetypeswrong/arethetypeswrong.github.io/pull/194/files#diff-0b810c38f3c138a3d5e44854edefd5eb966617ca84e62f06511f60acc40546c7R30 is an example of this working for import conditions.

@mitschabaude
Copy link

mitschabaude commented Sep 11, 2024

In today's TypeScript team meeting emerged that TypeScript will create an emit flag that rewrites the .ts => .js and supports Node module resolution. Consensus is that Node should not perform extension guessing or support extensionless imports.

Here's the PR to watch: microsoft/TypeScript#59767

Currently it says this:

⚠️ This PR does not indicate intent to ship; I just needed a published experimental build of it for some experiments and demos. ⚠️

I'll keep my fingers crossed 😅

@marco-ippolito
Copy link
Member

@marco-ippolito what do you think is the best way to handle package imports?

// package.json
{
  "imports": {
    "#dep": "./path/to/dep.ts"
  },

Would there be a way to re-write these paths as part of npm publish?

Interesting case, can you open a separate issue on the nodejs/typescript repo? I think this is worth discussing

@electrovir
Copy link

fwiw, I've already been using .ts exports in my package.json files (for mono-repo purposes) but then I use a custom npm run publish script (vs npm publish, but it does call npm publish) that converts them from src/index.ts to dist/index.js, then reverts it back after the publish is complete.

@damianobarbati
Copy link

damianobarbati commented Sep 12, 2024

Hey guys, I made a repro here => https://github.com/damianobarbati/strip-types-repro

For anyone looking for an all-in-one repro on how to have the following all working together:

  • ESM
  • Typescript
  • Relative paths
  • Biome
  • Vitest

cc @marco-ippolito @electrovir @aduh95 @JakobJingleheimer

😈

@mitschabaude
Copy link

Hey guys, I made a repro here => https://github.com/damianobarbati/strip-types-repro

Is that "#*": "./src/*", hack in package.json needed? If yes, why?

What's missing from your repo IMO is creating compiled JS output (tsc doesn't handle that because of noEmit: true)

@acutmore
Copy link

Interesting case, can you open a separate issue on the nodejs/typescript repo? I think this is worth discussing

I think @jakebailey 's comment links to good solutions for this. The package-scoped condition seem simplest:

"exports": {
     ".": {
       "@<package-name-here>/source": "./src/index.ts",
       "default": "./dist/index.js"
     },

Part of me wishes there was an environment variable (more granular than NODE_OPTIONS) to set the condition. To reduce the need to node -C @<package-name-here>/source ..., but that's a small thing that maybe only I feel would be nice 😄

@alshdavid
Copy link

alshdavid commented Sep 13, 2024

As a library maintainer who wants to run source TS files during development but distribute javascript for production; I have been experimenting with conditional imports + a bundler for production/distributed files

// package.json
{
  "name": "my-lib",
  "type": "module",
  "exports": {
    ".": {
       "source": "./src/lib.ts",
       "types": "./src/lib.ts", // Is it valid to use typescript sources as types?
       "default": "./dist/lib.mjs"
    }
  }
}

During development I run:

node --conditions="source" --experimental-strip-types ./test/consumer.ts
// ./test/consumer.ts
console.log(await import('my-lib'))

And to build I run:

npx esbuild --bundle --platform=node --packages=external --outfile=./dist/lib.mjs --format=esm ./src/lib.ts

Though this gets complicated when you need to use worker threads. For this I use package.json#import

// package.json
{
  "import": {
    "#worker": {
       "source": "./src/worker.ts",
       "default": "./dist/worker.mjs"
    }
  }
}
import * as url from 'node:url'
new Worker(url.fileURLToPath(import.meta.resolve('#worker')))

And for bin entries I use a javascript file that proxies to a package.json#import

// package.json
{
   "bin": "./bin.js",
   "import": {
      "#bin": {
        "source": "./src/bin.ts",
        "default": "./dist/bin.mjs"
      }
    }
  }
}
// ./bin.js
await import('#bin')

With Nodejs having a built-in test runner and typescript support, I can drop a bunch of dependencies, which is nice for project longevity.

Looking forward to eventually being able to use tsc directly for the build step and remove the bundler dependency.

@ljharb
Copy link
Member

ljharb commented Sep 14, 2024

Interesting case, can you open a separate issue on the nodejs/typescript repo? I think this is worth discussing

I think @jakebailey 's comment links to good solutions for this. The package-scoped condition seem simplest:

"exports": {
     ".": {
       "@<package-name-here>/source": "./src/index.ts",
       "default": "./dist/index.js"
     },

Part of me wishes there was an environment variable (more granular than NODE_OPTIONS) to set the condition. To reduce the need to node -C @<package-name-here>/source ..., but that's a small thing that maybe only I feel would be nice 😄

im confused; non-conditions objects’ LHS are all required to start with a dot, to prevent precisely the scenario of redirecting a package import to a dependency. did that change at some point?

@jakebailey
Copy link

Within that block, the property names are the condition names, not names of imports; the implication is that you write this when you want to run from source (taken from the ATTW PR):

$ node --experimental-transform-types --conditions=@arethetypeswrong/source --test 'test/**/*.test.ts'

(Using just "source" itself is potentially problematic because you can end up in the situation where a package you don't own may publish also with the condition "source" and now you're running someone else's source.)

@ljharb
Copy link
Member

ljharb commented Sep 14, 2024

Gotcha, that’s clever.

@mitschabaude
Copy link

TypeScript PR for .ts imports is merged 🚀
microsoft/TypeScript#59767

Woohoo!

@AlttiRi
Copy link

AlttiRi commented Oct 3, 2024

I have tested rewriteRelativeImportExtensions in TS 5.7.0-dev.

It's now possible to use allowImportingTsExtensions with the options for library projects (that are compiled with tsc) in tsconfig.json ("moduleResolution": "NodeNext" + "noEmit": false").

Also, I can now run .ts files inside such project with deno without using --unstable-sloppy-imports which prints annoying warnings. Assume, it will work with --experimental-strip-types of Node.js too.


However, currently there is a bug that imports in generated .d.ts are not rewritten. It's definitely a bug.
microsoft/TypeScript#59767 (comment)

And it does not support aliases, even while the alias is resolved to a relative path.
microsoft/TypeScript#59767 (comment)

@franklin-ross
Copy link

I wrote a small package to install a Node resolution hook so node --experimental-strip-types will work for existing regular TS code that includes .js extensions. Using .ts in your import specifiers does not seem like a good solution to me as it makes it harder to build your code into JS (if you still want to do that,) and it goes against years of Typescript coding practices.

https://www.npmjs.com/package/node-resolve-ts

@marco-ippolito
Copy link
Member

I wrote a small package to install a Node resolution hook so node --experimental-strip-types will work for existing regular TS code that includes .js extensions. Using .ts in your import specifiers does not seem like a good solution to me as it makes it harder to build your code into JS (if you still want to do that,) and it goes against years of Typescript coding practices.

https://www.npmjs.com/package/node-resolve-ts

You might want to take a look at
https://devblogs.microsoft.com/typescript/announcing-typescript-5-7-beta/

@franklin-ross
Copy link

You might want to take a look at https://devblogs.microsoft.com/typescript/announcing-typescript-5-7-beta/

Yeah sure, but there are still caveats around that as mentioned in the link, which aren't a problem with the Node resolution patch. That'll certainly make life a lot easier, and is a huge backflip considering how hard they've stuck to their guns on rewriting import extensions in the past, but I think there are plenty of cases it'll still make sense to patch Node like this.

Whatever works best for everyone, of course. Ideally we don't need glue packages like this 🤷‍♀️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
typescript Discussions related to TypeScript typescript-agenda Issue or PRs to be discussed during TypeScript team meeting
Projects
None yet
Development

No branches or pull requests