Fail to load module from node_moules

[zengfenfei]

The main ts file app.ts:

import m from "mod"
console.log('imported module', m)

The module file node_modules/mod.ts:

export default "TS module in node_modules";

app.ts and node_modules are in the same directory. When I run ts-node app.ts, reports following error:

SyntaxError: Unexpected token export
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:387:25)
at Module._extensions..js (module.js:422:10)
at Object.require.extensions.(anonymous function) as .ts
at Module.load (module.js:357:32)
at Function.Module._load (module.js:314:12)
at Module.require (module.js:367:17)
at require (internal/module.js:20:19)
at Object. (/Users/kevin.zeng/Projects/ringcentral-js-client/codegen/ts-sample/app.ts:1:1)
at Module._compile (module.js:413:34)

The code runs normally when compiled to js using tsc. Version info:

ts-node v1.0.0
node v5.11.1
tsc 1.8.10

[blakeembrey]

ts-node is not compiling dependencies. Do you feel like it should be?

[zengfenfei]

Since I'm using Typescript for my project, I also prefer Typescript modules as the dependencies. So I think it's necessary to compile dependencies or project won't work using ts-node.

[blakeembrey]

Realistically, the setup you're building is kind of awkward. You're relying on a global mutation that'll be done by a dependent. Why not just compile the files to JavaScript? What about having both projects register with ts-node? What if one project uses different compilation settings to the other? Or they have different dependencies on .d.ts files that happen to conflict when you put them together?

[huan]

@blakeembrey thanks for point me from #158 to here, sorry for not notice this issue before I post.

I do suggest ts-node to compiling dependencies. because is a good way to modulize pure typescript code if we do not want to compile .ts file any more. (it's why ts-node here, right?)

@zengfenfei I ran into the same issue as you today. I have to make this work by soft link this:

$ mv node_modules/my_mod .
$ ln ../my_mod node_modules

# then ts-node will compile it...

more detail you can find in my dup issue #158

[blakeembrey]

By the way, there's a similar issue here: #135. I think the correct solution, as mentioned in that issue, is to implement --ignore and --include flags which default to ignore: ['node_modules/**'].

[blakeembrey]

@cleavera No, they don't. You should probably do some research into the topic yourself to get familiar. TypeScript libraries should be distributed in JavaScript with declarations enabled. Please don't publish raw TypeScript libraries and fragment the ecosystem for no reason. Internally, you can do whatever you want of course.

[rjnienaber]

I had a similar problem to what @zixia had (sharing internal code between Typescript projects) which led me to this issue. Since it appears to be a design decision not to support .ts files in the node_modules folder, the solution I went with was to use a postinstall script in the library's package.json.

Example:

  "scripts": {
    "postinstall": "tsc -d -p ."
  },

I also added a tsconfig.json file to the root folder of the library with all the compiler settings.

[gaspard]

The hard coded node_modules is actually a problem for repositories relying on how require works to load sub-projects in the same monorepo (like Pouchdb, Cerebraljs). If you store your projects like this:

node_modules // <--- root node_modules for installed deps
packages
  + node_modules
    + @myorg
      + moduleA // <--- now this one can import latest '@myorg/moduleB' without linking and such
      + moduleB

This setup works fine until testing cannot happen because ts-node does not compile in node_modules (which is in the test path).

Please give us a way to fix this.

[blakeembrey]

@gaspard You can start with the README. I have no intention of supporting bad practice by default.

[gaspard]

@blakeembrey Can you please explain in what sense using a public API is bad practice ?

Using node_modules to load modules is totally part of node.js API. There is no hack whatsoever here. Reasons to use such a folder architecture are absolutely valid. Here is a discussion related to PouchDB: alle monorepo, we don't have a public summary of our discussion at Cerebral.js but the reasons are similar and I have also good reasons with Lucidity to move away from lerna.

I think there are simple ways to avoid compiling vendor modules and still allow complex open source projects to use ts-node for testing. One such way would be to be able to specify, like .gitignore does for example:

"ignore": [
  "node_modules",
  "!/packages/node_modules"
]

[blakeembrey]

@gaspard Yes, it's valid. Believe it or not, I know how the node.js module system works. I would never promote someone publishing TypeScript first class, you should be publishing JavaScript. If you've done development with node.js for a while, you'll probably remember how CoffeeScript turned out - you have a bunch of people writing .coffee files and just adding coffee-script/register so they could avoid transpiling - even after publishing to npm. I don't intend to ever support that, but it really is a two second change to support it via CLI or environment flags. You could probably have done that quicker than you posted your response.

[huan]

As my perspective, I agree with @gaspard before, but I have to say, today I'll agree with @blakeembrey.

Last year, I want to publish pure TypeScript module to NPM, and actually, I did. I also have a workaround about this, you could have a look at this thread at #155 (comment) .

The reason I want to do this is that at first:

1. I fall in love with TypeScript so I want to use it anywhere(ts only)
2. I'm not familiar with public TypeDefination file with my module
3. I'm not familiar with ES6 Module with Rollup tools

Today, I learned from Angular how to publish NPM in UMD format with TypeDefination, it's very easy to do this with an npm script, just as @blakeembrey said, really is a two seconds work. So I have no reason to keep my module in ts only.

At last, if there's no dark side of using pure TypeScript NPM module, I believe it is possible to support it by ts-node. However, the dark side I could image is: when a javascript developer found your module from NPM and npm -i it, but it could not be able to run.. This will do hurt the npm eco-system.

So today I vote NO for publishing pure TypeScript module to NPM. Hope this could help others like me to make the right decision today. :)

[gaspard]

@blakeembrey This is really a confused discussion. I will never publish a typescript module and this is not the issue.

The issue is that people work inside a repository that happens to be packages/node_modules. Look at PouchDB source or Cerebral.js.

The issue is that we cannot use ts-node to test this working directory. It has nothing to do with publishing.

For example, these are not installed modules, but the actual packages developed by Cerebral.js: https://github.com/cerebral/cerebral/tree/master/packages/node_modules

[blakeembrey]

It's exactly the same and nothing is confused here. If you check the README you can see how to get around it (--ignore=false should work, see https://github.com/TypeStrong/ts-node#configuration-options). I won't be changing it by default, which is what I already said. Unless you have some magic solution that detects just node modules of your package vs node modules installed, I don't see a better solution.

[gaspard]

I am really sorry. I don't know how I missed this --ignore flag. I never intended for this to be the default.

[blakeembrey]

No problem 😄 Let me know if there's any way to improve the docs to avoid issues in the future.

[qqilihq]

@gaspard

How does your final --ignore configuration look like? I also have a mono repo setup and find that simply specifying --ignore=false does not work for me, as ts-node would try to compile various files from the vendor's node_module directory which leads to errors.

On the other hand, I've not found a good regex to specify a robust ignore pattern:

• (?<!\/packages)\/node_modules\/ does not work due to JavaScript's lack of look behind
• \/project_name\/node_modules\/ works, but it hardcodes the root folder name of the repo and would fail if some developer renames the directory
• nesting everything in another subdirectory looks most robust to me, but is extremely ugly considering the fact that the mono repo structure already requires two directory levels.

Any suggestions?

PS: @blakeembrey: Please fix the typo in the ticket's title ("node_moules") -- it's impossible to find when searching for 'node_modules'.

[gaspard]

@qqilihq I decided to postpone my update of the repository until I have more time for this. If you manage to have something working, please share it here :-) there are not that many examples out there...

[qqilihq]

@gaspard Thank you for your reply. I went for the option 2 now and spent half a day refactoring. Still fighting with lots minor issues though. If I come to a good solution, I'll be glad to share it!

[romainPrignon]

Since ignore wants a pattern. We can use a regex to whitelist package.

Here is an example for my_package =>
--ignore='node_modules\/(?!my_package)'

[rosskevin]

Thanks @romainPrignon. It seems that the same type of thing is necessary when dealing with esnext tsc-produced libraries as well. We were getting SyntaxError: Unexpected identifier on our esnext or es2015 modules, but the following solved it.

I whitelisted our npm org and things started working:

NODE_ENV=production ts-node \
    -r tsconfig-paths/register \
    --files \
    --project tsconfig.run.json \
    --ignore 'node_modules\/(?!@alienfast)' \
    ./src/index.ts 

[marcj]

This doesn't work for me. I used this command:

NODE_PRESERVE_SYMLINKS=1 node_modules/.bin/ts-node \
    --ignore 'node_modules\/(?!core-utils)' \
    ./src/main.ts

and get this error:

/Users/marc/bude/deepkit/packages/server/node_modules/ts-node/src/index.ts:375
        throw new TypeError(
              ^
TypeError: Unable to require `.d.ts` file.
This is usually the result of a faulty configuration or import. Make sure there is a `.js`, `.json` or another executable extension and loader (attached before `ts-node`) available alongside `iterator.ts`.

My tsconfig.json looks pretty boring:

{
  "compileOnSave": false,
  "compilerOptions": {
    "baseUrl": "./",
    "outDir": "./dist/out-tsc",
    "declaration": false,
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "target": "es2017",
    "typeRoots": [
      "node_modules/@types"
    ],
    "lib": [
      "es2017",
      "dom"
    ]
  }
}

Anyone an idea how to get this TS-only dependency working?

[marcj]

Oh, I found it was related to cluster module of nodejs.

For anyone having the same issue, I fixed it using:

if (cluster.isMaster) {
    const PATH_TO_NODE = __dirname + '/../node_modules/.bin/ts-node';
    cluster.setupMaster({
        execArgv: [PATH_TO_NODE, '--ignore', 'node_modules\/(?!core-utils)'],
    } as cluster.ClusterSettings);
} else {
    //in fork
}

[hanvyj]

I agree typescript shoudn't be published. I'm having issues importing ES2015 modules though.

We need to use import and export provided by ES2015 to get tree-shaking, to reduce bundle size. Compiling down to CommonJS looses this very important feature.

I can't compile my typescript because it imports code that supports tree-shaking...

[blakeembrey]

@hanvyj I'm not quite sure I follow or understand why it's an issue with ts-node, care to elaborate?

[hanvyj]

It's not! Turns out you can do what I want with the ignore configuration. Ignore me.

[trusktr]

In case it helps anyone, here's how to do it using the API (instead of CLI) with regex(es):

require('ts-node').register({
  typeCheck: false,
  transpileOnly: true,
  files: true,

  // HERE, you can use RegExp literals here.
  ignore: [
    /node_modules\/(?!@scoped\/package)/,
    /node_modules\/(?!unscoped-package)/
  ],

  compilerOptions: require('./tsconfig.json').compilerOptions,
})

[trusktr]

Node.js v13 adds new complications. For anyone stumbling here, this is what you may face if you're using Node.js v13+, but I have not found a solution yet.

It seems the ignore trick no longer works if both using module: 'commonjs' and having dependencies that are published as ES Modules.

So although my ignore options is set correctly like I had it before Node 13 while I have module set to commonjs, I get an error like the following error when Node tries to import an ES Module from the CommonJS module output of ts-node:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js
require() of ES modules is not supported.
require() of /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js from /Users/trusktr/src/infamous+infamous/src/core/Observable.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/trusktr/src/infamous+infamous/node_modules/lowclass/package.json.

Now you may think "just change module to esnext". Well, it ain't that easy!

Here's the problem: the entry point that loads ts-node is still a CommonJS module, so as soon as ts-node supplies ES Module output, then you'll get an error when you import your first .ts file, like:

import * as fs from 'fs';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1055:16)
    at Module._compile (internal/modules/cjs/loader.js:1103:27)
    at Module.m._compile (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:536:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1159:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:539:12)
    at Module.load (internal/modules/cjs/loader.js:988:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1028:19)
    at require (internal/modules/cjs/helpers.js:72:18)

Next, you might think "well, now just convert the .js file that loads ts-node into an ES Module instead of CommonJS and give it the .mjs extension or use --input-type=module, and now import your .ts file instead of requireing it". For example:

import tsNode from 'ts-node'

tsNode.register({
    typeCheck: false,
    transpileOnly: true,
    files: true,
    project: './tsconfig.json',
    ignore: false,

    compilerOptions: {
        baseUrl: './',
        module: 'esnext', // out with the old, in with the new!
    },
})

// require('./readem.ts') // instead of this,
import './readem.ts'   // now try this.

Unfortunately, now this won't work, because .ts extensions are not something that ES Modules understand by default, so now errors like the following happen:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/trusktr/src/infamous+infamous/readem.ts imported from /Users/trusktr/src/infamous+infamous/readem.mjs

TLDR: ts-node is not yet in good shape to deal with the new ES Modules (unless I missed something).

[xmsz]

Node.js v13 adds new complications. For anyone stumbling here, this is what you may face if you're using Node.js v13+, but I have not found a solution yet.

It seems the ignore trick no longer works if both using module: 'commonjs' and having dependencies that are published as ES Modules.

So although my ignore options is set correctly like I had it before Node 13 while I have module set to commonjs, I get an error like the following error when Node tries to import an ES Module from the CommonJS module output of ts-node:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js
require() of ES modules is not supported.
require() of /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js from /Users/trusktr/src/infamous+infamous/src/core/Observable.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/trusktr/src/infamous+infamous/node_modules/lowclass/package.json.

Now you may think "just change module to esnext". Well, it ain't that easy!

Here's the problem: the entry point that loads ts-node is still a CommonJS module, so as soon as ts-node supplies ES Module output, then you'll get an error when you import your first .ts file, like:

import * as fs from 'fs';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1055:16)
    at Module._compile (internal/modules/cjs/loader.js:1103:27)
    at Module.m._compile (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:536:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1159:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:539:12)
    at Module.load (internal/modules/cjs/loader.js:988:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1028:19)
    at require (internal/modules/cjs/helpers.js:72:18)

Next, you might think "well, now just convert the .js file that loads ts-node into an ES Module instead of CommonJS and give it the .mjs extension or use --input-type=module, and now import your .ts file instead of requireing it". For example:

import tsNode from 'ts-node'

tsNode.register({
    typeCheck: false,
    transpileOnly: true,
    files: true,
    project: './tsconfig.json',
    ignore: false,

    compilerOptions: {
        baseUrl: './',
        module: 'esnext', // out with the old, in with the new!
    },
})

// require('./readem.ts') // instead of this,
import './readem.ts'   // now try this.

Unfortunately, now this won't work, because .ts extensions are not something that ES Modules understand by default, so now errors like the following happen:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/trusktr/src/infamous+infamous/readem.ts imported from /Users/trusktr/src/infamous+infamous/readem.mjs

TLDR: ts-node is not yet in good shape to deal with the new ES Modules (unless I missed something).

Same situation

[Bnaya]

I wrote i gist of how it cloud work together (ts + esm)
https://gist.github.com/Bnaya/0d544f4fba966af37da079ee7e8ec08d
Using experimental loaders hooks api. its not a complete impl by no-mean.
Also the api is considered none-stable

[renehamburger]

@Bnaya, the initial question did not relate to an ESM module (see #1007 for that), but to the direct import of a TypeScript file from a node_module. @romainPrignon's suggestion to change the default ignore pattern from node_modules to node_modules/(?!module1|module2|...) does the trick.

@cspotcode, would it be worth adding this to the documentation? I'd think it's quite a common scenario to want skip the compilation of one's own node module and import the TypeScript files directly.

[nicky1038]

Thank @blakeembrey for the explanation. I'll leave one more TL;DR here.

In most cases it really makes sense to only publish compiled Javascript files along with Typescript type definitions. Their compilation/generation should be properly handled by the project itself and not its dependents. From the opposite side, the dependent projects should just use ready Javascript files and know nothing about their generation.

If it's really needed, Typescript files inside node_modules folder still can be executed. For this one needs to use ts-node with --ignore="another-ignore-pattern" option that overrides the default ignore pattern (which is node_modules).

To accept all files you can pass a pattern that doesn't match any string, e. g. --ignore="(?!.*)".

It is also possible to run tools which use ts-node internally for parsing Typescript files that import code from node_modules. For example, it can be Webpack with Typescript configuration file. In order to do so, install cross-env and prepend cross-env TS_NODE_IGNORE="(?!.*)" to npm script command.

[cspotcode]

ts-node options can also be specified in your tsconfig.json file:

{
  "ts-node": {
    // ts-node options go here, such as "ignore"
  },
  "compilerOptions": { // this is a normal tsconfig file

See also, the official tsconfig JSON schema which includes our options: https://json.schemastore.org/tsconfig

I can certainly review a PR that adds or updates the relevant documentation to our README.

[benallfree]

To allow monorepo packages to be transpiled, something like this works:

// tsconfig.json - transpile all packages in the @monorepo namespace
{
   "ts-node": {
      "ignore": [ "/node_modules/(?!@monorepo)" ]
   }
}

[donmccurdy]

I believe I've hit this issue with a project using Node.js v14. Basically, I'm trying to use a dependency with ES Modules and "type": "module" in its package.json, and getting the error below:

ts-node script.ts

// script.ts
import { BoxBufferGeometry, BufferGeometry, Group, Matrix4, Mesh, Vector3 } from 'three';
import { Geometry } from 'three/examples/jsm/deprecated/Geometry';

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: $PROJECTS/three.js/examples/jsm/deprecated/Geometry.js
require() of ES modules is not supported.
require() of $PROJECTS/three.js/examples/jsm/deprecated/Geometry.js from $PROJECTS/three-to-cannon/test/index.test.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename Geometry.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from $PROJECTS/three.js/examples/jsm/package.json.

It seems to be the same issue described here (or have I got that wrong?) so I'm a bit surprised to see it marked as a "won't fix". Are ES Modules simply not supported, and there's no interest in supporting them? Support for ES Modules in Node.js itself has been improving in recent versions, and I'd be glad to see ts-node take advantage of that. :)

[lastmjs]

I'm also running into similar issues. I have a dependency that distributes itself compiled to ES2015 I believe, with ES modules. I'm having a difficult time messing with all of the possible Node.js and ts-node settings to try and get it to work. Fortunately for me, I control the dependency so I might just go add a commonjs compilation to it.

Until this point, ts-node has generally worked wonderfully for me. The dream for me would be for ts-node to simply handle any files or dependencies that you point at it, be they JavaScript, TypeScript, CommonJS, or ES Modules. IMO ts-node should simply take an initial file as input and just make it work

[fider-apptimia]

Same here... Any solution?

[cspotcode]

Use #1007