Support ES Modules for Node 14 functions

[mbleigh]

Currently, it is not possible to use native ES Modules with Cloud Functions for Firebase in Node 14 due to how triggers are parsed from the code. Unfortunately, adding support is not trivial (or I would have done it when I added Node 14 support generally), but we'd like to do it in the future.

Please upvote this issue if it's important to you!

[taeold]

In addition to problems with how triggers are parsed in firebase-tools repo, the functions framework (used to load and run user-provided code in google cloud functions) doesn't seem to support ESM yet: GoogleCloudPlatform/functions-framework-nodejs#233.

[mbleigh]

@taeold good catch. Seems like even if we finished the work here we'd be blocked on that. Perhaps we should put a pin in this until work has been done in the functions framework.

[mesqueeb]

I just want to mention to all who come here to upvote:

Don't forget to click "subscribe" on the right side to get updates!!

[larssn]

Does this affect typescript projects?

We use this tsconfig for node 14: https://github.com/tsconfig/bases/blob/master/bases/node14.json

I'm guessing it doesn't.

[mbleigh]

@larssn correct, since TypeScript with default configuration compiles down into CommonJS, it is not affected -- you can continue to use ESM in TypeScript just like you can in Node < 14.

[spicemix]

Although even in a Typescript project, the emulators refuse to use p-queue@7 for instance, leaving me using an old non-ESM version. Thanks in advance!

[Spencer-Easton]

@mbleigh Will top level await work if it compiles from TS?

[taeold]

@Spencer-Easton Yes. As long the compiled JS code that's deployed to Firebase Functions doesn't contain top level await, things will work.

[jthegedus]

@taeold

@Spencer-Easton Yes. As long the compiled JS code that's deployed to Firebase Functions doesn't contain top level await, things will work.

Do the ESLint rules generated by firebase init functions capture top-level await? If not, can we configure this rule in the template?

[taeold]

@jthegedus Good idea. We will probably want to update the template for both JS and TS to reflect ES module support in GCF.

[DibyodyutiMondal]

So if we manually add in the latest version of functions framework as well as "type":"modules" into our package.json right now, will it work?

[taeold]

@DibyodyutiMondal Not quite. We'll need to make a new release to Firebase CLI that bring support for deploying Firebase Functions using ES modules.

I will mark this issue fixed once we make the release (should be done in next couple of days).

Here's a quick sample for those interested:

// index.js
import functions from "firebase-functions";

export const helloWorld = functions.https.onRequest((request, response) => {
  response.send("Hello from Firebase!");
});

// package.json
...
  "type": "module",
  "dependencies": {
    "@google-cloud/functions-framework": "^1.9.0",     // Explicitly include functions-framework v1.9.0
...

[DibyodyutiMondal]

Does this aforementioned release include support in the emulator as well?

[taeold]

@DibyodyutiMondal Yes. It should work as long as the local version of node used to run the emulator supports ES modules (think that's v13 and up).

[paulvanj]

@DibyodyutiMondal Not quite. We'll need to make a new release to Firebase CLI that bring support for deploying Firebase Functions using ES modules.

I will mark this issue fixed once we make the release (should be done in next couple of days).

Here's a quick sample for those interested:

// index.js
import functions from "firebase-functions";

export const helloWorld = functions.https.onRequest((request, response) => {
  response.send("Hello from Firebase!");
});

// package.json
...
  "type": "module",
  "dependencies": {
    "@google-cloud/functions-framework": "^1.9.0",     // Explicitly include functions-framework v1.9.0
...

Hi guys, any progress with this?

[taeold]

ES module is now supported with the latest firebase-cli version:

$ npm install -g firebase-tools # Get the latest firebase-cli
$ firebase deploy --only functions # Deploy as usual

Note that:

1. You must select nodejs14 runtime.
2. You must manually include latest version of @google-cloud/functions-framework dependency.

e.g.

// package.json
...
  "engines": {
    "node": "14"
  },
  "type": "module",
  "dependencies": {
    "@google-cloud/functions-framework": "^1.9.0",
    ...
  },

If you have any problems with deploying your function packaged as an ES module, please consider reporting a new github issue or filing a support ticket.

[DibyodyutiMondal]

I am getting the following error:

i  emulators: Starting emulators: auth, functions, firestore, hosting, storage
!  functions: The following emulators are not running, calls to these services from the Functions emulator will affect production: database, pubsub
+  functions: Using node@14 from host.
i  firestore: Firestore Emulator logging to firestore-debug.log
i  hosting: Serving hosting files from: dist/apps/public-client
+  hosting: Local server: http://localhost:5005
i  ui: Emulator UI logging to ui-debug.log
i  functions: Watching "D:\Github\ros-firebase\dist\libs\cloud-functions" for Cloud Functions...
!  functions: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'
!  Your function was killed because it raised an unhandled error.

Is there anything I can do from my side to solve this problem, or am I forced to use commonjs as long as I am developing on windows? (I do not like the idea of adding simport throughout the entire dependency chain of my cloud-functions code.)

[paulvanj]

I am getting the following error:

i  emulators: Starting emulators: auth, functions, firestore, hosting, storage
!  functions: The following emulators are not running, calls to these services from the Functions emulator will affect production: database, pubsub
+  functions: Using node@14 from host.
i  firestore: Firestore Emulator logging to firestore-debug.log
i  hosting: Serving hosting files from: dist/apps/public-client
+  hosting: Local server: http://localhost:5005
i  ui: Emulator UI logging to ui-debug.log
i  functions: Watching "D:\Github\ros-firebase\dist\libs\cloud-functions" for Cloud Functions...
!  functions: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'
!  Your function was killed because it raised an unhandled error.

Is there anything I can do from my side to solve this problem, or am I forced to use commonjs as long as I am developing on windows? (I do not like the idea of adding simport throughout the entire dependency chain of my cloud-functions code.)

getting the same issue. Appears we need to use the type 'module' for the emulator to start up the functions as using 'commonjs' leads to many other issues. advice would be greatly appreciated to resolve this issue mentioned by DibyodyutiMondal.

[taeold]

Thanks for filing the issue @paulvanj . Let's carry the discussion on that issue.

[carlpaten]

I'm trying to migrate from CommonJS to ES Modules and I'm encountering a difficulty in that the loader algorithm changed from CJS to ESM. In particular, you can't use import red from 'color/red' as shorthand for import red from 'color/red/index.js anymore.

This poses two problems for me:

1. I have a large code base in the old style, and it would be a moderate pain to migrate it to the new style
2. I'm using TypeScript, and using node12 (ESM with standard loader) modules means giving up on resolveJsonModule (importing JSON files as if they were TypeScript)

Node.js supports customizing the loader via the --experimental-specifier-resolution flag. If I could pass --experimental-specifier-resolution=node to node then I could upgrade painlessly to ES modules, otherwise I'm going to suffer.

I'm guessing I'm not the only one this problem.

[DibyodyutiMondal]

Since all firebase functions are google cloud functions, there is the possibility of using the gcloud SDK to add NODE_OPTIONS=--experimental-module-specifier=node to the environment variables via the cli. Alternatively, you can do it via the Google cloud console. But...
a. Will it work
b. What if I have many many functions?

If only firebase allowed us to add it using the firebase cli or environment options...

[taeold]

We've been experimenting with bringing env var support to CF3 using dotenv format.

We are still working out few kinks, but folks interested in the feature can try this:

$ firebase --open-sesame dotenv
$ echo 'NODE_OPTIONS=--experimental-module-specifier=node' > functions/.env
# Directory layout:
#   my-project/
#     firebase.json
#     functions/
#       package.json
#       index.js
#       .env

$ cat functions/.env
NODE_OPTIONS=--experimental-module-specifier=node

$ firebase deploy --only function
...
I Loaded environment variables from .env.
# Deploys firebase functions with following user-defined environment variables:
#   NODE_OPTIONS=--experimental-module-specifier=node

Then, all the functions deployed will have the specified environment variable.

IMPORTANT Dotenv feature will not preserve existing environment variables on a function. Nor does it allow for setting environment variable per-function.

Interested in hearing your feedback.

[DibyodyutiMondal]

@taeold

NODE_OPTIONS=--experimental-module-specifier=node

The above won't work. But the approach is spot on.
The flag you are looking for is
NODE_OPTIONS=--experimental-specifier-resolution=node
(source: https://nodejs.org/docs/latest-v16.x/api/esm.html#customizing-esm-specifier-resolution-algorithm)

As of now, this is the only way that typescript-compiled esm can work without the file extensions.

Also, in VS Code, depending on how the terminal is launched, the NODE_OPTIONS env variable may already be populated (by the vs code debugging bootloader). In such a scenario, dot-env will ignore the NODE_OPTIONS set in the .env file. This is just how dot-env works - it will never overwrite existing env variables.
In the end, I had to set the NODE_OPTIONS in the vs code workspace settings, so that when I launch the terminal from inside vs code, the node options env variable includes the flag we need; in addition to .env which will be deployed to the cloud.
I assume that different editors will have different ways to go about this.

It would be great if the documentation for cloud functions mentioned all of this information, so that people who are new to all 3 of javascript, typescript and firebase at the same time (like I was) don't trip over one thing or the other. Even experienced developers are stumbling as the community is slowly shifting from commonjs to esm.
The following page is where I believe is the best place to put it in
https://firebase.google.com/docs/functions/typescript

[wliumelb]

ES module is now supported with the latest firebase-cli version:

$ npm install -g firebase-tools # Get the latest firebase-cli
$ firebase deploy --only functions # Deploy as usual

Note that:

1. You must select nodejs14 runtime.
2. You must manually include latest version of @google-cloud/functions-framework dependency.

e.g.

// package.json
...
  "engines": {
    "node": "14"
  },
  "type": "module",
  "dependencies": {
    "@google-cloud/functions-framework": "^1.9.0",
    ...
  },

If you have any problems with deploying your function packaged as an ES module, please consider reporting a new github issue or filing a support ticket.

This seems to have stopped working.

Still got the error message
Build failed: (node:95) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.

even after setting "type" to "module".

[taeold]

@wliumelb Do you mind sharing steps to reproduce?

I have a simple function:

import * as functions from "firebase-functions";

export const helloWorld = functions.https.onRequest((req, resp) => {
  functions.logger.info("Hello logs!", {structuredData: true});
  resp.send("Hello from Firebase!");
});

With package.json:

{
  "name": "functions",
  "description": "Cloud Functions for Firebase",
  "scripts": {
    "lint": "eslint .",
    "serve": "firebase emulators:start --only functions",
    "shell": "firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "14"
  },
  "type": "module",
  "dependencies": {
    "firebase-admin": "^9.2.0",
    "firebase-functions": "^3.11.0"
  },
  "devDependencies": {
    "eslint": "^7.6.0",
    "eslint-config-google": "^0.14.0",
    "firebase-functions-test": "^0.2.0"
  },
  "private": true
}

And it deploys fine.

[wliumelb]

@taeold here is an exemple:

index.js

import { initializeApp } from 'firebase-admin/app';
import { region } from "firebase-functions";
initializeApp();
export const testFunction = region('australia-southeast1').https.onCall(async (data, context) => {
    return "OK";
});

package.json

{
  "name": "functions",
  "scripts": {
    "build": "tsc",
    "serve": "npm run build && firebase emulators:start --only functions",
    "shell": "npm run build && firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "16"
  },
  "main": "lib/index.js",
  "type": "module",
  "dependencies": {
    "firebase-admin": "^10.0.0",
    "firebase-functions": "^3.16.0"
  },
  "devDependencies": {
    "typescript": "^4.5.0",
    "firebase-functions-test": "^0.3.3"
  },
  "private": true
}

Deployment failure only happens when using node 16. Node 14 is working fine.

[AverageHelper]

Deployment failure only happens when using node 16. Node 14 is working fine.

Can confirm. I get cryptic build errors when the deploying machine is running Node 16, even if package.json specifies Node 14. Switching to Node 14 (via NVM or the like) fixed me right up.

[codyaweber]

I ran into this as well. functions-framework repo has the same issue opened.

[taeold]

Thanks for sharing more info everyone.

It sounds like the underlying issue is with nodejs/node#41189 and - unfortunately - the only known workaround is to use node v14 instead :(.

[maganap]

@taeold Fix to issue nodejs/node#41198 seems to be part of node v17.4 already
(https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V17.md)
but latest release of node v16 (v16.13.2) is from before that fix.
(https://github.com/nodejs/node/blob/master/doc/changelogs/CHANGELOG_V16.md)

It means it will be available (soon? ever?) in node 16, correct?
Does it take too long for a new release of node v16.x to be available in Google Cloud Functions?

Thanks for clarifying :)

[taeold]

@maganap You are right - the fix needs to be backported to node v16, and the scheduling of that work is depended on the node core contributors. I believe Google Cloud Functions team (which Firebase Functions is built on) is intending to release a temporary fix for the issue - you can track that work GoogleCloudPlatform/functions-framework-nodejs#407.

IIUC, once the patch is backported to a later version of node 16, it should be made available in GCF pretty soon. That's how we first got into this mess anyway.

[senghuotlay]

Any update on this case, as i am still experiencing the same issue. and i am using nodejs 14.
This is the command i use to run gcloud functions deploy graphql --entry-point handler --runtime nodejs14 --project=klotti-app-dev --region=australia-southeast1 --trigger-http --allow-unauthenticated --set-env-vars NODE_ENV=development along with some config "type": "module", "private": true, "engines": { "node": "14" }, and some babel config { "presets": ["@babel/preset-env"] }

[juni0r]

I've set up my functions project as suggested in this thread, but I'm getting an error when importing another file from the same project.

$ firebase --version
10.2.2

$ firebase emulators:start --only functions 

Error: Error occurred while parsing your function triggers.

Error [ERR_MODULE_NOT_FOUND]: Cannot find module '<MY_PROJECT_DIR>/functions/dist/init' imported from <MY_PROJECT_DIR>/functions/dist/index.js
    at finalizeResolution (internal/modules/esm/resolve.js:285:11)
    at moduleResolve (internal/modules/esm/resolve.js:708:10)
    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:819:11)
    at Loader.resolve (internal/modules/esm/loader.js:89:40)
   ...

I've tried every suggestion here, including using node 14, adding @google-cloud/functions-framework to my deps, adding a `.env' file, but to no avail. I'm completely at loss here.

Directory layout

<MY_PROJECT_DIR>
  firebase.json
  functions/
    src/
      index.ts
      init.ts

index.ts

import init from './init'

package.json

  "type": "module",
  "engines": {
    "node": "16"
  },
  ...
  "dependencies": {
    "firebase-admin": "^9.8.0",
    "firebase-functions": "^3.14.1"
  },

tsconfig.json

{
  "compilerOptions": {
    "outDir": "dist",
    "module": "es6",
    "target": "es2017",
    "moduleResolution": "node",
    "allowSyntheticDefaultImports": true,
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "sourceMap": true,
    "strict": true
  },
  "include": ["src"]
}

[taeold]

@juni0r Can you try replacing

import init from './init'

with

import init from './init.js'

See https://www.typescriptlang.org/docs/handbook/2/modules.html#es-module-syntax for docs.

[juni0r]

@taeold While that might work, it would introduce inconstencies in our codebase. You see, this package is part of a monorepo in which we use Typescript throughout and every other package uses imports without the .js extension. We'd also need to change certain imports from say '../api' to '../api/index.js' which would certainly add to the confusion.

Since this convention isn't enforced by linting rules and the project will build even with a mixed usage pattern, this is very prone to errors that might only surface during ci or deployment.

We have instead opted for targetting CommonJS for functions which required building some internal dependencies for CommonJS as well, but at least it works.

Gosh, this whole module mess by far the biggest pain with JS. So much time wasted, so much complexity added. 😄

[taeold]

@juni0r i agree it's painful and confusing to work with ESmodule in typescript! Unfortunately, I don't think we can help with changes in this repo..

[senghuotlay]

I think this link https://zenn.dev/fjsh/articles/82e1b938ef96f4 could help a ton if you're trying to write it in typescript. I somehow got it working following this link

[DibyodyutiMondal]

@taeold

I dove into the node_modules and changed the code at the locations in the images below. All it does is add the NODE_OPTIONS environment variable. The alternative is to append .js and /index.js to imports as necessary.

And it works like a charm with nothing more needed.

It seems that while parsing the triggers, the environment variables are not picked up, even if they are specified in a .env file.

If you think that this is a good approach, I'd be happy to work on a pull request that would add an option to do this (maybe via the previews feature?)

Speaking of preview features, I have not been able to deploy this on functions v2, but I think that's because of some other issue, not this.

(the location of the file I am editing is visible in the left sidebar)

[DibyodyutiMondal]

Just saw that someone has opened an issue #4361 in connection with what I just sent. I shall post the above images there and continue there instead

[DibyodyutiMondal]

Per my experience so far, if we want typescript-based functions code that ultimately targets ESM, there are so many things that a developer has to think about: import endings, typescript settings, typescript versions, firebase tools versions, and even the node runtime itself.

In my case, using esbuild to bundle my functions code resulted in a guarantee of sorts that no matter how my codebase, typescript config, runtime or environment variables change, the functions will always run as intended. As a developer it significantly reduced the attention and time I had to give while setting up functions for a new project.

However, it's not necessary to use a bundler,
Step 1 would be to simply use '.js' endings for your imports inside your typescript files.
If that is not possible because it causes inconsistencies, bugs or other problems in your project, then I recommend using esbuild to bundle the functions code prior to execution and deployment.
Esbuild also supports using watch mode, which I have verified is compatible with the local functions emulator. Works great, in fact.

Any bundler will do, but esbuild is pretty fast, so I use that.
Some esbuild settings work better depending on your project, for example, public packages from node_modules can be excluded from the bundle, but do bundle private packages. So do your homework.

It works on node 18 and 16 runtimes for cloud-functions as well. On the basis of all this, I recommend that a bundler should be included in the scaffolded code for typescript cloud functions, or at least be mentioned in the documentation.

[simondotm]

I agree with you @DibyodyutiMondal and have myself moved to using bundled functions now, since it neatly resolves a number of issues.

My setup now uses es modules and rollup with Node 16, and I decided to use one project per function, and make use of the firebase config codebase feature for functions. This way, my functions only contain the code they use, due to esm tree shaking and it is much easier to write/import local libraries that can be shared across functions.

Firebase do have documents for Using module bundlers with Firebase.

Due to the benefits of bundling, I don't believe I will return to using pure typescript packages for function deployment again.

[DibyodyutiMondal]

@simondotm
Thanks for pointing me to the documentation. I missed it.

[fuelkoy]

Got just this working with Esbuild. I will post the solution here for others to benefit as i tried to solve this for some time.

I have the same problem for a long time. But finally got a solution.

Build with Esbuild

(Follow instructions from Matt Pocock https://www.totaltypescript.com/build-a-node-app-with-typescript-and-esbuild)

1. Adding Our Config Files

1.1 "type": "module" in package.json
Next, add "type": "module" to the package.json file.

{
  // ...other properties
  "type": "module"
  // ...other properties
}

1.2 Dependencies
Next, let's install our dependencies:

pnpm add -D typescript esbuild @types/node npm-run-all

1.3 TypeScript Config
Add a tsconfig.json file at the root of the project with the following configuration:

{
  "compilerOptions": {
    /* Base Options: */
    "esModuleInterop": true,
    "skipLibCheck": true,
    "target": "es2022",
    "verbatimModuleSyntax": true,
    "allowJs": true,
    "resolveJsonModule": true,
    "moduleDetection": "force",
    /* Strictness */
    "strict": true,
    "noUncheckedIndexedAccess": true,
    /* If NOT transpiling with TypeScript: */
    "moduleResolution": "Bundler",
    "module": "ESNext",
    "noEmit": true,
    /* If your code doesn't run in the DOM: */
    "lib": ["es2022"]
  }
}

2. Adding Our Scripts

2.1 build script
Add a build script to package.json:

{
  // ...other properties
  "scripts": {
   "build": "esbuild src/index.ts --bundle --platform=node --outfile=lib/index.js --format=esm --external:./node_modules/*",
  }
  // ...other properties
}

2.2 dev script

{
  // ...other properties
  "scripts": {
    "dev:tsc": "tsc --watch --preserveWatchOutput",
    "dev:node": "firebase emulators:start --only functions",
	"dev:esbuild": "pnpm run build --watch",
    "dev": "run-p dev:*",
  }
  // ...other properties
}

Run pnpm dev

And finally you got your Firebase functions emulator running with Typescript, ESM and even wathc mode (watch is guide slow for me, takes 5-8s for functions to handle new code, but still it does it automatically🎉)