[Universal Sentence Encoder] Not working in NodeJS (Failed to fetch)

[cfpwastaken]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow.js): Yes. But reproduceable with 3 lines of code
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 22.10
• TensorFlow.js installed from (npm or script link): npm
• TensorFlow.js version (use command below): TFJS: ^4.2.0 | USE: ^1.3.3
• NodeJS version: 19.7.0

Describe the current behavior
When running the code below, it fails with this error:

Platform node has already been set. Overwriting the platform with node.
2023-03-31 11:58:55.987966: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
node:internal/deps/undici/undici:11413
    Error.captureStackTrace(err, this);
          ^

TypeError: fetch failed
    at Object.fetch (node:internal/deps/undici/undici:11413:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 5) {
  cause: ConnectTimeoutError: Connect Timeout Error
      at onConnectTimeout (node:internal/deps/undici/undici:8380:28)
      at node:internal/deps/undici/undici:8338:50
      at Immediate._onImmediate (node:internal/deps/undici/undici:8369:13)
      at process.processImmediate (node:internal/timers:475:21) {
    code: 'UND_ERR_CONNECT_TIMEOUT'
  }
}

Node.js v19.7.0

Describe the expected behavior
The USE should load just fine, just like in the Browser.

Standalone code to reproduce the issue

import * as tf from "@tensorflow/tfjs";
import * as use from "@tensorflow-models/universal-sentence-encoder";
const encoder = await use.load();

Other info / logs
See above

[pyu10055]

@cfpwastaken you can try to include 'node-fetch' package in your project.

[cfpwastaken]

simply installing the package doesnt change anything, changing the global fetch object to it results in a different error:

node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:2978
    return env().platform.isTypedArray(a);
                          ^

TypeError: env(...).platform.isTypedArray is not a function
    at isTypedArray (node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:2978:27)
    at inferShape (node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:5428:9)
    at Module.tensor2d (node_modules/@tensorflow/tfjs/node_modules/@tensorflow/tfjs-core/dist/tf-core.node.js:18580:25)
    at file:///main.js:27:23
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v19.7.0

[gaikwadrahul8]

Hi, @cfpwastaken

Apologize for the delayed response and I was able to replicate the same issue without any error. It seems like there is some issue with latest @tensorflow/tfjs version ^4.2.0 and node.js version and I see currently you're using Node.js v19.7.0 which comes with latest features so Could you please go with recommended Node.js v18.15.0 ? you're getting this error message TypeError: env(...).platform.isTypedArray is not a function for that we have already submitted one PR #7489 and it got merged, you can check this similar issue #7273 where users issue got resolved after downgrading @tensorflow/tfjs version and I have added screenshot below for your reference with Node.js v18.15.0 where code is working as expected.

Could you please try with Node.js v18.15.0 by creating new project with below mentioned dependancies inpackage.jsonfile and let us know whether is it resolving your issue or not ? if issue still persists please let us know ? Thank you!

Here is details of package.json file which includes all dependancies:

{
  "dependencies": {
    "@tensorflow-models/universal-sentence-encoder": "^1.3.3",
    "@tensorflow/tfjs": "^4.2.0",
    "@tensorflow/tfjs-node": "^4.2.0",
    "node-dns": "^0.1.0",
    "node-fetch": "^3.3.1"
  },
    "type": "module"
}

[cfpwastaken]

Yep. that seems to fix it. However i need to set the global fetch again, which you did not do in your example in order to get it to work:

import _fetch from "node-fetch";
import * as tf from "@tensorflow/tfjs";
import * as use from "@tensorflow-models/universal-sentence-encoder";
global.fetch = _fetch;
await use.load();
console.log("Works!");

That really shouldn't be needed since node comes with fetch preinstalled, why is it acting differently?

The USE should also get support for the latest node version, since that will become LTS sometime.

[cfpwastaken]

Now I am running into another weird issue though.
Simply importing the USE seems to just break tensorflow

import * as tf from "@tensorflow/tfjs";
tf.tensor2d([1, 2, 3, 4], [2, 2]).print();
console.log("Works!");
// Works

import * as tf from "@tensorflow/tfjs";
import * as use from "@tensorflow-models/universal-sentence-encoder";
tf.tensor2d([1, 2, 3, 4], [2, 2]).print();
console.log("Works!");
// Doesnt work

Its that TypeError: env(...).platform.isTypedArray is not a function error

[gaikwadrahul8]

@cfpwastaken

I tried the below code with @tensorflow/[email protected] and it's working as expected and It seems like there is some issue with latest @tensorflow/[email protected]. Thank you!

import * as tf from "@tensorflow/tfjs";
import * as use from "@tensorflow-models/universal-sentence-encoder";
tf.tensor2d([1, 2, 3, 4], [2, 2]).print();
console.log("Works!");

Here is output log:

gaikwadrahul-macbookpro:Node-JS-Issues-set-up gaikwadrahul$ node index.js 
Platform node has already been set. Overwriting the platform with node.

============================
Hi, looks like you are running TensorFlow.js in Node.js. To speed things up dramatically, install our node backend, visit https://github.com/tensorflow/tfjs-node for more details. 
============================
Tensor
    [[1, 2],
     [3, 4]]
Works!
gaikwadrahul-macbookpro:Node-JS-Issues-set-up gaikwadrahul$ 

[mattsoulanille]

@gaikwadrahul8's issue should be fixed in the next release. Sorry for the issues this has caused. For details, see #7489

The original issue with node fetch still exists. Downgrading node below 18 should fix it, but we're looking into a more permanent fix for newer versions.

[cfpwastaken]

great to hear!

[gaikwadrahul8]

Hi, @cfpwastaken

We have published new release @tensorflow/[email protected] 6 days ago and this issue has been taken care and I also replicated from my end with @tensorflow/[email protected] and it's working as expected, For your reference, I have added screenshot below so please try it from your end and let us know whether issue got resolved or not ?

Could you please confirm if this issue is resolved for you ? Please feel free to close the issue if it is resolved ?

If issue still persists please let us know ? Thank you!

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you.

[cfpwastaken]

Sorry for the late response, it is working as expected now!

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No