cameraWithTensors Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor' #5972

sarons · 2021-12-30T04:10:31Z

I'm trying to run TensorCamera with React Native. Follwing the example code
but I get an error that says the ImageTensors are 'Tensor' not Tensor or TensorLike

import React from 'react'
import { Camera } from 'expo-camera'
import { cameraWithTensors } from '@tensorflow/tfjs-react-native'
import * as tf from '@tensorflow/tfjs'
import '@tensorflow/tfjs-react-native'

const TensorCamera = cameraWithTensors(Camera)

function CameraView() {
  handleCameraStream(images, updatePreview, gl) {
      const loop = async () => {
        const nextImageTensor = images.next().value;
  
        nextImageTensor.toFloat();
        // throws [Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor']
  
        nextImageTensor.expandDims(0);
        // throws [Error: Argument 'x' passed to 'expandDims' must be a Tensor or TensorLike, but got 'Tensor']
  
        // Solved both of it using tf.func but my model is giving a similar error now
        model.predict( tf.expandDims( tf.cast(nextImageTensor, 'float32'), 0) );
        // throws [Error: Argument 'x' passed to 'stridedSlice' must be a Tensor or TensorLike, but got 'Tensor']
      }
      loop();
    }
   
    return (
       <View>
         <TensorCamera
          // Standard Camera props
          style={styles.camera}
          type={Camera.Constants.Type.back}
          // Tensor related props
          cameraTextureHeight={textureDims.height}
          cameraTextureWidth={textureDims.width}
          resizeHeight={640}
          resizeWidth={640}
          resizeDepth={3}
          onReady={handleCameraStream}
          autorender={true}
         />
       </View>
     )
   }

I'm tried front & back camera, same issue with both. Tried printing out the image tensor and it looks alright

console.log(tf.tensor4d([[
          [[1, 3], [2, 8]],
          [[3, 9], [4, 2]]
      ]]))
// {"dataId": {"id": 247}, "dtype": "float32", "id": 253, "isDisposedInternal": false, "kept": false, "rankType": "4", "shape": [1, 2, 2, 2], "size": 8, "strides": [8, 4, 2]}

console.log( tf.expandDims( tf.cast(nextImageTensor, 'float32'), 0))
// {"dataId": {"id": 246}, "dtype": "float32", "id": 255, "isDisposedInternal": false, "kept": false, "rankType": "4", "scopeId": 14, "shape": [1, 640, 640, 3], "size": 1228800, "strides": [1228800, 1920, 3]}

Am I missing something? Seems like a simple usecase for cameraWithTensors. Please help

The text was updated successfully, but these errors were encountered:

rthadur · 2021-12-30T21:33:53Z

The link above refers to previous version(0.2) , can you try with latest version https://js.tensorflow.org/api_react_native/0.8.0/#cameraWithTensors and let us know. Thank you

sarons · 2021-12-31T00:55:31Z

Hey @rthadur, thanks for replying back.
I'm using the latest versions

    "@tensorflow/tfjs": "^3.12.0",
    "@tensorflow/tfjs-react-native": "^0.8.0",
    "expo": "^44.0.1",
    "expo-barcode-scanner": "~11.2.0",
    "expo-camera": "~12.1.0",
    "expo-face-detector": "~11.1.1",
    "expo-gl": "~11.1.1",
    "expo-gl-cpp": "~11.1.0",
    "react": "17.0.1",
    "react-native": "0.64.1",

And the code for 0.8.0 looks the same, except 0.2 has these camera configs

     cameraTextureHeight={textureDims.height}
     cameraTextureWidth={textureDims.width}

I removed those, still no go

sarons · 2022-01-03T02:03:34Z

I was reading through the documentation to make sure I hadn't missed anything and I saw it talks about npm run web and the example is a webcam one. I'm trying to run it on android and iOS.

Does tfjs-react-native only run on react native web?

sarons · 2022-01-03T05:55:23Z

I found this working example for mobile. I'll try to downgrade all the packages to what this example uses, maybe that will work 🤞

sarons · 2022-01-06T02:08:53Z

It's hard to find installtion documentation for older versions of expo and components. Lot of setup seems to have changed. I tried just downgrading @tensorflow/tfjs to "3.9.0" but still the same issue.

Were you able to reproduce this issue?

sarons · 2022-01-10T16:22:32Z

I tried creating a repo to reproduce the issue https://github.com/sarons/TensorCameraBug
but the app crashes when the camera comes on.

The problem seems to be pecular to repos that don't have expo. Projects created from expo and running on expo go seems to mask the issue. I only added expo sdk because tensorflowjs needs expo camera.
This repo is made the same way. First with npx react-native init and then expo sdk added

Been blocked on this for 2 weeks now,

jinjingforever · 2022-01-10T17:47:10Z

Hi @sarons sorry for the delay.. Our team is fairly busy lately and haven't had a chance to look into this. I quickly tried the current pose detection demo with expo and it doesn't work anymore on my iPhone.. still trying to figure out why. Most likely related to expo-gl not compatible with the new iOS maybe? I will try take a closer look some time this week. Thanks for your patience.

sarons · 2022-01-10T20:48:45Z

Thanks @jinjingforever. I'm trying this on an android device btw, same issue on it.

jinjingforever · 2022-01-11T00:18:18Z

Hi, I took a closer look today and here are what I found:

tfjs-react-native doesn't support inlineRequires: true. A working metro.config looks like the following. This probably explains the weird error you were seeing, because some packages were not bundled correctly:

const {getDefaultConfig} = require('metro-config');
module.exports = (async () => {
  const defaultConfig = await getDefaultConfig();
  const {assetExts} = defaultConfig.resolver;
  return {
    resolver: {
      // Add bin to assetExts
      assetExts: [...assetExts, 'bin'],
    },
  };
})();

You probably already know this, but before using TensorCamera, you need to make sure tf has been initialized. See here as an example.
After these changes, your repo code actually is still crashing on my iPhone (13 pro max, iOS 15.1.1). The reason is caused by expo-gl. It crashes here. I am not quite sure which particular package causes this, but the following is a working combination. Probably your react-native version is too high? I am using 0.64.3 below, while you used 0.66.4. expo 44 only officially supports 0.64 I think. But again, not entirely sure here..

    "@tensorflow/tfjs": "3.12.0",
    "@tensorflow/tfjs-react-native": "0.8.0",
    "expo": "~44.0.2",
    "expo-camera": "^11.2.2",
    "expo-file-system": "^10.0.0",
    "expo-gl": "^11.1.1",
    "expo-gl-cpp": "^11.1.0",
    "expo-image-manipulator": "^6.0.0",
    "expo-screen-orientation": "~4.1.1",
    "expo-splash-screen": "~0.10.2",
    "expo-updates": "~0.5.4",
    "react": "17.0.1",
    "react-dom": "17.0.1",
    "react-native": "~0.64.3",
    "react-native-fs": "2.14.1",
    "react-native-gesture-handler": "~1.10.2",
    "react-native-reanimated": "~2.1.0",
    "react-native-screens": "~3.0.0",
    "react-native-svg": "^9.13.6",
    "react-native-unimodules": "~0.13.3"

Give these a try and let me know how it goes. Thanks!

sarons · 2022-01-11T06:49:06Z

Hey @jinjingforever, thanks for looking into it. My metro config is the same as yours. I just didn't have it in the repo branch. Doing tf.ready() as well, still no go with my current setup which is

    "@react-native-async-storage/async-storage": "^1.15.14",
    "@tensorflow/tfjs": "^3.12.0",
    "@tensorflow/tfjs-react-native": "^0.8.0",
    "expo": "^44.0.1",
    "expo-barcode-scanner": "~11.2.0",
    "expo-camera": "~12.1.0",
    "expo-face-detector": "~11.1.1",
    "expo-gl": "~11.1.1",
    "expo-gl-cpp": "~11.1.0",
    "react": "17.0.1",
    "react-native": "0.64.1",
    "react-native-fs": "^2.18.0",
    "react-native-svg": "^9.13.6"

I downgraded to your versions. Not sure how you got expo 44 working with expo-camera 11 on device. react-native-unimodules has been depreciated since expo 43 but the old camera version needs it. So when I try to compile android on device, I'm hit with

Execution failed for task ':expo-camera:compileDebugJavaWithJavac'
> Compilation failed; see the compiler error output for details.

node_modules/expo-camera/android/src/main/java/expo/modules/camera/CameraModule.java:307: error: incompatible types: org.unimodules.core.Promise cannot be converted to expo.modules.core.Promise
    permissionsManager.askForPermissionsWithPromise(promise, Manifest.permission.CAMERA);

maybe unimodules and new expo are incompatible on android. Any chance you have an android device handy?

I'm open to using old expo, if it only works there. Just don't have old expo documentation. The new one adds a bunch of stuff to podfile and gradle.

jinjingforever · 2022-01-11T18:20:06Z

Hi @sarons

I created a PR that uses expo 44 with the latest expo-camera (12.1.0) without react-native-unimodules, and it seems to work on both my iPhone and Pixel 2. However it is an expo app:) Please give it a try and see if it works.

Frankly I am not too familiar with what the differences are between an expo app and a react-native CLI app. Is there any way you could diff them?

sarons · 2022-01-11T19:12:45Z

Thanks @jinjingforever
Just pushed the npm downgrade and webconfig changes to the tensorcamera bug repo here. Same android compile error there. Try it out

I'll try it out the changes you made. Altough I had done an expo upgrade on your pose app a while back and it was working fine through the expo go fine.
Other than the fact that expo go runs it from it's own app, one difference I found was how the app is registered expo vs react-native cli

I have also never used expo, there maybe other differences too. Only dealing with it since tensorflow is locked in with expo-camera. Tried to do it the expo way, but registerRootComponent hardcodes app name it to'main' here and withExpoRoot isn't avaiable publicly.
Not to mention for android, I need to add extra packages of expo-face-detector & expo-barcode-scanner because of this

At this point, I'm even considering creating a new expo app and move all my existing code to it. Desperate times.. ¯_(ツ)_/¯

sarons · 2022-01-11T22:05:25Z

Hey @jinjingforever, I have got it working in the bug repo with the npm versions you used in your PR .

Still getting the same error on my main repo with the same versions, but atleast I have a working reference now. Thanks

jinjingforever · 2022-01-12T00:13:08Z

Thanks for the update @sarons. Sorry for this tedious process.. Hope you find the root cause soon.

Looooong · 2022-04-23T04:23:44Z

I have the same problem trying to cast or use a tensor as well.

const imageTensor = images.next().value as tf.Tensor3D
// tf.node.js: [Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor']
imageTensor.cast('float32')

However, I can do tf.cast(imageTensor, 'float32') without any error.

For some reason, this inside getGlobalTensorClass().prototype.cast is not an instance of Tensor but is an instance of getGlobalTensorClass() when using imageTensor.cast:

getGlobalTensorClass().prototype.cast = function (dtype) {
    this.throwIfDisposed();
    console.log(this instanceof Tensor); // false
    console.log(this instanceof getGlobalTensorClass()); // true
    return cast(this, dtype);
};

I think this is a reason why it doesn't pass the if (x instanceof Tensor) check inside convertToTensor function.

On the other hand, I also get this error when using image tensor with a model. However, the error is thrown from tf-converter.node.js instead of tf.node.js in the case of casting.

Looooong · 2022-04-23T04:45:01Z

Good news: I get it to work now!

Since Tensor from tf.node.js and Tensor from tf-converter.node.js are 2 different classes, my conclusion is that the error is thrown because Tensor instance from tf.node.js is passed to tf-converter.node.js's method and vice versa.

My solution is to replace all the instanceof Tensor check with instanceof getGlobalTensorClass() in both tf.node.js and tf-converter.node.js.

google-ml-butler · 2022-05-12T18:59:03Z

Closing as stale. Please @mention us if this needs more attention.

google-ml-butler · 2022-05-12T18:59:07Z

Are you satisfied with the resolution of your issue?
Yes
No

Looooong · 2022-05-13T03:34:14Z

@rthadur @jinjingforever I think this issue still needs attention. The root cause is the fact that there are 2 definitions of Tensor class in 2 different packages: tfjs and tfjs-converter. Perhaps, we should move the class definition to a package that both libraries use, tfjs-core, maybe?

mingmingshen · 2022-05-21T10:53:26Z

Hello, I upgraded Expo and React-Native to 45, but 0.68.2 also has the same error. How should I deal with it? It works normally on 0.64 and EXPO 44, thank you

mingmingshen · 2022-05-21T11:49:57Z

I passed the video frame data into the face-landmarks- Detection detector.
{"dataId": {"id": 1085}, "dtype": "int32", "id": 1069, "isDisposedInternal": false, "kept": false, "rankType": "3", "shape": [224, 224, 3], "size": 150528, "strides": [672, 3]}

The detector execution returns the following error
DetectorError [Error: Argument 'x' passed to 'conv2d' must be a Tensor or TensorLike, but got 'Tensor']

This content worked normally in the previous EXPO build. I wonder if there was an incompatibility between TFJS-React-Native and the new EXPO 45 SDK, EXPO-GL and Expo-GL-CPP, which led to an error. The following is the version of the core package I use now. There are no major problems such as flash back, but there are errors in the prediction process of the model network.

Older versions of the environment can execute:
"@mediapipe/face_detection": "^0.4.1646425229",
"@mediapipe/face_mesh": "^0.4.1633559619",
"@tensorflow-models/blazeface": "^0.0.7",
"@tensorflow-models/face-detection": "^1.0.0",
"@tensorflow-models/face-landmarks-detection": "^1.0.1",
"@tensorflow/tfjs": "3.15.0",
"@tensorflow/tfjs-react-native": "^0.8.0",
"react": "17.0.2",
"react-native": "0.64.3",
"expo": "44.0.6",
"expo-camera": "^12.1.2",
"expo-gl": "^11.1.2",
"expo-gl-cpp": "^11.1.1",

New version environment execution error:
"@mediapipe/face_detection": "^0.4.1646425229",
"@mediapipe/face_mesh": "^0.4.1633559619",
"@tensorflow-models/blazeface": "^0.0.7",
"@tensorflow-models/face-detection": "^1.0.0",
"@tensorflow-models/face-landmarks-detection": "^1.0.1",
"@tensorflow/tfjs": "3.15.0",
"@tensorflow/tfjs-react-native": "^0.8.0",
"expo": "^45.0.4",
"expo-asset": "^8.5.0",
"expo-camera": "^12.2.0",
"expo-gl": "^11.3.0",
"expo-gl-cpp": "^11.3.0",

Execute detector error section

const faces = await faceDetector
.estimateFaces(image, estimationConfig)
.catch(e => console.log('DetectorError', e));

I don't know much about TFJS. Thank you for your help. Thank you

mingmingshen · 2022-05-21T12:33:02Z

I turned off Hermes to fix the above problem, and it is not clear if there was any deviation in compiling with Hermes

:hermes_enabled => flags[:hermes_enabled],
// :hermes_enabled => true, ->Note the enabling of Hermes

However, the following error occurs when running the own model on TFJS, which is similar to this topic return problem, and I will try to solve it:

Error: Argument 'x' passed to 'cast' must be numeric tensor, but got string tensor

mingmingshen · 2022-05-21T13:52:19Z

I turned off Hermes to fix the above problem, and it is not clear if there was any deviation in compiling with Hermes

:hermes_enabled => flags[:hermes_enabled], // :hermes_enabled => true, ->Note the enabling of Hermes

However, the following error occurs when running the own model on TFJS, which is similar to this topic return problem, and I will try to solve it:

Error: Argument 'x' passed to 'cast' must be numeric tensor, but got string tensor

This problem has been dealt with. The main reason is related to the data I sent in, which was modified during the update and forgot to restore. It has nothing to do with TFJS

google-ml-butler · 2022-05-28T14:26:05Z

Closing as stale. Please @mention us if this needs more attention.

google-ml-butler · 2022-05-28T14:26:08Z

Are you satisfied with the resolution of your issue?
Yes
No

d-kls · 2022-11-12T17:44:18Z

I am having the exact same issue when calling toFloat() after first calling decodeJpeg from tfjs-react-native. Then resizing and reshaping from tfjs. how did you guys fix this? @sarons @jinjingforever

Biki463 · 2023-04-26T06:47:10Z

i am also getting similar error from last 2 month

Error: Argument 'x' passed to 'expandDims' must be a Tensor or TensorLike, but got 'Promise

please help @Looooong @jinjingforever @mingmingshen @Google-ML-Butler

RachidZianne · 2023-09-03T17:44:17Z

Still the same issue: *Unhandled promise rejection: Error: Argument 'x' passed to 'conv2d' must be a Tensor or TensorLike, but got 'Tensor']
Help!)

safeer-ahmed · 2023-10-01T18:34:07Z

Still the same:
Error: Argument 'x' passed to 'reshape' must be a Tensor or TensorLike, but got 'Tensor'

Following are the versions of the important packages of my environment:

    "@tensorflow/tfjs": "^4.11.0",
    "@tensorflow/tfjs-react-native": "^0.8.0",
    "expo": "^49.0.0",
    "expo-camera": "~13.4.4",
    "expo-constants": "~14.4.2",
    "expo-gl": "~13.0.1",
    "expo-image-manipulator": "~11.3.0",
    "react": "18.2.0",
    "react-native": "0.72.4",

ALexandreM75013 · 2023-10-03T22:53:06Z

Same issue here unfortunately

 "@tensorflow/tfjs": "^3.18.0",
    "@tensorflow/tfjs-react-native": "^0.8.0",
    "expo": "^49.0.3",
    "expo-camera": "~13.4.4",
    "expo-constants": "~14.4.2",
    "expo-gl": "~13.0.1",
    "react": "18.2.0",
    "react-native": "0.72.5",

I am using an old version because the new versions fails to call tf.scalar(), but the error is exactly the same. I cannot test without hermes because I have react-native modules which needs hermes.

Looooong · 2023-10-04T09:10:52Z

I used to use patch-package to fix this issue with the following patch files:

Alternatively, yarn patch can be used as well.

Basically, the patchs replace all instanceof Tensor with instanceof getGlobalTensorClass() in tfjs/dist/tf.node.js and tfjs-converter/dist/tf-converter.node.js

nixoschu · 2023-10-18T14:31:37Z

For me, I can confirm that using jsc instead of hermes solved the issue.
On iOS, disabled Hermes in my Podfile, cleared my Pods Folder and the Podfile.lock and did a new install after the change.

I think this should be addresses ASAP, at least update the documentation about it.

    "@react-native-async-storage/async-storage": "^1.19.3",
    "@tensorflow/tfjs": "^4.11.0",
    "@tensorflow/tfjs-automl": "^1.3.0",
    "@tensorflow/tfjs-react-native": "^0.8.0",
    "expo": "^49.0.0",
    "expo-camera": "~13.4.4",
    "expo-gl": "~13.0.1",
    "react": "18.2.0",
    "react-native": "0.72.5",
    "react-native-fs": "^2.20.0",

sarons added the type:bug Something isn't working label Dec 30, 2021

rthadur self-assigned this Dec 30, 2021

rthadur added comp:react-native stat:awaiting response labels Dec 30, 2021

rthadur assigned jinjingforever and unassigned rthadur Jan 6, 2022

rthadur removed the stat:awaiting response label Jan 6, 2022

rthadur added stat:awaiting response stalled labels May 5, 2022

google-ml-butler bot closed this as completed May 12, 2022

jinjingforever reopened this May 16, 2022

google-ml-butler bot closed this as completed May 28, 2022

rthadur mentioned this issue Jun 13, 2022

tfjs does not work with hermes engine #6526

Closed

rthadur mentioned this issue Nov 22, 2022

Error: Argument "x" passed to 'conv2d' must be a Tensor or TensorLike, but got 'tensor' #7056

Open

paradite mentioned this issue Sep 6, 2023

fix: tensor class import for hermes #7947

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cameraWithTensors Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor' #5972

cameraWithTensors Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor' #5972

sarons commented Dec 30, 2021

rthadur commented Dec 30, 2021

sarons commented Dec 31, 2021

sarons commented Jan 3, 2022

sarons commented Jan 3, 2022

sarons commented Jan 6, 2022

sarons commented Jan 10, 2022

jinjingforever commented Jan 10, 2022

sarons commented Jan 10, 2022

jinjingforever commented Jan 11, 2022

sarons commented Jan 11, 2022 •

edited

Loading

jinjingforever commented Jan 11, 2022

sarons commented Jan 11, 2022 •

edited

Loading

sarons commented Jan 11, 2022

jinjingforever commented Jan 12, 2022

Looooong commented Apr 23, 2022 •

edited

Loading

Looooong commented Apr 23, 2022 •

edited

Loading

google-ml-butler bot commented May 12, 2022

google-ml-butler bot commented May 12, 2022

Looooong commented May 13, 2022

mingmingshen commented May 21, 2022

mingmingshen commented May 21, 2022

mingmingshen commented May 21, 2022

mingmingshen commented May 21, 2022

google-ml-butler bot commented May 28, 2022

google-ml-butler bot commented May 28, 2022

d-kls commented Nov 12, 2022 •

edited

Loading

Biki463 commented Apr 26, 2023

RachidZianne commented Sep 3, 2023

safeer-ahmed commented Oct 1, 2023

ALexandreM75013 commented Oct 3, 2023

Looooong commented Oct 4, 2023

nixoschu commented Oct 18, 2023

cameraWithTensors Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor' #5972

cameraWithTensors Error: Argument 'x' passed to 'cast' must be a Tensor or TensorLike, but got 'Tensor' #5972

Comments

sarons commented Dec 30, 2021

rthadur commented Dec 30, 2021

sarons commented Dec 31, 2021

sarons commented Jan 3, 2022

sarons commented Jan 3, 2022

sarons commented Jan 6, 2022

sarons commented Jan 10, 2022

jinjingforever commented Jan 10, 2022

sarons commented Jan 10, 2022

jinjingforever commented Jan 11, 2022

sarons commented Jan 11, 2022 • edited Loading

jinjingforever commented Jan 11, 2022

sarons commented Jan 11, 2022 • edited Loading

sarons commented Jan 11, 2022

jinjingforever commented Jan 12, 2022

Looooong commented Apr 23, 2022 • edited Loading

Looooong commented Apr 23, 2022 • edited Loading

google-ml-butler bot commented May 12, 2022

google-ml-butler bot commented May 12, 2022

Looooong commented May 13, 2022

mingmingshen commented May 21, 2022

mingmingshen commented May 21, 2022

mingmingshen commented May 21, 2022

mingmingshen commented May 21, 2022

google-ml-butler bot commented May 28, 2022

google-ml-butler bot commented May 28, 2022

d-kls commented Nov 12, 2022 • edited Loading

Biki463 commented Apr 26, 2023

RachidZianne commented Sep 3, 2023

safeer-ahmed commented Oct 1, 2023

ALexandreM75013 commented Oct 3, 2023

Looooong commented Oct 4, 2023

nixoschu commented Oct 18, 2023

sarons commented Jan 11, 2022 •

edited

Loading

sarons commented Jan 11, 2022 •

edited

Loading

Looooong commented Apr 23, 2022 •

edited

Loading

Looooong commented Apr 23, 2022 •

edited

Loading

d-kls commented Nov 12, 2022 •

edited

Loading