Specifying patch size in Node crashes

[thekevinscott]

Using the Div2k 3x model from @upscalerjs/esrgan-legacy, a Node implementation crashes if provided a patch size smaller than the image (128px).

Error

/Users/thekevinscott/www/UpscalerJS/examples/nodejs/node_modules/@tensorflow/tfjs-node/dist/nodejs_kernel_backend.js:214
        var outputMetadata = this.binding.executeOp(name, opAttrs, this.getInputTensorIds(inputs), 1);
                                          ^

Error: Invalid TF_Status: 3
Message: ConcatOp : Dimensions of inputs should match: shape[0] = [1,228,0,3] vs. shape[1] = [1,192,192,3]

Code

const express = require("express");
const path = require('path');
const fs = require('fs');
const Upscaler = require('upscaler/node');
const tf = require('@tensorflow/tfjs-node');
const model = require('@upscalerjs/esrgan-legacy/div2k/3x').default;
console.log(model);

const app = express();

const upscaler = new Upscaler({
  model,
});

app.get("/", async (req, res) => {
  const upscaledImage = await getUpscaledImage();
  res.set('Content-Type', 'image/png');
  res.write(upscaledImage, 'binary');
  res.end(null, 'binary');
});

app.listen(8080);

const getUpscaledImage = async () => {
  const file = fs.readFileSync(path.resolve(__dirname, './flower.png'));
  const image = tf.node.decodeImage(file, 3);
  const tensor = await upscaler.upscale(image, {
    output: 'tensor',
    patchSize: 64,
    padding: 6
  });
  image.dispose();
  const upscaledTensor = await tf.node.encodePng(tensor);
  tensor.dispose();
  return upscaledTensor;
}

[goldenjayr]

I have the same problem.. is there a fix for this???

[thekevinscott]

@goldenjayr not yet, still investigating as to why this happens.

[thekevinscott]

The issue appears to be related to this: tensorflow/tfjs#6125

Specifically, the behavior of concatenating tensors differs in TFJS between the browser and Node.

I am attempting to refactor the code to avoid the TFJS bug.

[thekevinscott]

This should be fixed and released as 1.0.0-beta.6. (Also ensure you're on 1.0.0-beta.6 for the model you're consuming.)

If you're still seeing issues @goldenjayr please feel free to re-open.