[BUG] incorrect behavior of RawBodyParams

[derevnjuk]

Information

• Version:: 6.x
• Packages: platform-express

In implementing HTTP signature verification (e.g. Stripe or GitHub), it might be useful to get the raw body. Currently, it is almost impossible to use RawBodyParams since it has limitations by content type and length.

RawBodyParams returns a string instead of an instance of Buffer

In docs stated that RawBodyParams returns the value from request.body as a Buffer (see https://api-docs.tsed.io/api/platform/platform-params/types/decorators/RawBodyParams.html).

But it returns a string since chunk is implicitly casting to string due to using a string concatenation in rawBodyMiddleware:

request.rawBody = "";
request.on("data", (chunk) => {
  request.rawBody += chunk;
});

Moreover, rawBodyMiddleware doesn't care about a charset at the same time.

Expecting data events to be synchronously

The middleware may not run well until the reading is finished. That means, rawBody may be undefined in some cases when consuming data takes more time than usual:

request.rawBody = "";
request.on("data", (chunk) => {
  request.rawBody += chunk;
});
next();

To fix that, you have to rewrite the middleware as follows:

request.rawBody = "";
request.on("data", (chunk) => {
  request.rawBody += chunk;
});
request.on("end", (chunk) => {
  next();
});

There is also another problem since the stream is switched in flowing mode by subscribing to the data event, it may lead to some race between different body parsers. For instance, you can use $afterInit to register express.json() middleware that consumes all data from the request preventing catching rawBody:

const parseBody = (req, res, next) => {
  const chunks = [];

  if (req._readableState.ended) {
    console.warn('cannot obtain raw body since stream is ended')
    return next()
  }

  req.on('data', data => {
    chunks.push(data);
  });

  req.on('end', () => {
    req.rawBody = Buffer.concat(chunks);
    next();
  });
};

app.post('/', express.json(), parseBody, (req, res) => {
  console.log(req.body);
  console.log(req.rawBody);

  res.send('Hello World!')
})

Since the readable stream is ended, the previous example will be stuck, and the current implementation will just set an empty string to req.rawBody.

Acceptance criteria

• User should be able to get rawBody independently of using other body parsers
• User should be able to override a middleware to parse a raw body
• RawBodyParams should return a Buffer instead of a string by default
• User should be able to configure RawBodyParams to obtain a string according to charset

[Romakita]

Hello @derevnjuk

perfect issue description ;)

User should be able to get rawBody independently of using other body parsers
is no really possible. When BodyParser consume the request data isn’t possible after to use the rawBodyMiddleware. And if the rawBodyMiddleware consume the req.data it the same thing.

I’m not sure how to fix that correctly. If you have a solution or an idea, let’s go ;)

See you
Romain

[Romakita]

Hello @derevnjuk
I worked on the bodyParser issue. Finally, The only correct way to manage that is to add bodyParser for each endpoint depending on the collected metadata.

it doesn’t fix issue if the server add a body parser during the $beforeRoutesInit. But if the body parser is removed, it works as expected.

There is no better option ^^.

The pr is almost ready. I need to update the documention.

see you
Romain

[Romakita]

🎉 This issue has been resolved in version 6.111.0 🎉

The release is available on:

• v6.111.0
• GitHub release

Your semantic-release bot 📦🚀

[Romakita]

🎉 This issue has been resolved in version 7.0.0-beta.4 🎉

The release is available on:

• v7.0.0-beta.4
• GitHub release

Your semantic-release bot 📦🚀