✅ Map and Set in output

[t1nky]

Currently, it is possible to send Map and Set as an output, but JS cannot serialize them:

It might be helpful to restrict possible output types. Also, there are functions and BigInt.

[RobinTail]

Currently, it is possible to send Map and Set as an output

What exactly do you mean, @t1nky ?

[t1nky]

My idea was different, making a wrapper generic type and applying it to IOSchema. I was thinking it would be easy to write a type that would ensure ZodMap wouldn't be used, something like ExcludeZodMap<T>. But tried it myself I see that it is not that easy.

I think that iterating over the whole result object in custom ResultHandler in runtime just to verify and serialize Map and Set can be costly, especially given that we already have zod serialization. I would avoid it.

I ended up with the following, since I know my schema and I can avoid iteration:

export const mapOutputEndpoint = taggedEndpointsFactory.build({
  method: "get",
  input: z.object({}),
  output: z.object({
    // My idea was to show a type error here, to demonstrate that the output type is not serializable
    // items: z.map(z.string(), z.map(z.string(), z.string())),
    items: z.record(z.record(z.string())),
  }),
  handler: async () => {
    const result = new Map<string, Map<string, string>>([
      ["a", new Map<string, string>([["b", "c"]])],
    ]);

    // Without this preprocessing it would still work fine with items being
    // a z.map(z.string(), z.map(z.string(), z.string()))
    // but in the response the items object is empty because it's not serializable by default
    const items = Object.fromEntries(
      Array.from(result.entries(), ([key, value]) => [
        key,
        Object.fromEntries(value),
      ]),
    );

    return { items };
  },
});

But to make it a general thing, I don't see it being possible without iteration over output (recursive?).

That's why the idea was to make it type level restriction.

[RobinTail]

I think I'm getting closer to understanding the problem.
You mean that it's possible to assign z.map and z.set to a property of the output schema, so it can be returned from the endpoint handler and reach the ResultHandler, the default version of which does not do any special processing of such kind of data, so it's being incorrectly serialized by response.json().
Is that correct problem description, @t1nky ?

[RobinTail]

But to make it a general thing, I don't see it being possible without iteration over output (recursive?).

Well, iterating is a runtime operation, it does not prevent the issue during the development.
So it should rather be a type error, as your initial idea indicate:

My idea was to show a type error here, to demonstrate that the output type is not serializable

But making such type validation is a complicated and non-trivial problem having an unknown complexity as well.
At the moment I don't see an elegant solution yet, but maybe it will come later.

[RobinTail]

It could possibly be an ESLint rule, @t1nky :)

[RobinTail]

ESLint observes the code during the development as AST nodes.

So the rule could be:

• when there is a property access expression having identifiers like z.map or z.set or z.date (or other unserializable)
  • having nearest parent node named output or input
    • having nearest parent node being property access expression like .build
      • then invalidate that syntax

[RobinTail]

I found out that runtime iteration is already happening in some limited way upon creating the endpoints and middlewares.
There are some programmatic checks already in place that prevent issues that could not be handled on the type level.
So maybe I could also check for unserializable types at the same time, @t1nky

[RobinTail]

I found an interesting thing.
Express uses app.set() vars in res.json() for JSON.stringify() replacer:
https://github.com/expressjs/express/blob/8cb53ea5c3329032a1db47be019b717d8350fb0e/lib/response.js#L232-L234

That approach is the recommended way for serializing Map/Set:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map#objects_vs._maps

[RobinTail]

Yeah! I'm going to make them serializable, @t1nky :)
In #2115

[t1nky]

Great find @RobinTail

[RobinTail]

There is tricky part regarding Map, @t1nky .
The keys of Map can have any type:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map#objects_vs._maps

And that makes 'em not fully compatible to object-based serialization.
For best compatibility I'm considering serializing Map as Array of key-value tuples.
On the consumer side, if that is considered safe, such array can be transformed into object using Object.fromEntries() method (ES2019 feature).
I hope it's OK for you.

[RobinTail]

After a series of serious considerations involving several buckets of caffeine and nicotine, I came to conclusion that this is not a good approach, because it does not really address the original problem. Instead, the system should notify about potentially unserializable entities used. The tricky part here is that it CAN technically be made serializable by the end-user using monkey-patch technique

Map.prototype.toJSON = () => "I am the map";

However, that would never be depictable properly by Documentation and Integration. Therefore, I decided that it should be a message anyway, but warn message, not runtime error. This decision also takes into account that the actual response can be not a JSON, but maybe a text. So the warning will appear for a confirmed JSON operating endpoint.

Addressing that in #2116

[RobinTail]

It will appear this way, @t1nky

2024-10-25T13:39:49.763Z warn: The final positive response schema of the endpoint contains an unsupported JSON payload type. {
  path: '/v1/user/list',
  method: 'get',
  reason: AssertionError [ERR_ASSERTION]: z.nan()

🚀 Version 20.15.0