Middleware: return a different file name/type

[altano]

Body

Summary

Allow middleware to return a response for a different file than the current request. Currently middleware can only return a response for the requested document, e.g. pages/home.astro will always be written to home/index.html.

Background & Motivation

Having this would enable me to make a much better opengraph image integration than what currently exists. Once my integration is installed, an opengraph-image.astro component would be served (or built, in SSG) as a PNG image instead of HTML.

More detail: the middleware powering said integration would be a trivial gluing together of existing pieces: html response => VNODE (using @natemoo-re's satori-html) => svg (using Satori) => png (using sharp) => Response. Configuration removed it's just a few lines of code. The only piece missing is that once the PNG Response is generated, the Astro middleware currently writes it to opengraph-image/index.html instead of a PNG file.

Goals

• Allow Astro middleware to better implement the existing web platform by handling Response headers like Content-Type and Content-Disposition more like how a browser handles them. Specifically:
  • Astro middleware can return a Response with a Content-Type header. This will get passed-through in SSR mode.
  • Astro middleware can return a Content-Disposition header that specifies a different filename. If this is specified in the Response then the file written will reflect the specified name instead of the default. This should work in SSR and SSG mode.

Example

if this is returned by the middleware when requesting opengraph-image.astro:

return new Response(body, {
  headers: {
    "Content-Type": "image/png", // Would be ignored during an SSG build
    "Content-Disposition": `inline; filename="opengraph-image.png"`,
  },
});

	BEFORE	AFTER
SSR	Response served with Content-Type: image/png	no change
SSR	Response served with opengraph-image.png filename	no change
SSG	Response served with Content-Type: text/html	Response served with Content-Type: image/png
SSG	Response written to opengraph-image/index.html	Response written to opengraph-image.png

EDIT: SSR already works as described. So the only thing left to do is make SSG write to the Content-Disposition filename.

[jlarmstrongiv]

That was one of the workarounds that didn’t work when trying to make an i18n integration with defaultLocale for astro https://github.com/jlarmstrongiv/astro-i18n-aut/blob/main/CONTRIBUTING.md#the-how

It could make implementing a defaultLocale easier. Although, when fetching content from other pages from middleware, would those other pages be passed through the same middleware or be returned directly?

[altano]

This proposal only covers the very simple change of tweaking the mime type (ssr) / filename (ssg) of the existing route/page. We’re still returning the requested page, just changing its output type (eg from a component’s html to an image).

I have ideas on what it would look like to actually return a different page with middleware. Maybe it would be as simple as allowing a redirect to a different route and restarting the request pipeline for some other content (re-processing that other content through all middleware again). Redirects in ssg right now just dump a meta tag redirect into the html, which is a cute feature but less useful.

[altano]

I've published an alpha version of my html-to-image middleware generator for Astro.

Here is what it looks like:
https://codesandbox.io/p/sandbox/friendly-feather-shgxxz?file=%2Fsrc%2Fpages%2Fimage.png.astro%3A1%2C1

I think this is really awesome, but it only works in SSR: it won't work in SSG until this issue is resolved.

As such, I'm going to leave my plugin in Alpha with a warning at the top until this is fixed.

@natemoo-re / @matthewp please let me know if there's anything I can do to make it more likely for this to get done. Thanks!

[altano]

The demo has been upgrade to use astro v3

[altano]

I've published an alpha version of my opengraph-image middleware generator for Astro.

Here is what it looks like:
https://codesandbox.io/p/sandbox/astro-opengraph-image-test-demo-hwhxr3?file=/src/pages/index.astro

Same deal as the other middleware: it only works in SSR: it won't work in SSG until this issue is resolved.

[altano]

The demo has been upgrade to use astro v3

[lilnasy]

I'm not sure about automatically inferring the file extension from headers.
Another solution is for Astro to give special meaning to certain symbols.

const response = new Response(body, {
  headers: {
    "Content-Type": "image/png", // Would be ignored during an SSG build
    "Content-Disposition": `inline; filename="opengraph-image.png`,
  },
})

// Would be ignored by the live server render
response[Symbol.for("astro.staticOutput.fileName")] = "opengraph-image.png"

// alternatively...
response[Symbol.for("astro.staticOutput.extension")] = "png"

return response

The symbols would have no effect for SSR as suggested by the naming.

[altano]

It's not "inference" in so much as it is just simply implementing support for a standard HTTP header that has a very precise implication to the response. There's no magic here.

Why duplicate the information between a standard HTTP header and a custom, astro-specific header? What benefit would that have?

[lilnasy]

You're right, it's not inference. I guess what I feel uncertain about is parsing one specific directive of one specific header and giving it some meaning in the build process.

It poses a few questions. How much should we parse - name, filename, filename*? What meaning do we give form-data, inline, attachment? There can be multiple Content-Disposition headers, do we allow a single response to create multiple files? Could it be unsafe?

[altano]

That's a very valid concern. The problem with having an astro.staticOutput.fileName option is that you're potentially still using Content-Type/Disposition for SSR, so now you're configuring the same thing in two places hoping they do the same thing.

The way I hope to use this is in an integration that adds a middleware, so to support both SSG and SSR I'd be adding both the Content-Disposition and the astro.staticOutput.fileName header (since I'd want my integration to work in both SSG and SSR). As an integration author that would not be awesome.

Either way though, I might take a crack at implementing this, since it hasn't captured the imagination of anyone on the core team. If I went ahead and implemented it and submitted a PR, would you as a maintainer rather see the astro.staticOutput.fileName option for SSG, or should I proceed with using Content-Disposition? Happy to have you make the call.

[altano]

Hey everyone.

I've created a codesandbox that shows off this integration with Astro v4.

The integration now:

• uses the built-in Astro image processor (which was possible with help from @Princesseuh in Discord and to whomever made sharp the default, since squoosh can't convert svg files to png)
• uses Astro's middleware integration API, so there is no more manual step required to create a middleware
• is configurable when you add the integration in the astro.config.mjs file (this was REALLY hard and required use of Vite virtual modules and some crazy code... shoutout to @hippotastic for help with that)

As with before, the feature requested in this discussion is still in the way of making this integration with in SSG mode. But I'm of the opinion that Astro v4 makes this integration even cooler and I humbly re-request help making it a reality.

Thanks!

[ematipico]

Allow Astro middleware to better implement the existing web platform by handling Response headers like Content-Type and Content-Disposition more like how a browser handles them.

I think this is an important piece of information that is missing from your proposal. Can you tell us how browsers use these headers? Especially the Content-Disposition

[altano]

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition

[Princesseuh]

I ran into this usecase yesterday (well, only because I was using a middleware to work around not being able to render Markdoc with our renderer inside an API endpoint, but still), would like this feature

[altano]

Glad to hear I’m not the only one that wants it :)

I decided to take a stab at implementing it myself. How hard could it be? :p

[ddamato]

Just closed my issue on this that was similar. Would really love to see this as a feature.

[altano]

I put up a branch that gets 90% of the way there: https://github.com/withastro/astro/compare/main...altano:astro:image-middleware?expand=1

The main missing piece is that my implementation doesn't update the manifest when using the middleware rewriting (specifically the options.manifest.assets array) and I couldn't figure out a good way to patch the manifest. Arsh was going to help me in Discord (https://discord.com/channels/830184174198718474/1219122560508366889/1219131376637313034) but they seem to have exited the Astro community (at least on Discord). I assume not messing up the manifest, even when rewriting urls via middleware, is important.

I can put up a more official PR with broken unit tests and ask for help again, but I'm not sure when I'll have time to do that. If someone wants to help me through the home stretch let me know.

[Fryuni]

I don't quite like the breaking symmetry between SSR and SSG that this causes.

On SSR, you'd be accessing one URL, like https://yoursite.com/foo and that URL would return a Content-Disposition header informing your browser that it should consider the file being downloaded as having the name bar.png. Accessing https://yoursite.com/bar.png would return a 404.

On SSG, you'd be writing a bar.png file, so accessing https://yoursite.com/foo would be the 404 and https://yoursite.com/bar.png would return the statically generated file.

Content-Disposition tells the browser that the filename should not come from the URL; if we modify the URL internally, we are not adopting a web standard; we are breaking it.

[altano]

You're correct about that asymmetry existing, but I don't see it as a problem. Asymmetry already exists in many Astro features because the goal isn't to make them behave identically, but rather to make each mode do the best possible thing in the spirit of the feature (e.g. Astro redirects).

Your example sounds like an image generator middleware. The behavior you describe, although not the same between SSG and SSR, sounds exactly like how you'd want an image generator to behave in each mode. I wouldn't want symmetry in that example.

[Fryuni]

I certainly would not want an Astro behavior as fundamental as the file routes to suddenly change just because I switched to server mode, which could have happened for entirely unrelated reasons.

Changing internal routes depending on the mode is fine. Changing an explicitly set project route would be very unexpected.

What I'd expect is that either both would generate just /foo, leaving the headers for wherever the code is deployed to replicate (as it is today) or outright break the build if such a header is returned for a pre-render.

This would, for example, prevent themes from providing og-image generation since the path of image would not be predictable from the code or the route, it would depend on how the user is deploying it.

Asymmetry already exists in many Astro features because the goal isn't to make them behave identically, but rather to make each mode do the best possible thing in the spirit of the feature (e.g. Astro redirects).

The idea is for them to behave the same, even if they are not implemented the same. Some features only exist in one mode or another, but those that exist in both aim to result in the same behavior. Your example with Astro Redirects results in different implementations, one using the Location header and another using a <meta> tag, but both reach the same behavior, causing the browser to load a different URL automatically. The different implementations are to hide the difference between the modes; it is quite the opposite of this case, where the implementation makes switching between modes a breaking change.

[altano]

This could be used to introduce asymmetry but it can also be used to fix asymmetry that is impossible to fix today. For example:

1. Let's say you have a middleware that intercepts requests to opengraph-image.png.astro and returns an image buffer. It returns a response with the headers Content-Type: image/png and Content-Disposition: inline; filename="opengraph-image.png"
2. Right now,
   • you could only request this file in SSR by requesting the page example.com/opengraph-image.png and you would get back an image. It would render correctly.
   • in SSG. when published, you could request this with either example.com/opengraph-image.png or example.com/opengraph-image.png/index.html and you would get back garbage text/html.
3. After this change,
   • you could only request this file in SSR by requesting the page example.com/opengraph-image.png and you would get back an image. It would render correctly.
   • in SSG. when published, you could only request this file by requesting the page example.com/opengraph-image.png and you would get back an image. It would render correctly.

In this case, routing is more symmetrical between SSR/SSG after this change than it was before.

[bnjmnrsh]

I have run into this problem as well, however this conversation appears to have stalled over (important) implantation details. The end goal however of a middle ware or template (such as .astro) being able return a different header type opens a lot of possibilities, which appears to be the desired end goal.

#881 expresses this goal succinctly, but has been closed in favour of this thread.

Image generators are but one possible result, but an interesting one which would benefit from this sort of API.

[altano]

Hey Benjamin, I can give an update:

I completed this most of the way. You can see that in this diff (note that this needs to be rebased and is quite old at this point).

There's one bug that I don't know how to fix: the manifest assets don't reflect the middleware rewrite. plugin-manifest runs during build:post, which runs immediately after viteBuild but BEFORE staticBuild.

staticBuild is when we handle middleware responses, so that's the earliest I can know the real file output path (if the middleware rewrites the file output path). So the manifest has to be generated after staticBuild.

In discord I asked:

Is anyone familiar with the manifest and build:done hooks? Can I move runPostBuildHooks to AFTER staticBuild?

Arsh was going to help me investigate but I don't think he's affiliated with Astro anymore. You can find more details in the thread on Discord

I can rebase my changes and push a unit test that demonstrates the wrong manifest problem if someone was willing to help me get this over the finish line.

[altano]

Interestingly a lot of the hook APIs have changed since I wrote this code many months ago. If I have time I'll take another look and see if this problem is either no longer relevant now, or maybe easier to fix. But if anyone is willing to help, still let me know.

[altano]

I've put up a PR to implement this idea: withastro/astro#13024