Idea: Handling 404 Errors

[trieloff]

Here's an idea on handling 404 errors in a Helix-ish way.

Assumptions:

• customers want nice (customizable) 404 error pages
• the HTML pipeline knows that there is a 404 error after the fetch step

Solution:

1. after the fetch step, add another error step, which will do the following
2. output HTML that has following content: <esi:include src="404.html">
3. finish the rendering pipeline
4. Fastly will then either include a static 404.html from the static repo, giving you full customizability, or go back to the html pipeline, which is this time retrieving 404.md from the content repository and renders it

Open Questions:

• how to make sure we don't end up in an endless loop if 404.md doesn't exist
• how to make sure the correct status code (404) gets set when serving the static 404.html

[ramboz]

Looking into this, I'm wondering how to test the <esi:include src="404.html"> locally. Is that possible?

I tried a simple default helix site with ./htdocs/404.html:

• adding <esi:include src="404.html"> in the html.htl just hangs the page at load time
• adding <esi:include src="404.html"> in the index.md just outputs the escaped tag in a <p>

I may be lacking some documentation on how to use the esi: instructions

[trieloff]

I'd look into the helix-simulator, it's using a node-esi module to implement ESI. You'd definitely put the ESI tags into the HTL, but I'm not sure why your request hangs.

[rofe]

Maybe it should be <esi:include src="/404.html">? Without the slash, it would try to resolve the request relative to the current resource.

[ramboz]

@rofe tried both. also tried self-closing tags (<esi:include/>) vs. explicit closing (<esi:include></esi:include>). I'll continue my investigations

[rofe]

@trieloff can ESIs even be static resources? I thought they have to correspond to an md file and get rendered by the pipeline.

[tripodsan]

can ESIs even be static resources? I thought they have to correspond to an md file and get rendered by the pipeline.

they can be anything. they are just refetched, going through the entire request processing again.
so having a loop might be a problem....

i.e.

request /foo.html -> request /404.html -> request /404.html ....

[ramboz]

So far, I have the following use cases covered:

1. Default demo project with no error md file => standard browser error page is shown with corresponding status code
2. Default demo project with 404.md or 500.md => custom error page is shown with corresponding status code

What I have not covered yet is serving a static 400.html through the <esi:include…/>. Not sure how to achieve that as a haven't been able to render a plain static HTML file locally

[tripodsan]

Not sure how to achieve that as a haven't been able to render a plain static HTML file locally

it works, when you put the 404.html in /htdocs/404.html (only in fastly, but not it in the simulator, though see adobe/helix-simulator#216 ).
I think this is ok, since producing a 404.html is more of a developer task than content author.

[tripodsan]

btw, this is actually a problem :-) relying on a 404 to fallback to the static resolution.

• if the new 404 handler returns now an ESI+200, fastly will always include the error document with a 200, which is wrong.
• if the new 404 handler returns the ESI+404, fastly will try to load the /404.html from the static, hence never load a /htdocs/mypage.html
• if the 404.html should be provided from /htdocs, we have the endlessloop described above.

Can we do the 404 handing in fastly https://docs.fastly.com/guides/basic-configuration/creating-error-pages-with-custom-responses ?

[trieloff]

We could return a 404 + X-Error: handled response from the pipeline, in which case there would not be a fallback (to be implemented). Then you have following scenarios:

• extension != html: fallback to static
• extension==html and handled by pipeline: serve nice HTML error message
• extension==html and not handled by pipeline: fallback to static

[tripodsan]

no, the problem is, that if we want to serve static html from htdocs (which is a requirement for the 404.html and also what @davidnuescheler wants), then we cannot use the 404 error handler in pipeline at all. because any 404 returned from the pipeline, VCL needs to try to fetch the static content.

[trieloff]

1. request /footer.html
2. HTML pipeline fails in fetch
3. HTML pipeline catches error and changes footer.md to 404.md and tries to re-catch (different from original idea, which delegated to an ESI here)
4. fetch fails again, 404 returned to Fastly
5. Fastly re-tries with static /htdocs/footer.html and delivers

So for Helix Pages, /footer.html and 404.md are mutually exclusive.

[tripodsan]

3. HTML pipeline catches error and changes footer.md to 404.md and tries to re-catch (different from original idea, which delegated to an ESI here)

I don't think this is really cool, since the error page most certainly needs a different script/template than the original requested page. using the 404.md, you would need to add the 404 special-use-case handling to every pre.js.

IMO, the proper way would be to handle the 404 processing in fastly at the end of the processing chain.

the simplest idea would be to locally render a /404.html and upload to fastly during publish as [synthetic response(https://docs.fastly.com/guides/basic-configuration/creating-error-pages-with-custom-responses).

another idea: we could implement the onerror="fail" support (adobe/helix-publish#83) and use esi.

[trieloff]

We already use custom error pages in Fastly, so having hlx publish bundle up the HTML for /[4-5][0-9x][0-9x]\.html/ wouldn't be too hard.

[ramboz]

Not sure how to move forward with this. Should I just go ahead and close so we focus on implementing this outside of the pipeline?

[trieloff]

Yes, this is superseded by adobe/helix-publish#96