Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

full-page templates #253

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import {readFile, stat} from "node:fs/promises";
import {basename, dirname, extname, join} from "node:path";
import {visitFiles} from "./files.js";
import {parseMarkdown} from "./markdown.js";
import {type Template, page} from "./page.js";

export interface Page {
name: string;
Expand All @@ -21,6 +22,7 @@ export interface TableOfContents {

export interface Config {
title?: string;
template: Template;
pages: (Page | Section)[]; // TODO rename to sidebar?
pager: boolean; // defaults to true
toc: TableOfContents;
Expand Down Expand Up @@ -62,7 +64,8 @@ export async function normalizeConfig(spec: any, root: string): Promise<Config>
pages = Array.from(pages, normalizePageOrSection);
pager = Boolean(pager);
toc = normalizeToc(toc);
return {title, pages, pager, toc};
const {template = page} = spec;
return {title, template, pages, pager, toc};
}

function normalizePageOrSection(spec: any): Page | Section {
Expand Down
150 changes: 150 additions & 0 deletions src/page.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
import {parseHTML} from "linkedom";
Copy link
Contributor Author

@Fil Fil Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of churn is due to moving the default rendering function and its utilities into a separate page.ts. I was (pleasantly) surprised that parseHtml is only needed for the toc.

import {type Config, type Page, type Section, mergeToc} from "./config.js";
import {type Html, html} from "./html.js";
import {findLink} from "./pager.js";
import {relativeUrl} from "./url.js";

export type Template = (
elements: {
/**
* The page’s path.
*/
path: string;
/**
* User-provided front-matter.
*/
data?: any;
/**
* A title element. Typically inserted in the head element.
*/
title?: string;
/**
* A series of link[rel=modulepreload] elements, to insert in the head.
*/
preloads: string;
/**
* The main module, a script[type=module] tag containing all the client-side
* JavaScript code for the page. Typically inserted at the top of the main
* element.
*/
module: string;
/**
* The main HTML contents. Typically inserted in a main or an article
* element.
*/
main: string;
/**
* A base tag for the project root (only made available for the 404 page,
* since it might be served from any sub-directory path).
*/
base?: string;
/**
* Relative path to the root of the project; "./" for top-level pages, "../"
* for pages in a subdirectory, etc.
*/
root?: string;
},
Copy link
Member

@mbostock mbostock Nov 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we instead passed a declarative representation of the page state — rather than partially rendered HTML fragments — so that the template has complete flexibility in turning the declarative representation of the page into HTML (or anything else, e.g., TeX)? Though that requires us to design an intermediate representation as part of the public API. Worth the effort? 🤔

I also think we should encourage people to use the Html helper here because any template implementation will be prone to escaping bugs without it. (The Html helper is not Hypertext Literal (htl) — htl runs in the browser.)

I suspect we will also need to have an API for the CLI, declare a main or exports in package.json, and probably run tsc to convert TypeScript to JavaScript so that people can import helpers to implement their template function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My approach here is very bare-bones, "stdout"-like. It treats observable markdown as a function that returns HTML fragments that you can just paste together and send as one big string to the browser for a severely unstyled but functional experience:

return title + preloads + module + main

All these fragments are made of safe HTML (unless I made a mistake). Users can assemble them in their own "page templates" (or placeholders) with zero difficulty. Pareto is happy.

A (crazy, or "pure") idea would be to return all of this as a single html fragment, and let the default template collect the elements with a querySelector (it already does it to build the TOC, so it wouldn't be much more costly). But probably too crazy.

Some elements are not HTML (path, root and the front-matter data); maybe we should put them in a separate argument?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mbostock could you explain your declarative representation of the page state idea?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes… I need more time to think about this one and then come up with a more actionable proposal.

options: Omit<Config, "template">
) => string;

export const page: Template = ({path, data, preloads, module, main, title, base, root}, options) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like we want to eventually add other parameters for overriding other HTML fragments - like footer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, no problem!

const {pages, title: projectTitle} = options;
const toc = mergeToc(data?.toc, options.toc);
const headers = toc.show ? findHeaders(main) : [];
return `<!DOCTYPE html>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
${title}${base}<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css2?family=Source+Serif+Pro:ital,wght@0,400;0,600;0,700;1,400;1,600;1,700&display=swap">
<link rel="stylesheet" type="text/css" href="${root}_observablehq/style.css">${
data?.template ? `\n<link rel="stylesheet" type="text/css" href="${root}_observablehq/${data.template}.css">` : ""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data in this case comes directly from the front matter but what if the user wants to apply the same styling across all pages? maybe I am missing something...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree we'll want to allow customization of the global CSS (adding a stylesheet, or modifying the base stylesheet), but it's not in this PR's scope.

}${preloads}
${module}${pages.length > 0 ? renderSidebar(projectTitle, pages, path) : ""}
${headers.length > 0 ? renderToc(headers, toc.label) : ""}<div id="observablehq-center">
<main id="observablehq-main" class="observablehq">
${main}</main>
<footer id="observablehq-footer">${renderPager(path, options)}
<div>© ${new Date().getUTCFullYear()} Observable, Inc.</div>
</footer>
</div>
`;
};

function renderSidebar(title = "Home", pages: (Page | Section)[], path: string): Html {
return html`\n<input id="observablehq-sidebar-toggle" type="checkbox">
<label id="observablehq-sidebar-backdrop" for="observablehq-sidebar-toggle"></label>
<nav id="observablehq-sidebar">
<ol>
<li class="observablehq-link${path === "/index" ? " observablehq-link-active" : ""}"><a href="${relativeUrl(
path,
"/"
)}">${title}</a></li>
</ol>
<ol>${pages.map((p, i) =>
"pages" in p
? html`${i > 0 && "path" in pages[i - 1] ? html`</ol>` : ""}
<details${p.open ? " open" : ""}>
<summary>${p.name}</summary>
<ol>${p.pages.map((p) => renderListItem(p, path))}
</ol>
</details>`
: "path" in p
? html`${i > 0 && "pages" in pages[i - 1] ? html`\n </ol>\n <ol>` : ""}${renderListItem(p, path)}`
: ""
)}
</ol>
</nav>
<script>{
const toggle = document.querySelector("#observablehq-sidebar-toggle");
const initialState = localStorage.getItem("observablehq-sidebar");
if (initialState) toggle.checked = initialState === "true";
else toggle.indeterminate = true;
}</script>`;
}

interface Header {
label: string;
href: string;
}

function findHeaders(html: string): Header[] {
return Array.from(parseHTML(html).document.querySelectorAll("h2"))
.map((node) => ({label: node.textContent, href: node.firstElementChild?.getAttribute("href")}))
.filter((d): d is Header => !!d.label && !!d.href);
}

function renderToc(headers: Header[], label = "Contents"): Html {
return html`<aside id="observablehq-toc">
<nav>
<div>${label}</div>
<ol>${headers.map(
({label, href}) => html`\n<li class="observablehq-secondary-link"><a href="${href}">${label}</a></li>`
)}
</ol>
</nav>
</aside>
`;
}

function renderListItem(p: Page, path: string): Html {
return html`\n <li class="observablehq-link${
p.path === path ? " observablehq-link-active" : ""
}"><a href="${relativeUrl(path, prettyPath(p.path))}">${p.name}</a></li>`;
}

function prettyPath(path: string): string {
return path.replace(/\/index$/, "/") || "/";
}

function renderPager(path: string, options: Pick<Config, "pages" | "pager" | "title">): Html | "" {
const link = options.pager && findLink(path, options);
return link
? html`\n<nav>${link.prev ? renderRel(path, link.prev, "prev") : ""}${
link.next ? renderRel(path, link.next, "next") : ""
}</nav>`
: "";
}

function renderRel(path: string, page: Page, rel: "prev" | "next"): Html {
return html`<a rel="${rel}" href="${relativeUrl(path, prettyPath(page.path))}"><span>${page.name}</span></a>`;
}
152 changes: 34 additions & 118 deletions src/render.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
import {parseHTML} from "linkedom";
import {type Config, type Page, type Section, mergeToc} from "./config.js";
import {type Config} from "./config.js";
import {type Html, html} from "./html.js";
import {type ImportResolver, createImportResolver} from "./javascript/imports.js";
import {type FileReference, type ImportReference} from "./javascript.js";
import {type CellPiece, type ParseResult, parseMarkdown} from "./markdown.js";
import {type PageLink, findLink} from "./pager.js";
import {relativeUrl} from "./url.js";

export interface Render {
Expand Down Expand Up @@ -49,107 +47,40 @@ type RenderInternalOptions =
| {preview?: false} // serverless
| {preview: true}; // preview

function render(parseResult: ParseResult, options: RenderOptions & RenderInternalOptions): string {
const {root, path, pages, title, preview, resolver} = options;
const toc = mergeToc(parseResult.data?.toc, options.toc);
const headers = toc.show ? findHeaders(parseResult) : [];
return String(html`<!DOCTYPE html>
<meta charset="utf-8">${path === "/404" ? html`\n<base href="/">` : ""}
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
${
parseResult.title || title
? html`<title>${[parseResult.title, parseResult.title === title ? null : title]
.filter((title): title is string => !!title)
.join(" | ")}</title>\n`
: ""
}<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css2?family=Source+Serif+Pro:ital,wght@0,400;0,600;0,700;1,400;1,600;1,700&display=swap">
<link rel="stylesheet" type="text/css" href="${relativeUrl(path, "/_observablehq/style.css")}">${renderImportPreloads(
parseResult,
path,
createImportResolver(root, "_import")
)}
<script type="module">${html.unsafe(`

import {${preview ? "open, " : ""}define} from ${JSON.stringify(relativeUrl(path, "/_observablehq/client.js"))};

${
preview ? `open({hash: ${JSON.stringify(parseResult.hash)}, eval: (body) => (0, eval)(body)});\n` : ""
}${parseResult.cells.map(resolver).map(renderDefineCell).join("")}`)}
</script>
${pages.length > 0 ? renderSidebar(title, pages, path) : ""}
${headers.length > 0 ? renderToc(headers, toc.label) : ""}<div id="observablehq-center">
<main id="observablehq-main" class="observablehq">
${html.unsafe(parseResult.html)}</main>
${renderFooter(path, options)}
</div>
`);
}

function renderSidebar(title = "Home", pages: (Page | Section)[], path: string): Html {
return html`<input id="observablehq-sidebar-toggle" type="checkbox">
<label id="observablehq-sidebar-backdrop" for="observablehq-sidebar-toggle"></label>
<nav id="observablehq-sidebar">
<ol>
<li class="observablehq-link${path === "/index" ? " observablehq-link-active" : ""}"><a href="${relativeUrl(
function render(
parseResult: ParseResult,
{root, path, preview, resolver, template, ...options}: RenderOptions & RenderInternalOptions
): string {
return template(
cinxmo marked this conversation as resolved.
Show resolved Hide resolved
{
path,
"/"
)}">${title}</a></li>
</ol>
<ol>${pages.map((p, i) =>
"pages" in p
? html`${i > 0 && "path" in pages[i - 1] ? html`</ol>` : ""}
<details${p.open ? " open" : ""}>
<summary>${p.name}</summary>
<ol>${p.pages.map((p) => renderListItem(p, path))}
</ol>
</details>`
: "path" in p
? html`${i > 0 && "pages" in pages[i - 1] ? html`\n </ol>\n <ol>` : ""}${renderListItem(p, path)}`
: ""
)}
</ol>
</nav>
<script>{
const toggle = document.querySelector("#observablehq-sidebar-toggle");
const initialState = localStorage.getItem("observablehq-sidebar");
if (initialState) toggle.checked = initialState === "true";
else toggle.indeterminate = true;
}</script>`;
}

interface Header {
label: string;
href: string;
}

function findHeaders(parseResult: ParseResult): Header[] {
return Array.from(parseHTML(parseResult.html).document.querySelectorAll("h2"))
.map((node) => ({label: node.textContent, href: node.firstElementChild?.getAttribute("href")}))
.filter((d): d is Header => !!d.label && !!d.href);
}

function renderToc(headers: Header[], label = "Contents"): Html {
return html`<aside id="observablehq-toc">
<nav>
<div>${label}</div>
<ol>${headers.map(
({label, href}) => html`\n<li class="observablehq-secondary-link"><a href="${href}">${label}</a></li>`
)}
</ol>
</nav>
</aside>
`;
}

function renderListItem(p: Page, path: string): Html {
return html`\n <li class="observablehq-link${
p.path === path ? " observablehq-link-active" : ""
}"><a href="${relativeUrl(path, prettyPath(p.path))}">${p.name}</a></li>`;
}

function prettyPath(path: string): string {
return path.replace(/\/index$/, "/") || "/";
title:
parseResult.title || options.title
? String(
html`<title>${[parseResult.title, parseResult.title === options.title ? null : options.title]
.filter((title): title is string => !!title)
.join(" | ")}</title>\n`
)
: "",
main: parseResult.html,
root: relativeUrl(path, "/"),
base: path === "/404" ? '<base href="/">\n' : "",
data: parseResult.data,
preloads: String(renderImportPreloads(parseResult, path, createImportResolver(root, "_import"))),
module: String(
html`<script type="module">

${html.unsafe(
`import {${preview ? "open, " : ""}define} from ${JSON.stringify(relativeUrl(path, "/_observablehq/client.js"))};\n${
preview ? `open({hash: ${JSON.stringify(parseResult.hash)}, eval: (body) => (0, eval)(body)});\n` : ""
}
${parseResult.cells.map(resolver).map(renderDefineCell).join("")}`
)}
</script>`
)
},
options
);
}

function renderImportPreloads(parseResult: ParseResult, path: string, resolver: ImportResolver): Html {
Expand All @@ -172,18 +103,3 @@ function renderImportPreloads(parseResult: ParseResult, path: string, resolver:
}
return html`${Array.from(preloads, (href) => html`\n<link rel="modulepreload" href="${href}">`)}`;
}

function renderFooter(path: string, options: Pick<Config, "pages" | "pager" | "title">): Html {
const link = options.pager ? findLink(path, options) : null;
return html`<footer id="observablehq-footer">${link ? renderPager(path, link) : ""}
<div>© ${new Date().getUTCFullYear()} Observable, Inc.</div>
</footer>`;
}

function renderPager(path: string, {prev, next}: PageLink): Html {
return html`\n<nav>${prev ? renderRel(path, prev, "prev") : ""}${next ? renderRel(path, next, "next") : ""}</nav>`;
}

function renderRel(path: string, page: Page, rel: "prev" | "next"): Html {
return html`<a rel="${rel}" href="${relativeUrl(path, prettyPath(page.path))}"><span>${page.name}</span></a>`;
}
3 changes: 3 additions & 0 deletions test/config-test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import assert from "node:assert";
import {normalizeConfig as config, mergeToc, readConfig} from "../src/config.js";
import {page} from "../src/page.js";

const root = "test/input/build/config";

Expand All @@ -11,6 +12,7 @@ describe("readConfig(root)", () => {
{path: "/one", name: "One<Two"},
{path: "/sub/two", name: "Two"}
],
template: page,
title: undefined,
toc: {label: "On this page", show: true},
pager: true
Expand All @@ -19,6 +21,7 @@ describe("readConfig(root)", () => {
it("returns the default config if no config file is found", async () => {
assert.deepStrictEqual(await readConfig("test/input/build/simple"), {
pages: [{name: "Build test case", path: "/simple"}],
template: page,
title: undefined,
toc: {label: "Contents", show: true},
pager: true
Expand Down
4 changes: 4 additions & 0 deletions test/input/build/template/.observablehq/config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export default {
pages: [{path: "/index", name: "Test page"}],
template: (elements, options) => JSON.stringify({elements, options}, null, 2)
};
5 changes: 5 additions & 0 deletions test/input/build/template/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Hello

```js
d3
```
Loading