Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(page-data): add key to allPages #10625

Merged
merged 52 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from 46 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
9fcbe95
fix(page-data): add key to allPages
goulvenclech Mar 31, 2024
b6dd048
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 1, 2024
284ee9e
add fryuni's test
goulvenclech Apr 1, 2024
d42376c
replaced object.entries(allpages)
goulvenclech Apr 1, 2024
05f1ede
tmp: change pagesByComponents by pagesByKeys
goulvenclech Apr 1, 2024
db7bca5
fix pagesByKeys.get() in plugin-ssr & plugin-manifest
goulvenclech Apr 1, 2024
172623b
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 1, 2024
92f3099
remove logs
goulvenclech Apr 1, 2024
3021f42
remove useless generators
goulvenclech Apr 1, 2024
6853313
another useless generator
goulvenclech Apr 1, 2024
5db6dc6
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 1, 2024
1fd2690
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 9, 2024
8ef0d88
use null byte in key
goulvenclech Apr 9, 2024
98e9e90
tmp function in pipeline.ts
goulvenclech Apr 9, 2024
9306c71
refactor getVirtualModulePageName
goulvenclech Apr 9, 2024
03d4c59
refactor getPageKeyFromVirtualModulePageName
goulvenclech Apr 9, 2024
db7fdd2
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 9, 2024
21885cc
clean & comments
goulvenclech Apr 9, 2024
7e7469f
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 22, 2024
57137fd
better key and fix build
goulvenclech Apr 22, 2024
73ba151
utils: add makePageDataKey
goulvenclech Apr 22, 2024
96a6518
fix(pipeline): retrieveRoutesToGenerate for ssr
goulvenclech Apr 24, 2024
8793017
internals: getPageData function
goulvenclech Apr 24, 2024
8be75d4
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 24, 2024
ae2b017
tmp(ssr-split-manifest): fix test ?
goulvenclech Apr 25, 2024
b3ccdfb
fix?: ssr clean static output
goulvenclech Apr 25, 2024
2423ac2
internals: getPageDatasWithPublicKey
goulvenclech Apr 25, 2024
29da012
internals: getPageDatasByHoistedScriptId & getPagesDatasByComponent
goulvenclech Apr 25, 2024
cdbdcae
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 25, 2024
9b02b89
remove broken & useless virtualModuleNameFromResolvedId
goulvenclech Apr 26, 2024
cfec229
Merge branch 'main' into fix-all-pages-key
goulvenclech Apr 27, 2024
cd01367
chore: changeset
Princesseuh Apr 29, 2024
5c3a75f
fix: sanitize slashes in filepaths
Princesseuh Apr 29, 2024
896caf9
Revert "fix: sanitize slashes in filepaths"
goulvenclech Apr 29, 2024
7490882
fix?: remove route from virtual module name
goulvenclech Apr 29, 2024
c84d72c
fix: concat & array.from
goulvenclech Apr 29, 2024
2168f47
update changeset
goulvenclech Apr 29, 2024
984f6bf
clean unnecessary change
goulvenclech Apr 29, 2024
ced6e0f
remove unnecessary pageInfo
goulvenclech Apr 29, 2024
dfdfa7b
add return types to utils functions
goulvenclech Apr 29, 2024
cbf727e
revert a comment deletion
goulvenclech Apr 30, 2024
37ca1ae
fix cleanStaticOutput
goulvenclech Apr 30, 2024
846dd6f
changes from ematipico review
goulvenclech May 2, 2024
2c350d5
moving a todo outside jsdoc (cc @ematipico )
goulvenclech May 3, 2024
b424796
Update .changeset/great-turtles-greet.md
goulvenclech May 5, 2024
98d2364
Update .changeset/great-turtles-greet.md
goulvenclech May 5, 2024
1e17032
Merge remote-tracking branch 'origin/main' into fix-all-pages-key
ematipico May 8, 2024
8e2c178
Merge remote-tracking branch 'origin/main' into fix-all-pages-key
ematipico May 8, 2024
7477dc2
chore: fix merge conflicts
ematipico May 8, 2024
9d69309
fix: incorrect function
ematipico May 8, 2024
b8f3abc
remove logs
ematipico May 8, 2024
27395f4
revert: codepoint change
ematipico May 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/great-turtles-greet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"astro": minor
---

Adds the ability for multiple pages to use the same component as an `entrypoint` when building an Astro integration. This change is purely internal, and aligns the build process with the behaviour in the development server.
27 changes: 16 additions & 11 deletions packages/astro/src/core/build/generate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,7 @@ import { createRequest } from '../request.js';
import { matchRoute } from '../routing/match.js';
import { getOutputFilename } from '../util.js';
import { getOutDirWithinCwd, getOutFile, getOutFolder } from './common.js';
import {
cssOrder,
getEntryFilePathFromComponentPath,
getPageDataByComponent,
mergeInlineCss,
} from './internal.js';
import { cssOrder, getPageData, mergeInlineCss } from './internal.js';
import { BuildPipeline } from './pipeline.js';
import type {
PageBuildData,
Expand All @@ -61,6 +56,8 @@ import type {
StylesheetAsset,
} from './types.js';
import { getTimeStat, shouldAppendForwardSlash } from './util.js';
import { getVirtualModulePageName } from './plugins/util.js';
import { ASTRO_PAGE_MODULE_ID } from './plugins/plugin-pages.js';

function createEntryURL(filePath: string, outFolder: URL) {
return new URL('./' + filePath + `?time=${Date.now()}`, outFolder);
Expand All @@ -75,7 +72,7 @@ async function getEntryForRedirectRoute(
throw new Error(`Expected a redirect route.`);
}
if (route.redirectRoute) {
const filePath = getEntryFilePathFromComponentPath(internals, route.redirectRoute.component);
const filePath = getEntryFilePath(internals, route.redirectRoute);
if (filePath) {
const url = createEntryURL(filePath, outFolder);
const ssrEntryPage: SinglePageBuiltModule = await import(url.toString());
Expand All @@ -95,7 +92,7 @@ async function getEntryForFallbackRoute(
throw new Error(`Expected a redirect route.`);
}
if (route.redirectRoute) {
const filePath = getEntryFilePathFromComponentPath(internals, route.redirectRoute.component);
const filePath = getEntryFilePath(internals, route.redirectRoute);
if (filePath) {
const url = createEntryURL(filePath, outFolder);
const ssrEntryPage: SinglePageBuiltModule = await import(url.toString());
Expand Down Expand Up @@ -259,16 +256,15 @@ async function generatePage(
// prepare information we need
const { config, internals, logger } = pipeline;
const pageModulePromise = ssrEntry.page;
const pageInfo = getPageDataByComponent(internals, pageData.route.component);


// Calculate information of the page, like scripts, links and styles
const styles = pageData.styles
.sort(cssOrder)
.map(({ sheet }) => sheet)
.reduce(mergeInlineCss, []);
// may be used in the future for handling rel=modulepreload, rel=icon, rel=manifest etc.
const linkIds: [] = [];
const scripts = pageInfo?.hoistedScript ?? null;
const scripts = pageData.hoistedScript ?? null;
if (!pageModulePromise) {
throw new Error(
`Unable to find the module for ${pageData.component}. This is unexpected and likely a bug in Astro, please report.`
Expand Down Expand Up @@ -618,3 +614,12 @@ function createBuildManifest(
checkOrigin: settings.config.experimental.security?.csrfProtection?.origin ?? false,
};
}

/**
* For a given pageData, returns the entry file path—aka a resolved virtual module in our internals' specifiers.
*/
function getEntryFilePath(internals: BuildInternals, pageData: RouteData) {
const id =
'\x00' + getVirtualModulePageName(ASTRO_PAGE_MODULE_ID, pageData.component);
return internals.entrySpecifierToBundleMap.get(id);
}
150 changes: 83 additions & 67 deletions packages/astro/src/core/build/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,8 @@ import type { RouteData, SSRResult } from '../../@types/astro.js';
import type { PageOptions } from '../../vite-plugin-astro/types.js';
import { prependForwardSlash, removeFileExtension } from '../path.js';
import { viteID } from '../util.js';
import {
ASTRO_PAGE_RESOLVED_MODULE_ID,
getVirtualModulePageIdFromPath,
} from './plugins/plugin-pages.js';
import { RESOLVED_SPLIT_MODULE_ID } from './plugins/plugin-ssr.js';
import { ASTRO_PAGE_EXTENSION_POST_PATTERN } from './plugins/util.js';
import type { AllPagesData, PageBuildData, StylesheetAsset, ViteID } from './types.js';
import type { PageBuildData, StylesheetAsset, ViteID } from './types.js';
import { makePageDataKey } from './plugins/util.js';

export interface BuildInternals {
/**
Expand Down Expand Up @@ -45,7 +40,7 @@ export interface BuildInternals {
/**
* A map for page-specific information.
*/
pagesByComponent: Map<string, PageBuildData>;
pagesByKeys: Map<string, PageBuildData>;

/**
* A map for page-specific output.
Expand Down Expand Up @@ -126,7 +121,7 @@ export function createBuildInternals(): BuildInternals {
inlinedScripts: new Map(),
entrySpecifierToBundleMap: new Map<string, string>(),
pageToBundleMap: new Map<string, string>(),
pagesByComponent: new Map(),
pagesByKeys: new Map(),
pageOptionsByPage: new Map(),
pagesByViteID: new Map(),
pagesByClientOnly: new Map(),
Expand All @@ -153,7 +148,7 @@ export function trackPageData(
componentURL: URL
): void {
pageData.moduleSpecifier = componentModuleId;
internals.pagesByComponent.set(component, pageData);
internals.pagesByKeys.set(pageData.key, pageData);
internals.pagesByViteID.set(viteID(componentURL), pageData);
}

Expand Down Expand Up @@ -221,16 +216,77 @@ export function* getPageDatasByClientOnlyID(
}
}

export function getPageDataByComponent(
/**
* From its route and component, get the page data from the build internals.
* @param internals Build Internals with all the pages
* @param route The route of the page, used to identify the page
* @param component The component of the page, used to identify the page
*/
export function getPageData(
internals: BuildInternals,
route: string,
component: string
): PageBuildData | undefined {
if (internals.pagesByComponent.has(component)) {
return internals.pagesByComponent.get(component);
}
let pageData = internals.pagesByKeys.get(makePageDataKey(route, component));
if (pageData) { return pageData;}
return undefined;
}

/**
* Get all pages datas from the build internals, using a specific component.
* @param internals Build Internals with all the pages
* @param component path to the component, used to identify related pages
*/
export function getPagesDatasByComponent(
internals: BuildInternals,
component: string
): PageBuildData[] {
const pageDatas: PageBuildData[] = [];
internals.pagesByKeys.forEach((pageData) => {
if (component === pageData.component) pageDatas.push(pageData);
})
return pageDatas;
}

// TODO: Should be removed in the future. (Astro 5?)
/**
* Map internals.pagesByKeys to a new map with the public key instead of the internal key.
* This function is only used to avoid breaking changes in the Integrations API, after we changed the way
* we identify pages, from the entrypoint component to an internal key.
* If the page component is unique -> the public key is the component path. (old behavior)
* If the page component is shared -> the public key is the internal key. (new behavior)
* The new behavior on shared entrypoint it's not a breaking change, because it was not supported before.
* @param pagesByKeys A map of all page data by their internal key
*/
export function getPageDatasWithPublicKey(pagesByKeys: Map<string, PageBuildData>): Map<string, PageBuildData> {
// Create a map to store the pages with the public key, mimicking internal.pagesByKeys
const pagesWithPublicKey = new Map<string, PageBuildData>();

const pagesByComponentsArray = Array.from(pagesByKeys.values()).map((pageData) => {
return { component: pageData.component, pageData: pageData };
});

// Get pages with unique component, and set the public key to the component.
const pagesWithUniqueComponent = pagesByComponentsArray.filter((page) => {
return pagesByComponentsArray.filter((p) => p.component === page.component).length === 1;
});

pagesWithUniqueComponent.forEach((page) => {
pagesWithPublicKey.set(page.component, page.pageData);
});

// Get pages with shared component, and set the public key to the internal key.
const pagesWithSharedComponent = pagesByComponentsArray.filter((page) => {
return pagesByComponentsArray.filter((p) => p.component === page.component).length > 1;
});

pagesWithSharedComponent.forEach((page) => {
pagesWithPublicKey.set(page.pageData.key, page.pageData);
});

return pagesWithPublicKey;
}

export function getPageDataByViteID(
internals: BuildInternals,
viteid: ViteID
Expand All @@ -245,44 +301,8 @@ export function hasPageDataByViteID(internals: BuildInternals, viteid: ViteID):
return internals.pagesByViteID.has(viteid);
}

export function* eachPageData(internals: BuildInternals) {
yield* internals.pagesByComponent.values();
}

export function* eachPageFromAllPages(allPages: AllPagesData): Generator<[string, PageBuildData]> {
for (const [path, pageData] of Object.entries(allPages)) {
yield [path, pageData];
}
}

export function* eachPageDataFromEntryPoint(
internals: BuildInternals
): Generator<[PageBuildData, string]> {
for (const [entrypoint, filePath] of internals.entrySpecifierToBundleMap) {
// virtual pages can be emitted with different prefixes:
// - the classic way are pages emitted with prefix ASTRO_PAGE_RESOLVED_MODULE_ID -> plugin-pages
// - pages emitted using `build.split`, in this case pages are emitted with prefix RESOLVED_SPLIT_MODULE_ID
if (
entrypoint.includes(ASTRO_PAGE_RESOLVED_MODULE_ID) ||
entrypoint.includes(RESOLVED_SPLIT_MODULE_ID)
) {
const [, pageName] = entrypoint.split(':');
const pageData = internals.pagesByComponent.get(
`${pageName.replace(ASTRO_PAGE_EXTENSION_POST_PATTERN, '.')}`
);
if (!pageData) {
throw new Error(
"Build failed. Astro couldn't find the emitted page from " + pageName + ' pattern'
);
}

yield [pageData, filePath];
}
}
}

export function hasPrerenderedPages(internals: BuildInternals) {
for (const pageData of eachPageData(internals)) {
for (const pageData of internals.pagesByKeys.values()) {
if (pageData.route.prerender) {
return true;
}
Expand Down Expand Up @@ -342,27 +362,23 @@ export function mergeInlineCss(
return acc;
}

export function isHoistedScript(internals: BuildInternals, id: string): boolean {
return internals.hoistedScriptIdToPagesMap.has(id);
}

export function* getPageDatasByHoistedScriptId(
/**
* Get all pages data from the build internals, using a specific hoisted script id.
* @param internals Build Internals with all the pages
* @param id Hoisted script id, used to identify the pages using it
*/
export function getPageDatasByHoistedScriptId(
internals: BuildInternals,
id: string
): Generator<PageBuildData, void, unknown> {
): PageBuildData[]{
const set = internals.hoistedScriptIdToPagesMap.get(id);
const pageDatas: PageBuildData[] = [];
if (set) {
for (const pageId of set) {
const pageData = getPageDataByComponent(internals, pageId.slice(1));
if (pageData) {
yield pageData;
}
getPagesDatasByComponent(internals, pageId.slice(1)).forEach((pageData) => {
pageDatas.push(pageData);
});
}
}
}

// From a component path such as pages/index.astro find the entrypoint module
export function getEntryFilePathFromComponentPath(internals: BuildInternals, path: string) {
const id = getVirtualModulePageIdFromPath(path);
return internals.entrySpecifierToBundleMap.get(id);
return pageDatas;
}
11 changes: 7 additions & 4 deletions packages/astro/src/core/build/page-data.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import type { AllPagesData } from './types.js';

import * as colors from 'kleur/colors';
import { debug } from '../logger/core.js';
import { makePageDataKey } from './plugins/util.js';

export interface CollectPagesDataOptions {
settings: AstroSettings;
Expand Down Expand Up @@ -35,6 +36,8 @@ export async function collectPagesData(
// and is then cached across all future SSR builds. In the past, we've had trouble
// with parallelized builds without guaranteeing that this is called first.
for (const route of manifest.routes) {
// Generate a unique key to identify each page in the build process.
const key = makePageDataKey(route.route, route.component);
// static route:
if (route.pathname) {
const routeCollectionLogTimeout = setInterval(() => {
Expand All @@ -47,8 +50,8 @@ export async function collectPagesData(
clearInterval(routeCollectionLogTimeout);
}, 10000);
builtPaths.add(route.pathname);

allPages[route.component] = {
allPages[key] = {
key: key,
component: route.component,
route,
moduleSpecifier: '',
Expand All @@ -72,8 +75,8 @@ export async function collectPagesData(
continue;
}
// dynamic route:

allPages[route.component] = {
allPages[key] = {
key: key,
component: route.component,
route,
moduleSpecifier: '',
Expand Down
Loading
Loading