Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat proposal enable fetching external resources using native fetch API in Node.js version 18 later #935

Merged
merged 11 commits into from
Sep 26, 2022
Merged

Feat proposal enable fetching external resources using native fetch API in Node.js version 18 later #935

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
3473559
Enable fetching external resources using native fetch API in Node.js …
takurinton Sep 18, 2022
c9d9d56
fix: remove fetch after redefinition to avoid infinite loop
takurinton Sep 18, 2022
aa0818d
fix: when Node.js version is only over 18 define `_fetch` instead na…
takurinton Sep 18, 2022
8acd946
add: changeset
takurinton Sep 19, 2022
74f90a9
fix
takurinton Sep 19, 2022
54e5d2a
fix: globalThis.fetch only calls when fetch is undefined and isExtern…
takurinton Sep 22, 2022
27def4e
fix: override with own implementation if you want to retrieve local f…
takurinton Sep 22, 2022
f1a51dd
fix: tests
takurinton Sep 22, 2022
38b7dd5
Merge branch 'main' into add-fetch-for-node-18
takurinton Sep 22, 2022
63954dd
fix: test names
takurinton Sep 22, 2022
7c1bb6c
Merge branch 'main' into add-fetch-for-node-18
takurinton Sep 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/young-guests-tap.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'wmr': minor
---

Add fetching external resources using native fetch API when Node.js version 18 later
38 changes: 35 additions & 3 deletions packages/wmr/src/lib/prerender.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,11 +96,43 @@ async function workerCode({ cwd, out, publicPath, customRoutes }) {
let head = { lang: '', title: '', elements: new Set() };
globalThis.wmr = { ssr: { head } };

// @ts-ignore
globalThis.fetch = async url => {
async function localFetcher(url) {
const text = () => fs.readFile(`${out}/${String(url).replace(/^\//, '')}`, 'utf-8');
return { text, json: () => text().then(JSON.parse) };
};
}

const pattern = /^\//;
if (globalThis.fetch === undefined) {
// When Node.js version under 17, native fetch API undefined
// So it will give a warning when the syntax to retrieve the file and the url is passed
// @ts-ignore
globalThis.fetch = async url => {
if (pattern.test(String(url))) {
return localFetcher(url);
}

console.warn(`fetch is not defined in Node.js version under 17, please upgrade to Node.js version 18 later`);
};
} else {
// At that time, implement using reassignment to avoid entering an infinite loop.
const fetcher = fetch;
// @ts-ignore
delete globalThis.fetch;

// When Node.js version 18 later, native fetch API is defined
// Override with own implementation if you want to retrieve local files in Node.js 18 later
// ref https://github.com/preactjs/wmr/pull/935#discussion_r977168334
// @ts-ignore
globalThis.fetch = async (url, options) => {
if (pattern.test(String(url))) {
return localFetcher(url);
}

// When fetching an external resource, it returns the native fetch API
// though `fetcher` function is an alias created to avoid infinite loops
return fetcher(url, options);
Comment on lines +106 to +133
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments explaining the workflow here are excellent! Really good stuff.

};
}

// Prevent Rollup from transforming `import()` here.
const $import = new Function('s', 'return import(s)');
Expand Down
10 changes: 10 additions & 0 deletions packages/wmr/test/fixtures/prerender-external-resource-fetch/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf8" />
<title>default title</title>
</head>
<body>
<script src="./index.js" type="module"></script>
</body>
</html>
4 changes: 4 additions & 0 deletions packages/wmr/test/fixtures/prerender-external-resource-fetch/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export async function prerender() {
const html = await fetch('https://preactjs.com').then(res => res.text());
return { html, links: ['/'] };
}
2 changes: 1 addition & 1 deletion packages/wmr/test/fixtures/prerender-resource-fetch/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export async function prerender() {
const md = await fetch('content.md').then(res => res.text());
const md = await fetch('/content.md').then(res => res.text());
rschristian marked this conversation as resolved.
Show resolved Hide resolved
return { html: md, links: ['/'] };
}
25 changes: 25 additions & 0 deletions packages/wmr/test/production.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -889,6 +889,31 @@ describe('production', () => {
const index = await fs.readFile(indexHtml, 'utf8');
expect(index).toMatch(/# hello world/);
});

it('should not support fetching resources from external link prerender', async () => {
await loadFixture('prerender-external-resource-fetch', env);
instance = await runWmr(env.tmp.path, 'build', '--prerender');
const code = await instance.done;

const nodeVersion = process.version;
const isNodeVersionOver18 = Number(nodeVersion.split('.')[0].slice(1)) >= 18;
if (isNodeVersionOver18) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to run tests on multiple versions?

Copy link
Contributor Author

@takurinton takurinton Sep 18, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, and two versions, 12 and 14, are running now. This felt like a no problem in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also considering breaking down this test.

Instead of branching using if in one test, write a test for Node.js version 17 or less and a test for Node.js version 18 or more, and skip if the version is not applicable. I think it is also possible.
This part is one of the things I want to discuss in this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of branching using if in one test, write a test for Node.js version 17 or less and a test for Node.js version 18 or more, and skip if the version is not applicable.

I like this idea. There's slightly different behavior between the two that (ideally) we'd test.

We can definitely alter the CI config to cover newer versions of Node.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I completely forgot about this.
Last time I called fetch without the leading slash here, but this test does not work with the regex changes you reviewed.
https://github.com/preactjs/wmr/blob/main/packages/wmr/test/fixtures/prerender-resource-fetch/index.js#L2

I think to need to modify the test here as well. I think the comments here will be helpful for that.
#935 (comment)

Copy link
Contributor Author

@takurinton takurinton Sep 22, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, no correction of above comment was necessary.

// when workflow of wmr runtime uses Node.js 18
// Until then check locally.
expect(code).toBe(0);
const indexHtml = path.join(env.tmp.path, 'dist', 'index.html');
const index = await fs.readFile(indexHtml, 'utf8');
expect(index).toMatch(/Preact/);
} else {
// Throw error when Node.js version under 17
await withLog(instance.output, async () => {
expect(code).toBe(1);
expect(instance.output.join('\n')).toMatch(
/fetch is not implemented in Node.js, please upgrade to Node.js 18 or above/
);
});
}
});
});

describe('Code Splitting', () => {
Expand Down