Skip to content

Commit

Permalink
feat: import.meta.glob support ?raw (#5545)
Browse files Browse the repository at this point in the history
  • Loading branch information
ygj6 authored Dec 28, 2021
1 parent 6d4ee18 commit 5279de6
Show file tree
Hide file tree
Showing 5 changed files with 58 additions and 19 deletions.
12 changes: 12 additions & 0 deletions packages/playground/glob-import/__tests__/glob-import.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,24 @@ const allResult = {
}
}

const rawResult = {
'/dir/baz.json': {
msg: 'baz'
}
}

test('should work', async () => {
expect(await page.textContent('.result')).toBe(
JSON.stringify(allResult, null, 2)
)
})

test('import glob raw', async () => {
expect(await page.textContent('.globraw')).toBe(
JSON.stringify(rawResult, null, 2)
)
})

if (!isBuild) {
test('hmr for adding/removing files', async () => {
addFile('dir/a.js', '')
Expand Down
16 changes: 16 additions & 0 deletions packages/playground/glob-import/index.html
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<pre class="result"></pre>
<pre class="globraw"></pre>

<script type="module" src="./dir/index.js"></script>
<script type="module">
Expand All @@ -25,3 +26,18 @@
document.querySelector('.result').textContent = JSON.stringify(res, null, 2)
})
</script>

<script type="module">
const rawModules = import.meta.globEager('/dir/*.json', {
assert: { type: 'raw' }
})
const globraw = {}
Object.keys(rawModules).forEach((key) => {
globraw[key] = JSON.parse(rawModules[key])
})
document.querySelector('.globraw').textContent = JSON.stringify(
globraw,
null,
2
)
</script>
1 change: 1 addition & 0 deletions packages/vite/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
"//": "READ CONTRIBUTING.md to understand what to put under deps vs. devDeps!",
"dependencies": {
"esbuild": "^0.13.12",
"json5": "^2.2.0",

This comment has been minimized.

Copy link
@TrySound

This comment has been minimized.

Copy link
@patak-dev

patak-dev Jan 13, 2022

Member

Do you have an idea to replace this one? We could remove comments with a simple regex, and then apply another regex for the JS Object. We may not need a full parser here, only very simple static JS objects are allowed. We went with json5 because we saw the dep was used somewhere else already

This comment has been minimized.

Copy link
@TrySound

TrySound Jan 13, 2022

Contributor

I see babel use it. But babel is not part of vite itself. Idk, maybe eval for simplicity? Or it can be a security issue?

This comment has been minimized.

Copy link
@TrySound

This comment has been minimized.

Copy link
@patak-dev

patak-dev Jan 13, 2022

Member

But we don't have an AST pass here. I don't see other parts where we are using an eval like this. A regex based approach sounds better to me, but maybe the regexes are too complex?

This comment has been minimized.

Copy link
@patak-dev

patak-dev Jan 16, 2022

Member

@TrySound we talked about json5 in the last team meeting. There is a high chance that we are going to need this dep for other things in the future, so I don't know if it makes sense to put much effort here. One possible option would be to replace it with a handwritten parser, since the grammar is quite simple (we have the same kind of parsers in other parts of the code), but there is also the tradeoff here of accepting that extra complexity in the code base. (also, eval isn't safe as we discussed in this context)

This comment has been minimized.

Copy link
@TrySound

TrySound Jan 16, 2022

Contributor

Alright. Is there a reason why json5 is not bundled?

This comment has been minimized.

Copy link
@patak-dev

patak-dev Jan 16, 2022

Member

I don't think there is a reason, if it would be better to have it bundled, we could do that

"postcss": "^8.4.5",
"resolve": "^1.20.0",
"rollup": "^2.59.0"
Expand Down
31 changes: 26 additions & 5 deletions packages/vite/src/node/importGlob.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import path from 'path'
import { promises as fsp } from 'fs'
import glob from 'fast-glob'
import * as JSON5 from 'json5'
import {
isModernFlag,
preloadMethod,
Expand All @@ -8,6 +10,12 @@ import {
import { cleanUrl } from './utils'
import type { RollupError } from 'rollup'

export interface AssertOptions {
assert?: {
type: string
}
}

export async function transformImportGlob(
source: string,
pos: number,
Expand Down Expand Up @@ -38,7 +46,7 @@ export async function transformImportGlob(
importer = cleanUrl(importer)
const importerBasename = path.basename(importer)

let [pattern, endIndex] = lexGlobPattern(source, pos)
let [pattern, assertion, endIndex] = lexGlobPattern(source, pos)
if (!pattern.startsWith('.') && !pattern.startsWith('/')) {
throw err(`pattern must start with "." or "/" (relative to project root)`)
}
Expand Down Expand Up @@ -79,8 +87,12 @@ export async function transformImportGlob(
;[importee] = await normalizeUrl(file, pos)
}
imports.push(importee)
const identifier = `__glob_${importIndex}_${i}`
if (isEager) {
if (assertion?.assert?.type === 'raw') {
entries += ` ${JSON.stringify(file)}: ${JSON.stringify(
await fsp.readFile(path.join(base, file), 'utf-8')
)},`
} else if (isEager) {
const identifier = `__glob_${importIndex}_${i}`
importsString += `import ${
isEagerDefault ? `` : `* as `
}${identifier} from ${JSON.stringify(importee)};`
Expand Down Expand Up @@ -115,7 +127,10 @@ const enum LexerState {
inTemplateString
}

function lexGlobPattern(code: string, pos: number): [string, number] {
function lexGlobPattern(
code: string,
pos: number
): [string, AssertOptions, number] {
let state = LexerState.inCall
let pattern = ''

Expand Down Expand Up @@ -163,7 +178,13 @@ function lexGlobPattern(code: string, pos: number): [string, number] {
}

const endIndex = getEndIndex(code, i)
return [pattern, endIndex + 1]
const options = code.substring(i + 1, endIndex)
const commaIndex = options.indexOf(`,`)
let assert = {}
if (commaIndex > -1) {
assert = JSON5.parse(options.substr(commaIndex + 1))
}
return [pattern, assert, endIndex + 1]
}

// reg without the 'g' option, only matches the first match
Expand Down
17 changes: 3 additions & 14 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 5279de6

Please sign in to comment.