Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot combine raw and url for a js file #15753

Open
7 tasks done
wmertens opened this issue Jan 30, 2024 · 14 comments
Open
7 tasks done

Cannot combine raw and url for a js file #15753

wmertens opened this issue Jan 30, 2024 · 14 comments

Comments

@wmertens
Copy link
Contributor

Describe the bug

I'm trying to get a url to a js file so it is not processed, but I can't figure out how. When I do import url from './foo.js?url', the foo.js file will be processed as if it's a module, which is fair enough.

However, when I prevent that with import url from './foo.js?raw?url', I get a js file that exports the original js as a string.

How can I provide a link to an unchanged js file?

Reproduction

https://stackblitz.com/edit/vitejs-vite-hqvqjf?file=main.js&terminal=dev

Steps to reproduce

load the repro

System Info

any vite version

Used Package Manager

npm

Logs

No response

Validations

Copy link

stackblitz bot commented Jan 30, 2024

Fix this issue in StackBlitz Codeflow Start a new pull request in StackBlitz Codeflow.

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 31, 2024

If I understand correctly, does the following method meet your expectations?

import('./main.js?raw')
  .then(res => res.default)
  .then((t) => {
    const s = document.querySelector('#app');
    s.textContent = t;
  });

@wmertens
Copy link
Contributor Author

@XiSenao unfortunately no, I need it to be a url to the unchanged source file

@XiSenao
Copy link
Collaborator

XiSenao commented Jan 31, 2024

Module as a static resource module and placed in the public directory.

fetch('/main.js')
  .then((res) => res.text())
  .then((t) => {
    const s = document.querySelector('#app');
    s.textContent = t;
  });

@wmertens
Copy link
Contributor Author

@XiSenao then main.js gets minified etc, it needs to be the exact file.

I suppose I need something like ?noop, which is the same as ?raw except that it doesn't convert the file into a js module.

@sapphi-red
Copy link
Member

What is the use case of this? Does simply placing the file in public directory work?

@wmertens
Copy link
Contributor Author

wmertens commented Feb 8, 2024

@sapphi-red I need to provide a URL of the unchanged source code to a service worker, and using a public dir means a fixed path even when the contents change.

@sapphi-red
Copy link
Member

@wmertens Why does it need to be unchanged? I'm asking because it's not supported and want to know the detailed usage so that I can evaluate whether this feature should be builtin.

@wmertens
Copy link
Contributor Author

wmertens commented Feb 8, 2024

The specific usecase is for the Qwik Playground, which has a service worker that does SSR, and it does it by running rollup with either the minified or regular qwik server code. Depending on the selected version the system uses the "bundled" or CDN-provided package.

When I tried using ?raw?url, it resulted in errors because it doesn't like the minified code, and the sad effect is also that the production build minifies the non-minified version so it's harder to see what's going on. I now worked around it by using /public, but that means the code doesn't update for new versions without explicitly managing that.

How about a loader like ?as-is?url?

@sapphi-red
Copy link
Member

@wmertens Thanks for the explanation. Would a code like this work?

import jsContent from './foo.js?raw'

const blob = new Blob([jsContent], { type: "application/javascript" })
const url = URL.createObjectURL(blob)

How about a loader like ?as-is?url?

If we are going to add this feature, I think will go with ?url&raw.

@wmertens
Copy link
Contributor Author

wmertens commented Mar 1, 2024

@sapphi-red thank you for the workaround, it works even between window and serviceworker.

However, it's not ideal because it means that all the code needs to be parsed in the window to be converted to blob, which puts them all in the same chunk. With ?url&raw, the serviceworker can directly load only the required files.

(The window could do on-demand blob loading but that's a whole different structure than an object with urls)

@wmertens
Copy link
Contributor Author

@sapphi-red could you point me at the code that handles ?url? I can't find it and I would like to have ?url&raw working.

@hi-ogawa
Copy link
Collaborator

could you point me at the code that handles ?url? I can't find it and I would like to have ?url&raw working.

I think ?url is handled by vite:asset plugin

name: 'vite:asset',

While following the code, it appears that some-file.js?url already works like "url of source as is" for build but that's not the case for dev. This inconsistency is tracked in #6757 though the perspective is opposite since people usually wish build to work like dev.

@wmertens
Copy link
Contributor Author

I ended up making a small plugin, but it would be way better if ?raw&url were supported.
I even had to sidestep the static server, it was still processing somehow (I was getting processed results for different files 😵)

It lets Vite do the resolving of the path, but then intercepts everything. The assets plugin should work similarly IMHO, using ctx.load when raw isn't specified.

// Workaround for https://github.com/vitejs/vite/issues/15753
// A vite plugin that implements what `?raw&url` should do
// Use `import url from 'file?raw-source'` to get the url for the raw file content
export function rawSource(): Plugin {
  let base: string;
  let isDev: boolean = false;
  let doSourceMap: boolean = false;
  const extToMime = {
    js: 'application/javascript',
    css: 'text/css',
    html: 'text/html',
    json: 'application/json',
    wasm: 'application/wasm',
  };
  return {
    name: 'raw-source',

    configResolved(config) {
      base = config.base;
      isDev = config.command === 'serve';
      doSourceMap = !!config.build.sourcemap;
    },

    configureServer(server) {
      // Vite still processes /@fs urls, so we need to run our own static server
      server.middlewares.use((req, res, next) => {
        if (req.url!.startsWith('/@raw-fs')) {
          const filePath = req.url!.slice('/@raw-fs'.length);
          if (existsSync(filePath)) {
            const ext = filePath.split('.').pop()! as keyof typeof extToMime;
            const contentType = extToMime[ext] || 'application/octet-stream';
            res.setHeader('Content-Type', contentType);
            res.end(readFileSync(filePath));
          } else {
            res.statusCode = 404;
            res.end('File not found');
          }
        } else {
          next();
        }
      });
    },

    resolveId: {
      order: 'pre',
      async handler(id, importer) {
        const match = /^(?<path>.*)\?(|(?<before>.+)&)raw-source($|&(?<after>.*))/.exec(id);

        if (match) {
          const newQuery = [match.groups!.before, match.groups!.after].filter(Boolean).join('&');
          const newId = `${match.groups!.path}${newQuery ? `?${newQuery}` : ''}`;
          const resolved = await this.resolve(newId, importer, {
            skipSelf: true,
          });
          if (!resolved) {
            throw new Error(`Could not resolve "${id}" from "${importer}"`);
          }
          return `\0raw-source:${resolved!.id.split('?')[0]}`;
        }
      },
    },

    load(id) {
      if (id.startsWith('\0raw-source:')) {
        let path = id.slice('\0raw-source:'.length);
        if (path.startsWith('/@fs/')) {
          path = path.slice('/@fs'.length);
        }
        if (isDev) {
          const devUrl = `${base}@raw-fs${path}`;
          return `export default "${devUrl}";`;
        }
        const fileContent = readFileSync(path);
        const ref = this.emitFile({
          type: 'asset',
          name: basename(path),
          source: fileContent,
        });
        return {
          code: `export default "__RAW-URL_${ref}__";`,
          map: { version: 3, sources: [path], mappings: '' },
        };
      }
    },

    renderChunk(code, chunk) {
      // Copied from vite assets code and simplified
      let s, match;
      while ((match = /__RAW-URL_([^_]+)__/g.exec(code))) {
        s ||= new MagicString(code);
        const ref = match[1];
        const fileName = this.getFileName(ref);
        chunk.viteMetadata!.importedAssets.add(fileName);
        s.overwrite(match.index, match.index + match[0].length, base + fileName);
      }
      return s
        ? {
            code: s ? s.toString() : code,
            map: doSourceMap ? s.generateMap({ hires: true }) : null,
          }
        : null;
    },
  };
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants