[Feature]: virtual module support

[hardfist]

What problem does this feature solve?

Definition

Virtual Module: module not exists on real filesystem

Background

Virtual module is widely used in bundler ecosystem( vite、webpack), and different bundlers take different approaches,
webpack uses https://www.npmjs.com/package/webpack-virtual-modules?activeTab=readme to support virtual module and vite use https://vitejs.dev/guide/api-plugin.html#virtual-modules-convention to support virtual module
Currently rspack implement a naive implementation of webpack-virtual-modules(https://github.com/rspack-contrib/rspack-plugins/tree/main/packages/plugin-virtual-module) which has some limitation.

Since Rspack already supports unplugin so it can actually support both implementation, but too many flexibilities may but ecosystem in chaos(just like vite's virtual module path can be virtual:my-module and \0xxx which makes it harder to unify plugin usage).

resolve hook vs input_filesystem

webpack use patched input_filesystem to implement virtual modules which limits virtual module path to be real path url(which may be not bad), so the internal resolver can handle it correctly, vite & rollup use resolve and load and virtual-module-convention to handle virtual path(which is more flexible).

It seems webpack-virtual-modules using patched input_filesystem strategy is not a strong limitation in real world usage and is much simpler and performance friendly.

There're 3 ways Rspack can implement virtual module like webpack did

• Implement builtin rust version of virtualModulePlugin: performance friendly
• Compatible with webpack-virtual-modules, which needs Rspack implement input_filesystem options, which may cause performance regression since every fs call needs trigger napi communication
• Maintain the status quo

It seems the second way is more flexible and ecosystem friendly, we may need to investigate the performance regression, if the regression is acceptable we can be compatible with webpack-virtual-modules, then do we still need implement a virtualModulePlugin for better out of box experience.

What does the proposed API of configuration look like?

if we choose implement rust version of this plugin then same as https://www.npmjs.com/package/webpack-virtual-modules

[ahabhgk]

This probably can split into two layer: memory input filesystem and virtual modules plugin, memory input filesystem for more general use cases and virtual modules plugin is just a wrapper plugin based on memory input filesystem

[hardfist]

This probably can split into two layer: memory input filesystem and virtual modules plugin, memory input filesystem for more general use cases and virtual modules plugin is just a wrapper plugin based on memory input filesystem

yeah, that's the plan, but when you say "memory input filesystem" you mean

• passing memory input filesystem from js side to rust side and every read trigger js side readfile call
• passing memory input(like object) to rust side and every read will only trigger rust side readfile call

[ahabhgk]

Definitely the second one, but we need to consider to make a better API for user. A MemoryFileSystemRspackPlugin or an enum/options of compiler.inputFileSystem that passes to rust

[hardfist]

Definitely the second one, but we need to consider to make a better API for user. A MemoryFileSystemRspackPlugin or an enum/options of compiler.inputFileSystem that passes to rust

yes, I will make a RFC about InputFilesystem and VirtualModule support

[stale]

This issue has been automatically marked as stale because it has not had recent activity. If this issue is still affecting you, please leave any comment (for example, "bump"). We are sorry that we haven't been able to prioritize it yet. If you have any new additional information, please include it with your comment!

[hardfist]

bump

[watsonhaw5566]

这个 feature 还有在进行吗， hardfist 兄

[hardfist]

这个 feature 还有在进行吗， hardfist 兄

no, maybe we will use inputFs to implement when we solve the performance problem