Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run object_store tests (on CI) with WASM build/runtime #6980

Open
Tracked by #6818
alamb opened this issue Jan 14, 2025 · 1 comment
Open
Tracked by #6818

Run object_store tests (on CI) with WASM build/runtime #6980

alamb opened this issue Jan 14, 2025 · 1 comment
Labels
enhancement Any new improvement worthy of a entry in the changelog object-store Object Store Interface

Comments

@alamb
Copy link
Contributor

alamb commented Jan 14, 2025

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

As we improve the story for using object_store with WASM we currently check that the code compiles with WASM, but have no coverage for actually running tests iina. WASM runtime to verify behavior. For example, as @XiangpengHao points out we have no way to verify support for files larger than 4GB (where WASM's 32-bit usize can be a problem):

Describe the solution you'd like
I would like to run all relevant tests from object_store on a wasm runtime to ensure they actually run

Describe alternatives you've considered
I don't really know much about how this works, but I can imagine potentially using this crate:

https://github.com/bytecodealliance/wasmtime

Somehow to run the generated wasm

Additional context

@alamb alamb added enhancement Any new improvement worthy of a entry in the changelog object-store Object Store Interface labels Jan 14, 2025
@alamb
Copy link
Contributor Author

alamb commented Jan 14, 2025

@Xuanwo provided an example of how they do it in opendal: https://github.com/apache/opendal/blob/781682cc05b31d1f12f91fb7568a868f038c4d3b/.github/workflows/test_edge.yml#L62-L107

Thanks for the review @alamb , I agree with the rationale but unfortunately we don't yet have the infra to test wasm32 yet. We currently only build on wasm32, but there's no runner that can actually execute the webassembly

OpenDAL has an edge test for this; it's worth taking a look.

https://github.com/apache/opendal/blob/781682cc05b31d1f12f91fb7568a868f038c4d3b/.github/workflows/test_edge.yml#L62-L107

The test code could be found here: https://github.com/apache/opendal/tree/main/core/edge/s3_read_on_wasm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog object-store Object Store Interface
Projects
None yet
Development

No branches or pull requests

1 participant