Use u64 range instead of usize, for better wasm32 support

[XiangpengHao]

Which issue does this PR close?

• Closes Support 32-bit Architectures / Replace usize with u64 #5351

Rationale for this change

Ranges should be u64 so that 32 bit platforms can read files larger than 4GB. More details can be found in #5351

What changes are included in this PR?

This is a rather simple change. No functionality change for 64 bit platforms. But is a breaking change for trait implementors.
Given that we already break one in #6619, it's seems like a good timing to also include this change.

This PR added and removed a few casting. I have checked the casting is ok, but please help me check again.

My principle of using usize vs u64:

• Anything already in memory should use usize, e.g., slicing a memory range
• O.w. use u64

Are there any user-facing changes?

[alamb]

The CI failure https://github.com/apache/arrow-rs/actions/runs/12713696201/job/35442156669?pr=6961 should be fixed with this PR:

• Fix WASM CI for Rust 1.84 release #6963

[alamb]

Thanks @XiangpengHao ❤️

Can you help me understand what currently doesn't work with 32 bit builds now?

We have an existing test for WASM32 that seems to work fine: https://github.com/apache/arrow-rs/actions/runs/12680096825/job/35341184364

I also locally tried checking with an i686 target and that worked fine too 🤔

rustup target add i686-unknown-linux-gnu
cd object_store
cargo check --target=i686-unknown-linux-gnu
...

🤔

[alamb]

🤔 maybe it is related to wanting to use the http features?

But it still seems to compile just fine for me 🤔

sudo apt-get install gcc-multilib
rustup target add i686-unknown-linux-gnu
cd object_store
cargo check --target=i686-unknown-linux-gnu --features=http
...

[alamb]

Can you help me understand what currently doesn't work with 32 bit builds now?

For anyone following along, the answer is here: #5351 (comment)

I still think we should try and add some sort of test that would fail on wasm32 before this change and not after the change. If we don't do that I feel like:

1. We are less sure we fixed all the places that require change
2. We may inadvertently introduce regressions but no test would fail

I'll see if I can find some time over the next day or two to help, if no on ebeats me to it

[XiangpengHao]

add some sort of test that would fail on wasm32

Thanks for the review @alamb , I agree with the rationale but unfortunately we don't yet have the infra to test wasm32 yet. We currently only build on wasm32, but there's no runner that can actually execute the webassembly

[Xuanwo]

Thanks for the review @alamb , I agree with the rationale but unfortunately we don't yet have the infra to test wasm32 yet. We currently only build on wasm32, but there's no runner that can actually execute the webassembly

OpenDAL has an edge test for this; it's worth taking a look.

https://github.com/apache/opendal/blob/781682cc05b31d1f12f91fb7568a868f038c4d3b/.github/workflows/test_edge.yml#L62-L107

The test code could be found here: https://github.com/apache/opendal/tree/main/core/edge/s3_read_on_wasm

[alamb]

Thank you @XiangpengHao -- I reviewed this PR carefully and I think the API changes are good but there are a few places where going from u64 to usize should be checked (as it could truncate on WASM).

I may be mistaken (my casting knowledge is mostly left over from C/C++ 😅 )

I also have a few other suggestions related to comments, but otherwise this is good to go from my perspectivel

Thank you again for helping push this along

[alamb]

again, I think we need to check the conversion here rather than truncating

[XiangpengHao]

this function doesn't return error, I just unwrap here, as it's only a test

[alamb]

BTW I merged up from main. I also think the obejct_store emulator tests are unrelated to this PR. I have filed

• Intermittent failure with object_store aws::tests::s3_test emulator  #6981

To track

[alamb]

Thanks for the review @alamb , I agree with the rationale but unfortunately we don't yet have the infra to test wasm32 yet. We currently only build on wasm32, but there's no runner that can actually execute the webassembly

OpenDAL has an edge test for this; it's worth taking a look.

https://github.com/apache/opendal/blob/781682cc05b31d1f12f91fb7568a868f038c4d3b/.github/workflows/test_edge.yml#L62-L107

The test code could be found here: https://github.com/apache/opendal/tree/main/core/edge/s3_read_on_wasm

Thank you @Xuanwo -- I didn't see this response before posting my update

[alamb]

I restarted the emulator test to see if it passes:
Old: https://github.com/apache/arrow-rs/actions/runs/12770295148/job/35594807418?pr=6961

[alamb]

Now the emulator test is passing after 2 retries: https://github.com/apache/arrow-rs/actions/runs/12770295148/job/35607985446?pr=6961

🤔