fix: partial workaround for high address support #62
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some SDKs, like the js-sdk, store block index information in the top 16 bits of the 64-bit memory address. This is incompatible with the JS-PDK at present, since we truncate addresses to 32 bits. In advance of BigInt support in Javy/quickjs-wasm-sys, expose memory offset and lengths as JS Float values. This leans on the JS convention of storing (up to 53 bits of) integer data losslessly in an f64. This gives us 5 bits of block index for the JS-SDK, enough room for *slaps roof of memory allocator* 31 allocations in this bad boy (before requiring an extism memory reset.) QuickJS itself supports bigint, so it's a matter of looping that functionality out through quickjs-sys. (I haven't been able to find a technical reason this wasn't done initially, though that doesn't mean there aren't reasons!)
🤣 |
nilslice
reviewed
Apr 5, 2024
bhelx
reviewed
Apr 8, 2024
| @@ -442,47 +442,63 @@ fn build_memory(context: &JSContextRef) -> anyhow::Result<JSValueRef> { | |||
| let data = data.as_bytes()?; | |||
| let m = extism_pdk::Memory::from_bytes(data)?; | |||
| let mut mem = HashMap::new(); | |||
| let offset = JSValue::Int(m.offset() as i32); | |||
| let len = JSValue::Int(m.len() as i32); | |||
| let offset = JSValue::Float(m.offset() as f64); | |||
There was a problem hiding this comment.
Looks good and makes sense to me, but only really in the context of this PR. I think a single comment here about why we're packing the offset into a float may be helpful.
There was a problem hiding this comment.
Excellent point – added a note!
bhelx
approved these changes
Apr 8, 2024
chrisdickinson
force-pushed
the
chris/20240405-workaround-for-high-bytes
branch
from
6497fc3 to
03fb8cd
Compare
|
Thanks for the reviews, all! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Some SDKs, like the js-sdk, store block index information in the top 16 bits of the 64-bit memory address. This is incompatible with the JS-PDK at present, since we truncate addresses to 32 bits. In advance of BigInt support in Javy/quickjs-wasm-sys, expose memory offset and lengths as JS Float values. This leans on the JS convention of storing (up to 53 bits of) integer data losslessly in an f64.
This gives us 5 bits of block index for the JS-SDK, enough room for slaps roof of memory allocator 32 allocations in this bad boy (before requiring an extism memory reset.)
QuickJS itself supports bigint, so it's a matter of looping that functionality out through quickjs-sys. (I haven't been able to find a technical reason this wasn't done initially, though that doesn't mean there aren't reasons!)