I write this project to help myself explore WASM pottentials.
Most of the time when you deal with wasm memory, You want to read a complex data like a C++ class or a simple c struct.
To do that you need to master how to offset and use raw byte-array,
And for Complex use cases it will quickly become P.i.t.A.
It can run in almost all TS/JS runtimes ( browser / bun / nodejs / deno ).
It uses JS DataViews and ArrayBuffer to handle memory operations.
Because of that, it can handle unaligned layouts and endianess.
This project first idea came from @Tsoding project Elf.JS which tries to parse elf64 binaries in browser.
We can define our memory layout in JS/TS for a data type .
Then without any error or raw byte handling complxity access (R/W) data on to ArrayBuffers.
It uses JS's DataViews to ensure cross platform abilities and littleEndian.
You can define memory layout (like C-struct) to define how a byte-array data must be read from or wrote to.
// TS / JS
const Ball = new Strukt('Ball', {
'id' : 'u64',
'posx': 'f32',
'posy': 's16',
'target': ['s8', 5],
});The above definition is equalant to C-struct bellow:
// C / C++
typedef struct {
unsigned long long id,
float posx,
signed short int posy,
signed char target[5],
} Ball;Or in V:
struct Ball {
id u64
posx f32
posy s16
target [5]i8
}Any other language that generate c-style struct layout works as well.
To import the library in different runtimes, we need to use different methods.
For bun and nodejs , first we need to install it via a package manager (pnpm, yarn, npm) from npm repository:
pnpm install struktjs
# or
yarn add struktjs
# or
npm i --save struktjs@latestThen in bun we can import strukt.ts file directly.
// Bun
import {Strukt} from "struktjs/strukt.ts"; // typescript file
import {Strukt} from "struktjs"; // js file// NodeJS esm module:
import {Strukt} from "struktjs";
// NodeJS commonjs: NOT POSSIBLE right now
const {Strukt} = require("struktjs/index.cjs");In deno we can directly import the file.
Notice we can remove or change version (@latest in the URL) to get what you want.
like:
// jsdelivr
import {Strukt} from "https://cdn.jsdelivr.net/npm/struktjs@latest/strukt.ts";
// or unpkg
import {Strukt} from "http://unpkg.com/struktjs@latest/strukt.ts";
// or github
import {Strukt} from "https://github.com/HKhademian/Strukt.js/raw/main/strukt.ts";Like deno, we need to use direct file import:
<head>
<script type="module">
// jsdelivr
import {Strukt} from "https://cdn.jsdelivr.net/npm/struktjs@latest/strukt.mjs";
// or unpkg
import {Strukt} from "http://unpkg.com/struktjs@latest/strukt.mjs";
// or github
import {Strukt} from "https://github.com/HKhademian/Strukt.js/raw/main/strukt.mjs";
/* ......... REST OF LOGIC ......... */
</script>
</head>Now we can read/write using ArrayBuffers.
This ArrayBuffers usually provided by WASM memory layouts.
For example see tsoding/elf.js
Then to parse that bare bytes, using previous defined Ball struct:
const ballByteSize = ball.size; // sizeof(Ball) in c which gives bytesize of a Ball Strukt
const buf = new ArrayBuffer(ballByteSize*2); // allocat memory for 2 ball obj
// ----- WRITE 2 balls in buf -----
// notice we can use BigInt, Number, even string to assign values
Ball.writeTo(buf, {
'id' : 1,
'posx': 5.5,
'posy': '5',
'target': [1,2,3,4,5],
}, /*offset:*/0*ballByteSize);
Ball.writeTo(buf, {
'id' : 2n,
'posx': '6',
'posy': 2,
'target': [1,1,2,2,9],
}, /*offset:*/1*ballByteSize, /*littleEndian:*/false);
// ----- READ second ball from buf -----
const ball2 = Ball.readOneFrom(buf, /*offset:*/1*ballByteSize, /*littleEndian:*/false);
// READ first ball
const ball1 = Ball.readOneFrom(buf);
/* here ball1 uses BigInt for 'id' and Number for others */
// edit and rewite
ball1.posx += 10;
Ball.writeTo(buf, ball1);Strukt.prototype.constructor( schema ) // => a schema for strukt
// schema can be Object / Map / TuppleArray
// use Map in environments (like older browsers) to keep orders
// see (Map vs Object) bellow
// also you can use array of tuples [name, type][] instead.
// remmember TS's auto schema type detection for both Map and TuppleArray is ***not*** supported
Strukt.prototype.size // => get calculate bytesize of schema
Strukt.prototype.schema // => clone of strukt inner schema
Strukt.prototype.typeOf(fieldName) // => returns type of that field
Strukt.prototype.sizeOf(fieldName) // => returns bytesize of that field
Strukt.prototype.offsetOf(fieldName) // => returns bytesize offset from beginning of struct data
Strukt.prototype.write(record, offset=0, littleEndian?) // => create a new arrayBuffer with size of that record and `writeTo` it
Strukt.prototype.writeTo(buf, record, offset=0, littleEndian?) // => wites `record` data to `buf` start from `offset` with respect of `littleEndian`
Strukt.prototype.readOneFrom(buf, offset=0, littleEndian?) // => reads one record from buf
Strukt.prototype.readAllFrom(buf, offset=0, count=1, littleEndian?) // => reads `count` records sequentialy from `buf`see: Map vs Object
in todays environments it's ok to use regular key/val dictionaries (JS Object) but in older definitions there is no garentee to keep entries order as it appears in src code.
So you can use js Map class to precise declaration in older environments.
This project source has developed in Typescript and transpiled to JavaScript. Also you can use it in bare JS environments like browsers or nodejs, But you can get type hints in typescript envs like bun/deno. I will add type declaration for js later.
const MyStrukt = new Strukt('my', {
'a': ['u8',2],
'b': 'ptr',
});
// auto calculation of Schema type in TS code
type myStructType = typeof MyStrukt;
// = Strukt<type {'a': ['u8',2], 'b': 'ptr'}>
type mySchemaType = SchemaOf<typeof MyStrukt>;
// = type {'a': ['u8',2], 'b': 'ptr'}
type myRecordType = RecordOf<typeof MyStrukt>;
// = type {'a': number[], 'b': BigInt}
// TODO: not completed yet
const x: myRecordType = MyStrukt.readOneFrom(...);
x[/*just `mySchemaType` keys (a / b)*/ 'a'];
MyStrukt.sizeOf(/*just `mySchemaType` keys*/);
MyStrukt.typeOf(/*just `mySchemaType` keys*/);
MyStrukt.offsetOf(/*just `mySchemaType` keys*/);
MyStrukt.write(record /*: myRecordType*/);There is a limit number of known numbers in DataView ( u8 , u16 , u32 , u64 , s8 , s16 , s32 , s64 , f32 , f64 )
So all we can do is to define aliases. like 'i8' <=> 's8', etc.
You can redefine inner type alises or add new ones to use in Strukt methods.
Keep in mind that TS type system cannot handle these changes (you may need to use casting to ignore errors/warnings)
import {field_types_data} from 'strukt/strukt.ts';
// you can change default size type to use 32bit numbers
field_types_data['size'] = field_types_data['size32'];
// or define your new type aliases
field_types_data['my_awesome_int'] = field_types_data['u16'];Another thing to note is that in JS all numbers are floating-point.
And because of how they defined, they cannot hold 64bit integer range.
So you need too use BigInt to handle these large number ranges.
You can use any fitting number/string to write on records.
Just Keep it legal to use in Number/BigInt constructor.
If the value fits in, then there is no worry on how to write values in records.
When you read 64bit signed/unsigned integers it will return a BigInt.
Remember to cast it to Number when value ranges in 64bit float scope.
Like:
ball1.posy = 100n; // BigInt variant of number 100 = BigInt(100) but `posy` needs a `Number`
ball1.id = '5'; // '5' is string (containing convertable value to bigint) but `id` type is `size <=> u64` which needs BigInt
const [posx, posy, id] = ball2
// posx => Number (f32)
// posy => Number (i16)
// id => BigInt (ptr64)
let id32 = Number(id)For reading values from records (loaded from buffers) you may need to convert BigInt values to Number.
I'd like to add support TS auto schema type detection for array of tupples input
Feel free to send any bug reports, ideas, recommandations, and problems to issues/discussions section.
Plz if you want consider to support me with USDT/ETC/BTC.
USDT-TRC20 (Tron net) : TDqx4hDYJWZ3fhFytrKgfxLMA9165mMX3Y
TRX-TRC10 : TDqx4hDYJWZ3fhFytrKgfxLMA9165mMX3Y
USDT-ERC20 (Etherium net) : 0x8893fb935a3fff6c4e12633ade1d4a0f1223d3f8
USDT-BSC (Binance net) : 0x8893fb935a3fff6c4e12633ade1d4a0f1223d3f8
ETH-ERC20 (Etherium net) : 0x8893fb935a3fff6c4e12633ade1d4a0f1223d3f8
BTC (Bitcoin net) : 1Cy7xEAC4YXRRJLiaoeRDXUGdhnwq1FsqQ