Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
Co-authored-by: Nicolas Patry <[email protected]>
  • Loading branch information
akx and Narsil authored Aug 3, 2023
1 parent f036926 commit 30dbf94
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ with safe_open("model.safetensors", framework="pt", device="cpu") as f:
- 8 bytes: `N`, an unsigned little-endian 64-bit integer, containing the size of the header
- N bytes: a JSON UTF-8 string representing the header.
- The header data MUST begin with a `{` character (0x7B).
- The header data MAY be trailing padded with whitespace (any bytes 0x09, 0x0A, 0x0D, 0x20).
- The header data MAY be trailing padded with whitespace (0x20).
- The header is a dict like `{"TENSOR_NAME": {"dtype": "F16", "shape": [1, 16, 256], "data_offsets": [BEGIN, END]}, "NEXT_TENSOR_NAME": {...}, ...}`,
- `data_offsets` point to the tensor data relative to the beginning of the byte buffer (i.e. not an absolute position in the file),
with `BEGIN` as the starting offset and `END` as the one-past offset (so total tensor byte size = `END - BEGIN`).
Expand Down

0 comments on commit 30dbf94

Please sign in to comment.