Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a set of sample files to test other implementations. #84

Closed
wants to merge 1 commit into from

Conversation

KOLANICH
Copy link
Contributor

No description provided.

@Narsil
Copy link
Collaborator

Narsil commented Nov 17, 2022

Not sure I understand the purpose of this PR.
I think this is the script you used to do some testing right ?

Maybe we should create real tests ?
Also I don't think we should commit actual tensor files. there are some deserialization examples with hardcoded values directly in Python to verify the capacity to deserialize some types.

@KOLANICH
Copy link
Contributor Author

These are just a few sample files. Ones who want to create an own impl of safetensors (I have written a Kaitai Struct-based impl for npy a few years ago, I guess it'd be easy to reuse its code to create an impl of safetensors, the hardest part there was the json header) can use them to test and debug an own impl. In order to do it he doesn't have to to build safetensors python package, he can just download the precreated files.

The script just generates the files. It also generates license files for reuse tool (but they are not commited into the repo, we usereuse tool in Kaitai Struct own repo of test files to track the copyright status of files, since not all the files are generated by us, but some can be fetched from other places).

@Narsil
Copy link
Collaborator

Narsil commented Nov 17, 2022

Thank you for this work, but I don't think this belongs within the repository.

Just like the gists I post sometimes to showcase how to load without this library directly in PyTorch, they are helpful, but they do not belong here. The code here should only be about manipulating the files or helping testing this particular implementation works.

@KOLANICH KOLANICH closed this Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants