Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gguf.md: add sharding to naming convention #826

Merged

Conversation

mofosyne
Copy link
Contributor

As per #822 (review) suggestion, adding support for sharding in naming convention.

Was a bit worried about if -of- is going to be annoying to parse, but I think it's not too much to ask programmers to check if XXXXX-of-XXXXX exist first and delete that before processing the rest from right to left by splitting by -. The requirement for 5 digits and -of- is pretty distinctive anyway.

@ggerganov this is a PR to add this idea on behalf of @mishig25 and based on checking hf it appears to be a common enough convention already e.g. vicuna.gguf

@mofosyne mofosyne force-pushed the 820-add-shard-to-naming-convention branch 2 times, most recently from 68fcf99 to 5f75cab Compare May 17, 2024 14:06
docs/gguf.md Outdated Show resolved Hide resolved
docs/gguf.md Outdated Show resolved Hide resolved
@mofosyne mofosyne force-pushed the 820-add-shard-to-naming-convention branch from 5f75cab to c0f25bf Compare May 17, 2024 15:32
@mofosyne mofosyne requested a review from mishig25 May 17, 2024 15:36
Copy link

@mishig25 mishig25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀

@mofosyne mofosyne force-pushed the 820-add-shard-to-naming-convention branch from 1fd0058 to 0b3cbe4 Compare May 17, 2024 16:25
@mofosyne
Copy link
Contributor Author

mofosyne commented May 17, 2024

Minor commit to address Justine's observation that

One of the mistakes I made in my upload scripts, was sometimes not choosing exactly what the original project called itself, casing and all. So as a result, I've been having a hard time consolidating my upload scripts.

So recommended that the model, version and expert count be derived from gguf metadata. This consistency should hopefully reduce surprises when creating upload scripts by making the filename generation a bit more repeatable.

Copy link

@julien-c julien-c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love it!

@mofosyne
Copy link
Contributor Author

@ggerganov ready to merge

@mofosyne mofosyne force-pushed the 820-add-shard-to-naming-convention branch from 1ef8674 to 25d5656 Compare May 18, 2024 03:51
@mofosyne
Copy link
Contributor Author

Still ready for review @ggerganov , just added a regex example and tighten up the description.

@mofosyne mofosyne requested review from mishig25 and julien-c May 18, 2024 11:39
@ggerganov ggerganov merged commit 0cbb7c0 into ggerganov:master May 19, 2024
@mofosyne
Copy link
Contributor Author

@mishig25 it's now merged in, so you can use it now :)

@mofosyne mofosyne deleted the 820-add-shard-to-naming-convention branch May 19, 2024 08:58
@mishig25
Copy link

@mofosyne Awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants