Codegen25-7B & Codegen2-4B Fail to download

[ghthor]

Describe the bug

The Codegen25-7B[1] & Codegen2-4B[2] model fail to download using tabby download.

[1] https://huggingface.co/TabbyML/Codegen25-7B
[2] https://huggingface.co/TabbyML/Codegen2-4B

Command being run with error output

✦ ❯ docker run -it -v /opt/tabby/data/:/data --rm tabbyml/tabby download --model TabbyML/Codegen25-7B
2023-10-05T13:03:21.265769Z ERROR tabby::download: crates/tabby/src/download.rs:21: Failed to fetch model 'TabbyML/Codegen25-7B' due to 'Failed to fetch url https://huggingface.co/TabbyML/Codegen25-7B/resolve/main/tabby.json'

Thu Oct  5 09:03:22 2023 exit 1 ❌ ERROR took 13s

✦ ❯ docker run -it -v /opt/tabby/data/:/data --rm tabbyml/tabby download --model TabbyML/Codegen2-4B
2023-10-05T13:04:03.735169Z ERROR tabby::download: crates/tabby/src/download.rs:21: Failed to fetch model 'TabbyML/Codegen2-4B' due to 'Failed to fetch url https://huggingface.co/TabbyML/Codegen2-4B/resolve/main/ggml/q8_0.gguf'

Thu Oct  5 09:04:04 2023 exit 1 ❌ ERROR took 10s

Information about your GPU
Please provide output of nvidia-smi

Additional context

These models are my focus as they have support for more languages then python. I previously downloaded Codegen2-4B successfully and that is the model I'm currently running, but was hoping to try Codegen25-7B.

[ghthor]

Codegen2-4B has by far the best completions for my workload of any of the models I've tried.

[wsxiaoys]

May I recommend considering StarCoder-3B and StarCoder-7B from our Models Directory? They are well-maintained, provide infilling support, and have been trained on a considerably larger range of languages (approximately 80).

The failure to download Codegen2-4B seems to be a bug; however, we will address it in an upcoming patch release.

[ghthor]

I will give StarCoder 7B a longer try, but I wasn’t initially impressed.

…

On Thu, Oct 5, 2023 at 12:28 Meng Zhang ***@***.***> wrote: May I recommend considering StarCoder-3B and StarCoder-7B from our Models Directory <https://tabby.tabbyml.com/docs/models/>? They are well-maintained, provide infilling support, and have been trained on a considerably larger range of languages (approximately 80). The failure to download Codegen2-4B seems to be a bug; however, we will address it in an upcoming patch release. — Reply to this email directly, view it on GitHub <#511 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABHEKWFESDD4YERQ6FFGHLX53N4DAVCNFSM6AAAAAA5UHVMICVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBZGI3DONBXGI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[ghthor]

I will give StarCoder 7B a longer try, but I wasn’t initially impressed.

Unfortunately I don't have enough GPU Memory to run StarCoder 7B, and I'm getting better completion suggestions with Codegen2-4B vs StarCoder-3B

[wsxiaoys]

I'm getting better completion suggestions with Codegen2-4B vs StarCoder-3B

Understood. We should enhance the documentation at https://github.com/TabbyML/tabby/blob/main/MODEL_SPEC.md to facilitate the community in creating their own model implementations.

[ghthor]

Understood. We should enhance the documentation at https://github.com/TabbyML/tabby/blob/main/MODEL_SPEC.md to facilitate the community in creating their own model implementations.

That would be really awesome. I've already looked into it some but wasn't able to make any notable progress.