Does not operate on Llama-3 or Phi-3.

[ymuichiro]

Thank you for the wonderful project. I was able to run it by building locally and adding the coi-serviceworker mentioned in the README.md and #5.

However, when using a different model, the following behavior occurs:

1. Download the model.
2. Complete the model download.
3. Freeze without any logs appearing.

Is there a specification, for example, that does not support models above a certain size?

Example model

const models = [
   // ok
  "https://huggingface.co/stabilityai/stablelm-2-zephyr-1_6b/resolve/main/stablelm-2-zephyr-1_6b-Q4_1.gguf",
  // ng
  "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf/resolve/main/Phi-3-mini-4k-instruct-q4.gguf", // 2.32GB
  // ng
  "https://huggingface.co/QuantFactory/Meta-Llama-3-8B-GGUF/resolve/main/Meta-Llama-3-8B.Q4_K_M.gguf", // 4.92GB
];

[flatsiedatsie]

The maximum file size is 2Gb. Have a look at Wllama for a way around this.

[ymuichiro]

@flatsiedatsie
Thank you! I was able to split it. However, a similar issue has arisen with another issue, so I will proceed with resolving that issue.

split-tool
https://github.com/ggerganov/llama.cpp/tree/master/examples/gguf-split

new issue
ngxson/wllama#12

[flatsiedatsie]

Let me guess: memory issues :-)