From ec308220654dd40571739f685658cc4c82edb592 Mon Sep 17 00:00:00 2001 From: Michael Clifford Date: Wed, 24 Apr 2024 16:43:24 -0400 Subject: [PATCH] update suggested model in chat and summary docs Signed-off-by: Michael Clifford --- recipes/natural_language_processing/chatbot/README.md | 10 ++++------ .../natural_language_processing/summarizer/README.md | 10 ++++------ 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/recipes/natural_language_processing/chatbot/README.md b/recipes/natural_language_processing/chatbot/README.md index 48e4a755..0b217f72 100644 --- a/recipes/natural_language_processing/chatbot/README.md +++ b/recipes/natural_language_processing/chatbot/README.md @@ -62,19 +62,17 @@ podman pod rm chatbot ## Download a model -If you are just getting started, we recommend using [Mistral-7B-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1). This is a well +If you are just getting started, we recommend using [granite-7b-lab](https://huggingface.co/instructlab/granite-7b-lab). This is a well performant mid-sized model with an apache-2.0 license. In order to use it with our Model Service we need it converted and quantized into the [GGUF format](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md). There are a number of -ways to get a GGUF version of Mistral-7B, but the simplest is to download a pre-converted one from -[huggingface.co](https://huggingface.co) here: https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF. - -There are a number of options for quantization level, but we recommend `Q4_K_M`. +ways to get a GGUF version of granite-7b-lab, but the simplest is to download a pre-converted one from +[huggingface.co](https://huggingface.co) here: https://huggingface.co/instructlab/granite-7b-lab-GGUF. The recommended model can be downloaded using the code snippet below: ```bash cd ../../../models -curl -sLO https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf +curl -sLO https://huggingface.co/instructlab/granite-7b-lab-GGUF/resolve/main/granite-7b-lab-Q4_K_M.gguf cd ../recipes/natural_language_processing/chatbot ``` diff --git a/recipes/natural_language_processing/summarizer/README.md b/recipes/natural_language_processing/summarizer/README.md index fd98bb7d..2c4b87ce 100644 --- a/recipes/natural_language_processing/summarizer/README.md +++ b/recipes/natural_language_processing/summarizer/README.md @@ -62,19 +62,17 @@ podman pod rm summarizer ## Download a model -If you are just getting started, we recommend using [Mistral-7B-Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1). This is a well +If you are just getting started, we recommend using [granite-7b-lab](https://huggingface.co/instructlab/granite-7b-lab). This is a well performant mid-sized model with an apache-2.0 license. In order to use it with our Model Service we need it converted and quantized into the [GGUF format](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md). There are a number of -ways to get a GGUF version of Mistral-7B, but the simplest is to download a pre-converted one from -[huggingface.co](https://huggingface.co) here: https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF. - -There are a number of options for quantization level, but we recommend `Q4_K_M`. +ways to get a GGUF version of granite-7b-lab, but the simplest is to download a pre-converted one from +[huggingface.co](https://huggingface.co) here: https://huggingface.co/instructlab/granite-7b-lab-GGUF/blob/main/granite-7b-lab-Q4_K_M.gguf. The recommended model can be downloaded using the code snippet below: ```bash cd ../../../models -curl -sLO https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf +curl -sLO https://huggingface.co/instructlab/granite-7b-lab-GGUF/resolve/main/granite-7b-lab-Q4_K_M.gguf cd ../recipes/natural_language_processing/summarizer ```