Add support for Mixtral 8x7B-Dolphin 2.5 and WizardLM (#997)

* Add support for Mixtral 8x7B - Dolphin 2.5 * Add Support for WizardLM * Add new LLMs - Add support for Mixtral 8x7B - Dolphin 2.5 - Add support for WizardLM - Update README.md * Remove support of WizardLM 7B -UNCENSORED * Remove support of WizardLM 7B -UNCENSORED --------- Co-authored-by: Juan Calderon-Perez <[email protected]>
serge-chat · Dec 19, 2023 · 98bdb9f · 98bdb9f
1 parent 70b2db3
commit 98bdb9f
Show file tree

Hide file tree

Showing 2 changed files with 51 additions and 1 deletion.
diff --git a/README.md b/README.md
@@ -66,7 +66,7 @@ Instructions for setting up Serge on Kubernetes can be found in the [wiki](https
 | **Medalpaca** | 13B |
 | **Meditron** | 7B, 7B-Chat, 70B |
 | **Mistral** | 7B-V0.1, 7B-Instruct-v0.2, 7B-OpenOrca |
-| **Mixtral** | 8x7B-v0.1, 8x7B-Instruct-v0.1
+| **Mixtral** | 8x7B-v0.1, 8x7B-Dophin-2.5, 8x7B-Instruct-v0.1 |
 | **Neural-Chat** | 7B-v3.3 | 
 | **Notus** | 7B-v1 |
 | **OpenChat** | 7B-v3.5-1012 |
@@ -76,6 +76,7 @@ Instructions for setting up Serge on Kubernetes can be found in the [wiki](https
 | **PsyMedRP** | 13B-v1, 20B-v1 |
 | **Starling LM** | 7B-Alpha |
 | **Vicuna** | 7B-v1.5, 13B-v1.5, 33B-v1.3, 33B-Coder |
+| **WizardLM** | 7B-v1.0, 13B-v1.2, 70B-v1.0 |
 | **Zephyr** | 3B, 7B-Alpha, 7B-Beta |
 
 Additional models can be requested by opening a GitHub issue.

diff --git a/api/src/serge/data/models.json b/api/src/serge/data/models.json
@@ -371,6 +371,17 @@
                         "disk_space": 4368438912.0
                     }
                 ]
+            },
+            {
+                "name": "Mixtral-8X7B-Dolphin-2_5",
+                "repo": "TheBloke/dolphin-2.5-mixtral-8x7b-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "dolphin-2.5-mixtral-8x7b.Q4_K_M.gguf",
+                        "disk_space": 26441544544.0
+                    }
+                ]
             },            
             {
                 "name": "Mistral-7B-Instruct-v0_2",
@@ -671,6 +682,44 @@
             }             
         ]
     },
+    {   
+        "name": "WizardLM",
+        "models": [
+            {
+                "name": "WizardLM-7B-v1_0",
+                "repo": "TheBloke/wizardLM-7B-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "wizardLM-7B.Q4_K_M.gguf",
+                        "disk_space": 4081009920.0
+                    }
+                ]
+            },          
+            {
+                "name": "WizardLM-13B-v1_2",
+                "repo": "TheBloke/WizardLM-13B-V1.2-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "wizardlm-13b-v1.2.Q4_K_M.gguf",
+                        "disk_space": 7865956224.0
+                    }
+                ]
+            },
+            {
+                "name": "WizardLM-70B-v1_0",
+                "repo": "TheBloke/WizardLM-70B-V1.0-GGUF",
+                "files": [
+                    {
+                        "name": "q4_K_M",
+                        "filename": "wizardlm-70b-v1.0.Q4_K_M.gguf",
+                        "disk_space": 41422921728.0
+                    }
+                ]
+            }            
+        ]
+    },    
     {
         "name": "Zephyr",
         "models": [