FEAT: Zephyr

README.md

@@ -30,7 +30,8 @@ potential of cutting-edge AI models.
 - Support grammar-based sampling for ggml models: [#525](https://github.com/xorbitsai/inference/pull/525)
 - Incorporate vLLM: [#445](https://github.com/xorbitsai/inference/pull/445)
 ### New Models
-- Built-in support for [chatglm3](https://huggingface.co/THUDM/chatglm3-6b)
+- Built-in support for [zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha) and [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta): [#597](https://github.com/xorbitsai/inference/pull/597) 
+- Built-in support for [chatglm3](https://huggingface.co/THUDM/chatglm3-6b): [#587](https://github.com/xorbitsai/inference/pull/587)
 - Built-in support for [mistral-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) and [mistral-instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1): [#510](https://github.com/xorbitsai/inference/pull/510)
 ### Integrations
 - [Dify](https://docs.dify.ai/advanced/model-configuration/xinference): an LLMOps platform that enables developers (and even non-developers) to quickly build useful applications based on large language models, ensuring they are visual, operable, and improvable.

README_zh_CN.md

@@ -27,7 +27,8 @@ Xorbits Inference（Xinference）是一个性能强大且功能全面的分布
 - 投机采样: [#509](https://github.com/xorbitsai/inference/pull/509)
 - 引入 vLLM: [#445](https://github.com/xorbitsai/inference/pull/445)
 ### 新模型
-- 内置 [chatglm3](https://huggingface.co/THUDM/chatglm3-6b)
+- 内置 [zephyr-7b-alpha](https://huggingface.co/HuggingFaceH4/zephyr-7b-alpha) 与 [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta): [#597](https://github.com/xorbitsai/inference/pull/597)
+- 内置 [chatglm3](https://huggingface.co/THUDM/chatglm3-6b): [#587](https://github.com/xorbitsai/inference/pull/587)
 - 内置 [mistral-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) 与 [mistral-instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1): [#510](https://github.com/xorbitsai/inference/pull/510)
 ### 集成
 - [Dify](https://docs.dify.ai/advanced/model-configuration/xinference): 一个涵盖了大型语言模型开发、部署、维护和优化的 LLMOps 平台。

doc/source/locale/ja_JP/LC_MESSAGES/examples/ai_podcast.po

@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2023-10-16 10:33+0800\n"
+"POT-Creation-Date: 2023-11-01 10:48+0800\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: ja_JP\n"
@@ -163,6 +163,12 @@ msgid ""
 msgstr ""
 
 #: ../../source/examples/ai_podcast.rst:79
-msgid "AI_Podcast_ZH (Chinese Version)"
+msgid ""
+"`AI_Podcast_ZH "
+"<https://github.com/xorbitsai/inference/blob/main/examples/AI_podcast_ZH.py>`"
+" (Chinese Version)"
 msgstr ""
 
+#~ msgid "AI_Podcast_ZH (Chinese Version)"
+#~ msgstr ""
+

doc/source/locale/ja_JP/LC_MESSAGES/examples/chatbot.po

@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2023-10-16 10:33+0800\n"
+"POT-Creation-Date: 2023-11-01 10:48+0800\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: ja_JP\n"
@@ -20,7 +20,7 @@ msgstr ""
 "Generated-By: Babel 2.12.1\n"
 
 #: ../../source/examples/chatbot.rst:5
-msgid "Example: chatbot 🤖️"
+msgid "Example: CLI chatbot 🤖️"
 msgstr ""
 
 #: ../../source/examples/chatbot.rst:7
@@ -30,7 +30,7 @@ msgstr ""
 #: ../../source/examples/chatbot.rst:9
 msgid ""
 "Demonstrate how to interact with Xinference to play with LLM chat "
-"functionality with an AI agent 💻"
+"functionality with an AI agent in command line💻"
 msgstr ""
 
 #: ../../source/examples/chatbot.rst:11
@@ -96,3 +96,12 @@ msgid ""
 "<https://github.com/RayJi01/Xprobe_inference/blob/main/examples/chat.py>`_"
 msgstr ""
 
+#~ msgid "Example: chatbot 🤖️"
+#~ msgstr ""
+
+#~ msgid ""
+#~ "Demonstrate how to interact with "
+#~ "Xinference to play with LLM chat "
+#~ "functionality with an AI agent 💻"
+#~ msgstr ""
+

doc/source/locale/ja_JP/LC_MESSAGES/examples/gradio_chatinterface.po

@@ -0,0 +1,96 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) 2023, Xorbits Inc.
+# This file is distributed under the same license as the Xinference package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, 2023.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: Xinference \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2023-11-01 10:48+0800\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language: ja_JP\n"
+"Language-Team: ja_JP <[email protected]>\n"
+"Plural-Forms: nplurals=1; plural=0;\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.12.1\n"
+
+#: ../../source/examples/gradio_chatinterface.rst:5
+msgid "Example: Gradio ChatInterface🤗"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:7
+msgid "**Description**:"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:9
+msgid ""
+"This example showcases how to build a chatbot with 120 lines of code with"
+" Gradio ChatInterface and Xinference local LLM"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:11
+msgid "**Used Technology**:"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:13
+msgid ""
+"@ `Xinference <https://github.com/xorbitsai/inference>`_ as a LLM model "
+"hosting service"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:15
+msgid ""
+"@ `Gradio <https://github.com/gradio-app/gradio>`_ as a web interface for"
+" the chatbot"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:17
+msgid "**Detailed Explanation on the Demo Functionality** :"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:19
+msgid ""
+"Parse user-provided command line arguments to capture essential model "
+"parameters such as model name, size, format, and quantization."
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:21
+msgid ""
+"Establish a connection to the Xinference framework and deploy the "
+"specified model, ensuring it's ready for real-time interactions."
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:23
+msgid ""
+"Implement helper functions (flatten and to_chat) to efficiently handle "
+"and store chat interactions, ensuring the model has context for "
+"generating relevant responses."
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:25
+msgid ""
+"Set up an interactive chat interface using Gradio, allowing users to "
+"communicate with the model in a user-friendly environment."
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:27
+msgid ""
+"Activate the Gradio web interface, enabling users to start their chat "
+"sessions and receive model-generated responses based on their queries."
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:29
+msgid "**Source Code** :"
+msgstr ""
+
+#: ../../source/examples/gradio_chatinterface.rst:30
+msgid ""
+"`Gradio ChatInterface "
+"<https://github.com/xorbitsai/inference/blob/main/examples/gradio_chatinterface.py>`_"
+msgstr ""
+

doc/source/locale/ja_JP/LC_MESSAGES/examples/index.po

@@ -8,7 +8,7 @@ msgid ""
 msgstr ""
 "Project-Id-Version: Xinference \n"
 "Report-Msgid-Bugs-To: \n"
-"POT-Creation-Date: 2023-10-16 10:33+0800\n"
+"POT-Creation-Date: 2023-11-01 10:48+0800\n"
 "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
 "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
 "Language: ja_JP\n"
@@ -19,7 +19,107 @@ msgstr ""
 "Content-Transfer-Encoding: 8bit\n"
 "Generated-By: Babel 2.12.1\n"
 
-#: ../../source/examples/index.rst:5
+#: ../../source/examples/index.rst:5 ../../source/examples/index.rst:19
 msgid "Examples"
 msgstr ""
 
+#: ../../source/examples/index.rst:16
+msgid ""
+"Here you can find examples and resources to learn about how to use "
+"Xinference."
+msgstr ""
+
+#: ../../source/examples/index.rst:21
+msgid "End-to-end examples of using Xinference for various tasks:"
+msgstr ""
+
+#: ../../source/examples/index.rst:23
+msgid "`Voice Conversations with AI Agents on M2 Max <ai_podcast.html>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:25
+msgid "`Interacting with LLM Models: A Command-Line Example <chatbot.html>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:27
+msgid ""
+"`Interacting with LLM Models: A Gradio ChatInterface Example "
+"<gradio_chatinterface.html>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:29
+msgid "`PDF Chatbot with Local LLM and Embeddings <pdf_chatbot.html>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:31
+msgid ""
+"If you come across other examples in your own workflows we encourage you "
+"to contribute a `PR <https://github.com/xorbitsai/inference/pulls>`_!"
+msgstr ""
+
+#: ../../source/examples/index.rst:35
+msgid "Tutorials"
+msgstr ""
+
+#: ../../source/examples/index.rst:37
+msgid ""
+"The following tutorials cover the basics of using Xinference in different"
+" scenarios:"
+msgstr ""
+
+#: ../../source/examples/index.rst:39
+msgid ""
+"`Build a QA Application with Xinference and LangChain "
+"<https://github.com/RayJi01/Xprobe_inference/blob/main/examples/LangChain_QA.ipynb>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:41
+msgid ""
+"`Using Xinference local LLMs within LlamaIndex <https://gpt-"
+"index.readthedocs.io/en/stable/examples/llm/XinferenceLocalDeployment.html>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:43
+msgid ""
+"`[Chinese] 如何让 Chatbox 接入开源大模型，实现免费聊天 "
+"<https://twitter.com/benn_huang/status/1701420060240490785>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:45
+msgid ""
+"`[Chinese] 摆脱 OpenAI 依赖，8 分钟教你用开源生态构建全栈 AI 应用 "
+"<https://mp.weixin.qq.com/s/cXBC0dikldNiGwOwPuJfUQ>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:47
+msgid ""
+"`[Chinese] 使用全套开源工具构建 LLM 应用实战 "
+"<https://mp.weixin.qq.com/s/regqYkF0cNDQIdOkOeyeXQ>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:51
+msgid "Third-Party Library Integrations"
+msgstr ""
+
+#: ../../source/examples/index.rst:53
+msgid ""
+"Xinference is designed to seamlessly integrate and deploy open-sourced AI"
+" models, so we want to incorporate support for mainstream toolkits in the"
+" AI landscape. Xinference can be used with the following third-party "
+"libraries:"
+msgstr ""
+
+#: ../../source/examples/index.rst:56
+msgid ""
+"LangChain `Text Embedding Models "
+"<https://python.langchain.com/docs/integrations/text_embedding/xinference>`_"
+" and `LLMs "
+"<https://python.langchain.com/docs/integrations/llms/xinference>`_"
+msgstr ""
+
+#: ../../source/examples/index.rst:58
+msgid ""
+"`LlamaIndex Xinference LLM "
+"<https://docs.llamaindex.ai/en/stable/api_reference/llms/xinference.html>`_"
+msgstr ""
+

doc/source/locale/ja_JP/LC_MESSAGES/examples/pdf_chatbot.po

@@ -0,0 +1,95 @@
+# SOME DESCRIPTIVE TITLE.
+# Copyright (C) 2023, Xorbits Inc.
+# This file is distributed under the same license as the Xinference package.
+# FIRST AUTHOR <EMAIL@ADDRESS>, 2023.
+#
+#, fuzzy
+msgid ""
+msgstr ""
+"Project-Id-Version: Xinference \n"
+"Report-Msgid-Bugs-To: \n"
+"POT-Creation-Date: 2023-11-01 10:48+0800\n"
+"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
+"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
+"Language: ja_JP\n"
+"Language-Team: ja_JP <[email protected]>\n"
+"Plural-Forms: nplurals=1; plural=0;\n"
+"MIME-Version: 1.0\n"
+"Content-Type: text/plain; charset=utf-8\n"
+"Content-Transfer-Encoding: 8bit\n"
+"Generated-By: Babel 2.12.1\n"
+
+#: ../../source/examples/pdf_chatbot.rst:5
+msgid "Example: PDF Chatbot📚"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:7
+msgid "**Description**:"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:9
+msgid ""
+"This example showcases how to build a PDF chatbot with local LLM and "
+"Embedding models"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:11
+msgid "**Used Technology**:"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:13
+msgid ""
+"@ `Xinference <https://github.com/xorbitsai/inference>`_ as a LLM model "
+"hosting service"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:15
+msgid ""
+"@ `LlamaIndex <https://github.com/run-llama/llama_index>`_ for "
+"orchestrating the entire RAG pipeline"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:17
+msgid "@ `Streamlit <https://streamlit.io/>`_ for interactive UI"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:19
+msgid "**Detailed Explanation on the Demo Functionality** :"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:21
+msgid ""
+"Crafted a Dockerfile to simplify the process and ensure easy "
+"reproducibility."
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:23
+msgid "Set up models with Xinference and expose two ports for accessing them."
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:25
+msgid ""
+"Leverage Streamlit for seamless file uploads and interactive "
+"communication with the chat engine."
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:27
+msgid "5x faster doc embedding than OpenAI's API."
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:29
+msgid ""
+"Leveraging the power of GGML to offload models to the GPU, ensuring swift"
+" acceleration. Less long waits for returns."
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:31
+msgid "**Source Code** :"
+msgstr ""
+
+#: ../../source/examples/pdf_chatbot.rst:32
+msgid ""
+"`PDF Chatbot <https://github.com/onesuper/PDF-Chatbot-Local-LLM-"
+"Embeddings>`_"
+msgstr ""
+