'load llm infomation from a json file and add support for OpenRouter' (…

…infiniflow#1533) ### What problem does this PR solve? infiniflow#1467 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Zhedong Cen <[email protected]>
Halfknow · Jul 16, 2024 · b560417 · b560417
1 parent 6ccc923
commit b560417
Show file tree

Hide file tree

Showing 8 changed files with 2,000 additions and 904 deletions.
diff --git a/api/apps/llm_app.py b/api/apps/llm_app.py
@@ -57,8 +57,8 @@ def set_api_key():
             mdl = ChatModel[factory](
                 req["api_key"], llm.llm_name, base_url=req.get("base_url"))
             try:
-                m, tc = mdl.chat(None, [{"role": "user", "content": "Hello! How are you doing!"}], {
-                                 "temperature": 0.9})
+                m, tc = mdl.chat(None, [{"role": "user", "content": "Hello! How are you doing!"}], 
+                                 {"temperature": 0.9,'max_tokens':50})
                 if not tc:
                     raise Exception(m)
             except Exception as e: