RAGFlow-0.11.0

Summary

• Adds AI search, aiming to provide PerplexityAI for each enterprise.
• TTS output based on FishAudio, Tongyi Qwen TTS.
• Supports using Postgres instead of MySQL to store RAGFlow metadata.
• Supports using S3/Azure blob as file storage.
• Supports LLMs above Anthropic, Voyage AI, Google Cloud.
• Supports Tencent Cloud ASR, to convert speech files into text.
• Adds several financial-related agent operators.
• Adds a medical assistant agent.
• Adds RAGFlow evaluation benchmark, currently supported:
  • ms_marco_v1.1:https://huggingface.co/datasets/microsoft/ms_marco
  • trivia_qa:https://huggingface.co/datasets/mandarjoshi/trivia_qa
  • miracl:<https://huggingface.co/datasets/miracl/mira

What's Changed

• fix: Filter out disabled values ​​from the llm options #2072 by @cike8899 in #2073
• create dataset by @Feiue in #2074
• prepare for sdk http api by @KevinHuSh in #2075
• Update quickstart.mdx by @writinwaters in #2084
• Update license by @JinHai-CN in #2086
• feat: Add FeedbackModal #2088 by @cike8899 in #2089
• add message id to conversions by @KevinHuSh in #2090
• add inferface for message thumbup by @KevinHuSh in #2091
• add thumb up api by @KevinHuSh in #2092
• VolcEngine SDK V3 adaptation by @yungongzi in #2082
• Provide detailed error information. by @JinHai-CN in #2093
• alter way of alerting info of empty task by @KevinHuSh in #2094
• fix callback function error by @KevinHuSh in #2096
• add support for TTS model by @hangters in #2095
• add prompt to message by @KevinHuSh in #2099
• Fix API key validation api/conversation by @guoyuhao2330 in #2100
• fix no tts issue by @KevinHuSh in #2101
• update groq llm by @hangters in #2103
• add support for Tencent Cloud ASR by @hangters in #2102
• add tts api by @hangters in #2107
• add fish audio zh and zh-traditional by @hangters in #2109
• feat: Fetch conversation list by @tanstack/react-query and displays error message that task_executor does not exist #2088 by @cike8899 in #2112
• create and update dataset by @Feiue in #2110
• feat: Pop-up prompt message after modifying the dialog settings #2088 by @cike8899 in #2114
• feat: Modify the modal style of creating an agent so that there are more templates in the field of view #2122 by @cike8899 in #2123
• feat: Align the cards on the Model Providers page #2111 by @cike8899 in #2125
• fix the max token of Tongyi-Qianwen text-embedding-v3 model to 8k by @hwzhuhao in #2118
• fix: Delete the model.ts file of chat #1306 by @cike8899 in #2129
• make task resumable by @KevinHuSh in #2132
• Fix Text2SQL by @guoyuhao2330 in #2131
• remove alter log by @KevinHuSh in #2140
• fix: build on ARM #2137 by @netandreus in #2139
• fix(graphrag): variable refernence error by @dearjane in #2136
• Fix exeSQL component output by @guoyuhao2330 in #2141
• feat: Submit Feedback #2088 by @cike8899 in #2134
• optimize text parser by @KevinHuSh in #2144
• Fix Logical operator by @guoyuhao2330 in #2143
• feat: Show prompt send to LLM for assistant answer #2098 by @cike8899 in #2142
• alter message id by @KevinHuSh in #2146
• feat: Send message with uuid #2088 by @cike8899 in #2149
• Update Dockerfile.arm by @yangboz in #2150
• add support for Anthropic by @hangters in #2148
• add retry count to task by @KevinHuSh in #2152
• fix delete message error by @KevinHuSh in #2153
• complete implementation of dataset SDK by @Feiue in #2147
• feat: Delete message by id #2088 by @cike8899 in #2155
• Fix the issue about No module named 'graspologic' #2157 by @LiuHao-1443 in #2158
• add support for Voyage AI by @hangters in #2159
• Fix some security vulnerabilities. by @hangters in #2160
• fix anthropic bug by @hangters in #2161
• re-generate for conversation by @KevinHuSh in #2165
• feat: Regenerate chat message #2088 by @cike8899 in #2166
• feat: Hide the delete button on the agent page #2088 by @cike8899 in #2167
• Complete DataSet SDK implementation by @Feiue in #2171
• feat: Select derived messages from backend #2088 by @cike8899 in #2176
• ignore when save image fail by @KevinHuSh in #2178
• fix: Copied API link error #2188 by @cike8899 in #2189
• add support for Google Cloud by @hangters in #2175
• feat: add api of tenant app by @chinamerp in #2177
• fix: After sending the message for the first time, the returned content is not streamed out #2067 #1832 by @cike8899 in #2191
• Fixed a broken link by @writinwaters in #2190
• Fix component PubMed by @guoyuhao2330 in #2192
• fix folder name suffix checking` by @KevinHuSh in #2194
• fix tts interface error by @hangters in #2197
• Fix component PubMed by @guoyuhao2330 in #2195
• fix re sub bug by @KevinHuSh in #2199
• Update api.md by @JobSmithManipulation in #2196
• feat: Play audio #2088 by @cike8899 in #2200
• feat: add tenant api of create & delete user by @chinamerp in #2204
• feat: Add tts Switch to chat configuration modal #2088 by @cike8899 in #2206
• Minor editorial updates by @writinwaters in #2207
• enable 3 char words to finegrind tokenize by @KevinHuSh in #2210
• Add Authorization checks by @Feiue in #2209
• make term similarity robust by @KevinHuSh in #2212
• fix error response disformat usage by @KevinHuSh in #2213
• fix zhipuCV bug by @hangters in #2215
• Add Authorization checks by @Feiue in #2218
• fix tts add bug by @hangters in #2224
• feat: Supports pronunciation while outputting text #2088 by @cike8899 in #2227
• add stream chat with TTS by @KevinHuSh in #2228
• Fix bug : bad escape \x at position xxx (line xx, column xx) in agent by @guoyuhao2330 in #2226
• Add Authorization checks by @Feiue in #2221
• feat: Comment out tts item #2088 by @cike8899 in #2232
• Add Authorization checks by @Feiue in #2235
• Fix component qweather by @guoyuhao2330 in #2240
• fix QWenSeq2txt bug by @KevinHuSh in #2245
• fix issue wrong agent prologue for api by @KevinHuSh in #2246
• Add component yahoo finance by @guoyuhao2330 in #2244
• feat: Add SearchPage #2247 by @cike8899 in #2248
• Bump aiohttp from 3.9.4 to 3.10.2 by @dependabot in #2254
• Bump cryptography from 42.0.5 to 43.0.1 by @dependabot in #2253
• Bump nltk from 3.8.1 to 3.9 by @dependabot in #2250
• Bump flask-cors from 4.0.1 to 5.0.0 by @dependabot in #2251
• feat: Make agent template support HTML #1842 by @cike8899 in #2259
• feat: Add Sider to SearchPage #2247 by @cike8899 in #2262
• update agent\templates\medical_consultation.json by @JobSmithManipulation in #2260
• feat: Set the global scroll bar style #2247 by @cike8899 in #2265
• SDK for Assistant by @Feiue in #2266
• feat: Add sidebar to SearchPage #2247 by @cike8899 in #2267
• Fine tweaks to template descriptions by @writinwaters in #2264
• Add component WenCai by @guoyuhao2330 in #2269
• feat: send question with retrieval api #2247 by @cike8899 in #2272
• Add component Jin10 by @guoyuhao2330 in #2271
• Fix graphrag : "role" user by @guoyuhao2330 in #2273
• Fix agent retrieval nothing by @wingjson in #2283
• feat: Search for the answers you want based on the selected knowledge base #2247 by @cike8899 in #2287
• Added kibana by @yuzhichang in #2286
• feat: Added md.svg #345 by @cike8899 in #2289
• Add component TuShare by @guoyuhao2330 in #2288
• Add component AkShare by @guoyuhao2330 in #2290
• feat: Fetch mind map in search page #2247 by @cike8899 in #2292
• Storage: Support the s3, azure blob as the object storage of ragflow. by @baifachuan in #2278
• add a lib by @KevinHuSh in #2306
• feat: Show chat tab #2247 by @cike8899 in #2307
• Updated component descriptions by @writinwaters in #2293
• reduce interval of task executor heart beat by @KevinHuSh in #2308
• add support for TongyiQwen tts by @hangters in #2311
• fix SILICONFLOW rerank error by @hangters in #2313
• add elapsed time of conversation by @KevinHuSh in #2316
• fix azure-openai add bug by @hangters in #2314
• Update .env For CVE-2024-37288 by @fashioncj in #2318
• Doc updates for newly updates by @KevinHuSh in #2317
• fix minio error by @KevinHuSh in #2321
• feat: Click on the relevant question tag to continue searching for answers #2247 by @cike8899 in #2320
• Update .env by @guoyuhao2330 in #2319
• feat: Hide search tab #2247 by @cike8899 in #2322
• fix error for files from filemanager by @KevinHuSh in #2323
• updates dead link by @KevinHuSh in #2324
• feat: The search box is displayed globally when the page is loaded for the first time #2247 by @cike8899 in #2325
• SDK for session by @Feiue in #2312
• Add benchmark ndcg@10 by @guoyuhao2330 in #2326
• feat: Add RetrievalDocuments to SearchPage #2247 by @cike8899 in #2327
• minor by @writinwaters in #2328
• Added a brief introduction to Agentic RAG by @writinwaters in #2331
• Fixed a docusaurus display issue by @writinwaters in #2340
• feat: Dynamically change the background image on the search homepage every day #2247 by @cike8899 in #2338
• add model types to factories API by @KevinHuSh in #2341
• add default model types by @KevinHuSh in #2342
• fix: Make markdown support line breaks #2315 by @cike8899 in #2343
• fix categorize error by @KevinHuSh in #2348
• fix: 123.60.95.134 redirect attack #2350 by @cike8899 in #2352
• fix: empty or contains only empty strings. by @Chunshan-Theta in #2347
• SDK for session by @Feiue in #2354
• fix SILICONFLOW embedding error by @hangters in #2363
• Update templates:Text2sql and DB Description by @guoyuhao2330 in #2361
• fix OpenRouter add bug and the way to add OpenRouter model by @hangters in #2364
• feat: Proxy the api address to the local nginx address #2350 by @cike8899 in #2366
• feat: Catch errors in getting mindmap #2247 by @cike8899 in #2368
• fix json error by @KevinHuSh in #2372
• feat: Retrieval chunks by page #2247 by @cike8899 in #2373
• add search TAB backend api by @KevinHuSh in #2375
• feat: Catching errors with IndentedTree #2247 by @cike8899 in #2380
• update sdk document by @JobSmithManipulation in #2374
• feat: Cover the entire search page with a background image #2247 by @cike8899 in #2381
• Add a default value for do_refer in Dialog by @Feiue in #2383
• Common: Support postgreSQL database as the metadata db. by @baifachuan in #2357
• feat: Use Tree to display the knowledge base list on the search page #2247 by @cike8899 in #2385
• debug backend API for TAB 'search' by @KevinHuSh in #2389
• feat: Add component WenCai #1739 by @cike8899 in #2388
• feat: Add component AkShare #1739 by @cike8899 in #2390
• Fix volcengine yfinance confliction by @guoyuhao2330 in #2386
• Add RAGFlow benchmark by @guoyuhao2330 in #2387
• feat: Wrap the searched chunk with a Popover #2247 by @cike8899 in #2398
• feat: Click on the chunk on the search page to locate the corresponding file location #2247 by @cike8899 in #2399
• fix pip install error by @KevinHuSh in #2407
• feat: Use Spin to wrap the chunk list on the search page #2247 by @cike8899 in #2409
• feat: Modify the style of the answer card on the search page #2247 by @cike8899 in #2412
• feat: When the mindmap data is empty, it will not be displayed on the search page #2247 by @cike8899 in #2414
• add updates for README by @KevinHuSh in #2413
• make highlight friendly to English by @KevinHuSh in #2417
• fix empty children in mindmap by @KevinHuSh in #2418
• Initial draft of Create a General-purpose chatbot by @writinwaters in #2411
• boost highlight performace by @KevinHuSh in #2419
• minor by @writinwaters in #2422
• fix parsing spaces in russian language PDFs (#1987) by @Hyperb0t in #2427
• update sdk document and chunk by @JobSmithManipulation in #2421
• refine the warning message for rewrite component by @KevinHuSh in #2429
• fix batch size error for qianwen embedding by @KevinHuSh in #2431
• feat: Display mindmap in drawer #2247 by @cike8899 in #2430
• feat: Add html to the description text of the parsing method general #336 by @cike8899 in #2432
• feat: Supports text output and sound output #1877 by @cike8899 in #2436
• fix duplicate function name by @KevinHuSh in #2437
• prepare document for release by @KevinHuSh in #2438
• feat: After the voice in the new conversation window is played, jump to the tab of the conversation #1877 by @cike8899 in #2440
• Update category.json by @toronan in #2442
• Update and rename agentic_rag_introduction.md to agent_introduction.md by @toronan in #2443
• feat: If the tts model is not set, the Text to Speech switch is not allowed to be turned on #1877 by @cike8899 in #2446

New Contributors

• @Feiue made their first contribution in #2074
• @netandreus made their first contribution in #2139
• @dearjane made their first contribution in #2136
• @yangboz made their first contribution in #2150
• @LiuHao-1443 made their first contribution in #2158
• @JobSmithManipulation made their first contribution in #2196
• @yuzhichang made their first contribution in #2286
• @baifachuan made their first contribution in #2278
• @fashioncj made their first contribution in #2318
• @Hyperb0t made their first contribution in #2427
• @toronan made their first contribution in #2442

Full Changelog: v0.10.0...v0.11.0