Bodhi App allows you to run Open Source LLMs locally. It utilizes the Huggingface ecosystem for accessing open-source LLM weights and information and is powered by llama.cpp.
While many apps that help you run LLMs locally are targeted at technical users, Bodhi App is designed with both technical and non-technical users in mind.
For technical users, it provides OpenAI-compatible chat completions and models API endpoints. It includes comprehensive API documentation following OpenAPI standards and features a built-in SwaggerUI that allows developers to explore and test all API endpoints live.
For non-technical users, it comes with a built-in Chat UI that is quick to start and easy to understand. Users can quickly get started with open-source models and adjust various settings to suit their needs. The app also enables users to discover, explore, and download new open-source models that fit their requirements and are compatible with their local hardware.
- Built-in Chat UI: Enjoy an intuitive, responsive chat interface with real-time streaming, markdown support, and customizable settings
- Model Management: Download and manage GGUF model files directly from HuggingFace
- API Token Management: Securely generate and manage API tokens for external integrations
- Dynamic App Settings: Easily adjust application parameters (like execution variant and idle timeout) on the fly
- Responsive Design: A fully adaptive layout that works seamlessly across desktop and mobile devices
- Robust Error Handling: Comprehensive error logging and troubleshooting guides to help quickly identify and resolve issues
Bodhi App is currently released only for the Mac platform. You can install it either by downloading the release from the GitHub release page or using Homebrew.
Bodhi App hosts its external cask at BodhiSearch/homebrew-apps. Install Bodhi App using this command:
brew install --cask BodhiSearch/apps/bodhiOnce installed, launch Bodhi App.app from the /Applications folder. You should see the Bodhi App icon in your system tray. Launch the homepage from the system tray menu by selecting Open Homepage.
Download the latest release for your platform from the Releases page.
Unzip and move Bodhi.app to your /Applications folder, then launch it. You should see the Bodhi App icon in your system tray. Launch the homepage from the system tray menu by selecting Open Homepage.
On first launch, Bodhi App starts with a setup flow. Follow this process to configure and install Bodhi App for your local machine and get started.
Bodhi App comes with built-in documentation:
- User Guide: Access at http://localhost:1135/docs/
- Technical Documentation: Available as OpenAPI Swagger UI at http://localhost:1135/swagger-ui/
{width=600px}
(Open up a pull request on README.md to include community integrations)