🔍 Observability	🕸️ Agent Tracing	💬 Prompt Management
📊 Evaluations	📚 Datasets	🎛️ Fine-tuning

Docs • Discord • Roadmap • Changelog • Bug reports

See Helicone in Action! (Free)

Helicone is the all-in-one, open-source LLM developer platform

• 🔌 Integrate: One-line of code to log all your requests to OpenAI, Anthropic, LangChain, Gemini, TogetherAI, LlamaIndex, LiteLLM, OpenRouter, and more
• 📊 Observe: Inspect and debug traces & sessions for agents, chatbots, document processing pipelines, and more
• 📈 Analyze: Track metrics like cost, latency, quality, and more. Export to PostHog in one-line for custom dashboards
• 🎮 Playground: Rapidly test and iterate on prompts, sessions and traces in our UI
• 🧠 Prompt Management: Version and experiment with prompts using production data. Your prompts remain under your control, always accessible.
• 🔍 Evaluate: Automatically run evals on traces or sessions using the latest platforms: LastMile or Ragas (more coming soon)
• 🎛️ Fine-tune: Fine-tune with one of our fine-tuning partners: OpenPipe or Autonomi (more coming soon)
• 🛜 Gateway: Caching, custom rate limits, LLM security, and more with our gateway
• 🛡️ Enterprise Ready: SOC 2 and GDPR compliant

🎁 Generous monthly free tier (100k requests/month) - No credit card required!

Quick Start ⚡️ One line of code

1. Get your write-only API key by signing up here.

2. Update only the baseURL in your code:

   import OpenAI from "openai";
   
   const openai = new OpenAI({
     apiKey: process.env.OPENAI_API_KEY,
     baseURL: `https://oai.helicone.ai/v1/${process.env.HELICONE_API_KEY}`,
   });

or - use headers for more secure environments

import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  baseURL: `https://oai.helicone.ai/v1`,
  defaultHeaders: {
   "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,
  },
});

3. 🎉 You're all set! View your logs at Helicone.

This quick start uses Helicone Cloud with OpenAI. For other providers or self-hosted options, see below.

Get Started For Free

Helicone Cloud (Recommended)

The fastest and most reliable way to get started with Helicone. Get started for free at Helicone US or Helicone EU. Your first 100k requests are free every month, after which you'll pay based on usage. Try our demo to see Helicone in action!

Integrations: View our supported integrations.

Latency Concerns: Helicone's Cloud offering is deployed on Cloudflare workers and ensures the lowest latency (~10ms) add-on to your API requests. View our latency benchmarks.

Self-Hosting Open Source LLM Observability with Helicone

Docker

Helicone is simple to self-host and update. To get started locally, just use our docker-compose file.

Pre-Request:

• Copy the shared directory to the valhalla directory
• Create a valhalla folder in the valhalla directory and put /valhalla/jawn in it

# Clone the repository
git clone https://github.com/Helicone/helicone.git
cd docker
cp .env.example .env

# Start the services
docker compose up

Helm

For Enterprise workloads, we also have a production-ready Helm chart available. To access, contact us at [email protected].

Manual (Not Recommended)

Manual deployment is not recommended. Please use Docker or Helm. If you must, follow the instructions here.

Architecture

Helicone is comprised of five services:

• Web: Frontend Platform (NextJS)
• Worker: Proxy Logging (Cloudflare Workers)
• Jawn: Dedicated Server for serving collecting logs (Express + Tsoa)
• Supabase: Application Database and Auth
• ClickHouse: Analytics Database
• Minio: Object Storage for logs.

LLM Observability Integrations

Main Integrations

Integration	Supports	Description
Generic Gateway	Python, Node.js, Python w/package, LangChain JS, LangChain, cURL	Flexible integration method for various LLM providers
Async Logging (OpenLLMetry)	JS/TS, Python	Asynchronous logging for multiple LLM platforms
OpenAI	JS/TS, Python	-
Azure OpenAI	JS/TS, Python	-
Anthropic	JS/TS, Python	-
Ollama	JS/TS	Run and use large language models locally
AWS Bedrock	JS/TS	-
Gemini API	JS/TS	-
Gemini Vertex AI	JS/TS	Gemini models on Google Cloud's Vertex AI
Vercel AI	JS/TS	AI SDK for building AI-powered applications
Anyscale	JS/TS, Python	-
TogetherAI	JS/TS, Python	-
Hyperbolic	JS/TS, Python	High-performance AI inference platform
Groq	JS/TS, Python	High-performance models
DeepInfra	JS/TS, Python	Serverless AI inference for various models
OpenRouter	JS/TS, Python	Unified API for multiple AI models
LiteLLM	JS/TS, Python	Proxy server supporting multiple LLM providers
Fireworks AI	JS/TS, Python	Fast inference API for open-source LLMs

Supported Frameworks

Framework	Supports	Description
LangChain	JS/TS, Python	-
LlamaIndex	Python	Framework for building LLM-powered data applications
CrewAI	-	Framework for orchestrating role-playing AI agents
Big-AGI	JS/TS	Generative AI suite
ModelFusion	JS/TS	Abstraction layer for integrating AI models into JavaScript and TypeScript applications

Other Integrations

Integration	Description
PostHog	Product analytics platform. Build custom dashboards.
RAGAS	Evaluation framework for retrieval-augmented generation
Open WebUI	Web interface for interacting with local LLMs
MetaGPT	Multi-agent framework
Open Devin	AI software engineer
Mem0 EmbedChain	Framework for building RAG applications
Dify	LLMOps platform for AI-native application development

This list may be out of date. Don't see your provider or framework? Check out the latest integrations in our docs. If not found there, request a new integration by contacting [email protected].

Community 🌍

Learn this repo with Greptile

learnthisrepo.com/helicone |

Contributing

We ❤️ our contributors! We warmly welcome contributions for documentation, integrations, costs, and feature requests.

• If you have an idea for how Helicone can be better, create a GitHub issue or vote on the roadmap
• Update costs instructions in costs/README.md
• Join discord to ask questions

License

Helicone is licensed under the Apache v2.0 License.

Additional Resources

• Data Management: Manage and export your Helicone data with our API.

  • Guides: ETL, Request Exporting

• Data Ownership: Learn about Data Ownership and Autonomy

For more information, visit our documentation.

Contributors