Prometheus - custom metrics support + other improvements

[krrishdholakia]

Title

• feat(prometheus.py): add 'request_model' to 'litellm_input_tokens_metric'
• feat(prometheus.py): emit requested model in 'litellm_output_tokens_metric'
• feat(prometheus.py): refactor all tags to support custom metadata tags

Custom Metrics Tutorial (BETA)

1. Define the custom metrics in the config.yaml

model_list:
  - model_name: openai/gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: ["prometheus"]
  custom_prometheus_metadata_labels: ["metadata.foo", "metadata.bar"]

2. Make a request with the custom metadata labels

curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <LITELLM_API_KEY>' \
-d '{
    "model": "openai/gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What'\''s in this image?"
          }
        ]
      }
    ],
    "max_tokens": 300,
    "metadata": {
        "foo": "hello world"
    }
}'

3. Check your /metrics endpoint for the custom metrics

... "tag": "hello world" ...

Relevant issues

Adds more detail on prometheus usage metrics
Allows tracking spend by task/use-case/prompt template/etc.

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locally

If UI changes, send a screenshot/GIF of working UI fixes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 1, 2025 3:42pm