Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.60.0.dev4
What's Changed
- Azure AI Foundry - Deepseek R1 by @elabbarw in #8188
- fix(main.py): fix passing openrouter specific params by @krrishdholakia in #8184
- Complete o3 model support by @krrishdholakia in #8183
- Easier user onboarding via SSO by @krrishdholakia in #8187
- LiteLLM Minor Fixes & Improvements (01/16/2025) - p2 by @krrishdholakia in #7828
- Added deprecation date for gemini-1.5 models by @yurchik11 in #8210
- docs: Updating the available VoyageAI models in the docs by @fzowl in #8215
- build: ui updates by @krrishdholakia in #8206
- Fix tokens for deepseek by @SmartManoj in #8207
- (UI Fixes for add new model flow) by @ishaan-jaff in #8216
Full Changelog: v1.60.0.dev2...v1.60.0.dev4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.0.dev4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 150.0 | 179.79463683736907 | 6.359486247668494 | 0.0 | 1900 | 0 | 123.9115270000184 | 3798.7273850000065 |
| Aggregated | Passed ✅ | 150.0 | 179.79463683736907 | 6.359486247668494 | 0.0 | 1900 | 0 | 123.9115270000184 | 3798.7273850000065 |
Contributors
SmartManoj, yurchik11, and 4 other contributors
Assets 4
v1.60.0.dev2
What's Changed
- Control Model Access by IDP 'groups' by @krrishdholakia in #8164
- build(schema.prisma): add new
sso_user_idto LiteLLM_UserTable by @krrishdholakia in #8167 - Litellm dev contributor prs 01 31 2025 by @krrishdholakia in #8168
- Improved O3 + Azure O3 support by @krrishdholakia in #8181
- test: add more unit testing for team member endpoints by @krrishdholakia in #8170
- Add azure/deepseek-r1 by @Klohto in #8177
- [Bug Fix] -
/vertex_ai/was not detected as llm_api_route on pass through butvertex-aiwas by @ishaan-jaff in #8186 - (UI + SpendLogs) - Store SpendLogs in UTC Timezone, Fix filtering logs by start/end time by @ishaan-jaff in #8190
New Contributors
Full Changelog: v1.60.0...v1.60.0.dev2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.0.dev2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 160.0 | 179.3387644765704 | 6.274867330705683 | 0.0 | 1878 | 0 | 134.8906900000202 | 3148.732781000035 |
| Aggregated | Passed ✅ | 160.0 | 179.3387644765704 | 6.274867330705683 | 0.0 | 1878 | 0 | 134.8906900000202 | 3148.732781000035 |
Contributors
Klohto, krrishdholakia, and ishaan-jaff
Assets 4
v1.60.0.dev1
What's Changed
- Control Model Access by IDP 'groups' by @krrishdholakia in #8164
- build(schema.prisma): add new
sso_user_idto LiteLLM_UserTable by @krrishdholakia in #8167 - Litellm dev contributor prs 01 31 2025 by @krrishdholakia in #8168
- Improved O3 + Azure O3 support by @krrishdholakia in #8181
- test: add more unit testing for team member endpoints by @krrishdholakia in #8170
- Add azure/deepseek-r1 by @Klohto in #8177
New Contributors
Full Changelog: v1.60.0...v1.60.0.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.0.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 170.0 | 193.07171769197802 | 6.24812141882662 | 0.0 | 1870 | 0 | 149.06627900001013 | 846.9972659999883 |
| Aggregated | Passed ✅ | 170.0 | 193.07171769197802 | 6.24812141882662 | 0.0 | 1870 | 0 | 149.06627900001013 | 846.9972659999883 |
Contributors
Klohto and krrishdholakia
Assets 4
v1.60.0
What's Changed
Important Changes between v1.50.xx to 1.60.0
def async_log_stream_eventanddef log_stream_eventno longer supported forCustomLoggershttps://docs.litellm.ai/docs/observability/custom_callback. If you want to log stream events usedef async_log_success_eventanddef log_success_eventfor logging success stream events
Known Issues
🚨 Detected issue with Langfuse Logging when Langfuse credentials are stored in DB
- Adding gemini-2.0-flash-thinking-exp-01-21 by @marcoaleixo in #8089
- add groq/deepseek-r1-distill-llama-70b by @miraclebakelaser in #8078
- (UI) Fix SpendLogs page - truncate
bedrockmodels + showend_userby @ishaan-jaff in #8118 - UI Fixes - Newly created key does not display on the View Key Page + Updated the validator to allow model editing when
keyTeam.team_alias === "Default Team"by @ishaan-jaff in #8122 - (Refactor / QA) - Use
LoggingCallbackManagerto append callbacks and ensure no duplicate callbacks are added by @ishaan-jaff in #8112 - (UI) fix adding Vertex Models by @ishaan-jaff in #8129
- Fix json_mode parameter propagation in OpenAILikeChatHandler by @miraclebakelaser in #8133
- Doc updates - add key rotations to docs by @krrishdholakia in #8136
- Enforce default_on guardrails always run + expose new
litellm.disable_no_log_paramparam by @krrishdholakia in #8134 - Doc updates + management endpoint fixes by @krrishdholakia in #8138
- New stable release - release notes by @krrishdholakia in #8148
- FEATURE: OpenAI o3-mini by @ventz in #8151
- build: fix model cost map with o3 model pricing by @krrishdholakia in #8153
- (Fixes) OpenAI Streaming Token Counting + Fixes usage track when
litellm.turn_off_message_logging=Trueby @ishaan-jaff in #8156 - (UI) Allow adding custom pricing when adding new model by @ishaan-jaff in #8165
- (Feat) add bedrock/deepseek custom import models by @ishaan-jaff in #8132
- Adding Azure OpenAI o3-mini costs & specs by @yigitkonur in #8166
- Adjust model pricing metadata by @yurchik11 in #8147
New Contributors
- @marcoaleixo made their first contribution in #8089
- @yigitkonur made their first contribution in #8166
Full Changelog: v1.59.10...v1.60.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 281.07272626532927 | 6.158354312051399 | 0.0 | 1843 | 0 | 215.79772499995897 | 3928.489000000013 |
| Aggregated | Passed ✅ | 240.0 | 281.07272626532927 | 6.158354312051399 | 0.0 | 1843 | 0 | 215.79772499995897 | 3928.489000000013 |
Contributors
ventz, yigitkonur, and 5 other contributors
Assets 4
v1.59.10
What's Changed
- (UI) - View Logs Page - Refinement by @ishaan-jaff in #8087
- (Feat) pass through vertex - allow using credentials defined on litellm router for vertex pass through by @ishaan-jaff in #8100
- (UI) Allow using a model / credentials for pass through routes by @ishaan-jaff in #8099
- ui - fix chat ui tab sending
modelparam by @ishaan-jaff in #8105 - Litellm dev 01 29 2025 p1 by @krrishdholakia in #8097
- Support new
bedrock/converse_like/<model>route by @krrishdholakia in #8102 - feat(databricks/chat/transformation.py): add tools and 'tool_choice' param support by @krrishdholakia in #8076
Full Changelog: v1.59.9...v1.59.10
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.10
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 239.24647793068146 | 6.21745665443628 | 0.00334092243655899 | 1861 | 1 | 73.25327600000264 | 3903.3159660000083 |
| Aggregated | Passed ✅ | 210.0 | 239.24647793068146 | 6.21745665443628 | 0.00334092243655899 | 1861 | 1 | 73.25327600000264 | 3903.3159660000083 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.59.8-stable
Full Changelog: v1.57.8-stable...v1.59.8-stable
Known Issues
🚨 Detected issue with Langfuse Logging when Langfuse credentials are stored in DB
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.59.8-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 260.0 | 291.2207591958183 | 6.075260080470321 | 0.0 | 1818 | 0 | 223.10552599998346 | 3813.1267819999266 |
| Aggregated | Passed ✅ | 260.0 | 291.2207591958183 | 6.075260080470321 | 0.0 | 1818 | 0 | 223.10552599998346 | 3813.1267819999266 |
v1.59.9
What's Changed
- Fix custom pricing - separate provider info from model info by @krrishdholakia in #7990
- Litellm dev 01 25 2025 p4 by @krrishdholakia in #8006
- (UI) - Adding new models enhancement - show provider logo by @ishaan-jaff in #8033
- (UI enhancement) - allow onboarding wildcard models on UI by @ishaan-jaff in #8034
- add openrouter/deepseek/deepseek-r1 by @paul-gauthier in #8038
- (UI) - allow assigning wildcard models to a team / key by @ishaan-jaff in #8041
- Add smolagents by @aymeric-roucher in #8026
- (UI) fixes to add model flow by @ishaan-jaff in #8043
- github - run stale issue/pr bot by @ishaan-jaff in #8045
- (doc) Add nvidia as provider by @raspawar in #8023
- feat(handle_jwt.py): initial commit adding custom RBAC support on jwt… by @krrishdholakia in #8037
- fix(utils.py): handle failed hf tokenizer request during calls by @krrishdholakia in #8032
- Bedrock document processing fixes by @krrishdholakia in #8005
- Fix bedrock model pricing + add unit test using bedrock pricing api by @krrishdholakia in #7978
- Add openai
metadataparam preview support + newx-litellm-timeoutrequest header by @krrishdholakia in #8047 - (beta ui - spend logs view fixes & Improvements 1) by @ishaan-jaff in #8062
- (fix) - proxy reliability, ensure duplicate callbacks are not added to proxy by @ishaan-jaff in #8067
- (UI) Fixes for Adding model page - keep existing page as default, have 2nd tab for wildcard models by @ishaan-jaff in #8073
New Contributors
- @aymeric-roucher made their first contribution in #8026
- @raspawar made their first contribution in #8023
Full Changelog: v1.59.8...v1.59.9
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.9
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 270.0 | 301.01550717582927 | 6.14169679840119 | 0.0 | 1837 | 0 | 234.85362500002793 | 3027.238808999982 |
| Aggregated | Failed ❌ | 270.0 | 301.01550717582927 | 6.14169679840119 | 0.0 | 1837 | 0 | 234.85362500002793 | 3027.238808999982 |
Contributors
krrishdholakia, ishaan-jaff, and 3 other contributors
Assets 4
v1.59.8-dev1
What's Changed
- Fix custom pricing - separate provider info from model info by @krrishdholakia in #7990
- Litellm dev 01 25 2025 p4 by @krrishdholakia in #8006
- (UI) - Adding new models enhancement - show provider logo by @ishaan-jaff in #8033
- (UI enhancement) - allow onboarding wildcard models on UI by @ishaan-jaff in #8034
- add openrouter/deepseek/deepseek-r1 by @paul-gauthier in #8038
- (UI) - allow assigning wildcard models to a team / key by @ishaan-jaff in #8041
- Add smolagents by @aymeric-roucher in #8026
- (UI) fixes to add model flow by @ishaan-jaff in #8043
- github - run stale issue/pr bot by @ishaan-jaff in #8045
- (doc) Add nvidia as provider by @raspawar in #8023
- feat(handle_jwt.py): initial commit adding custom RBAC support on jwt… by @krrishdholakia in #8037
- fix(utils.py): handle failed hf tokenizer request during calls by @krrishdholakia in #8032
New Contributors
- @aymeric-roucher made their first contribution in #8026
- @raspawar made their first contribution in #8023
Full Changelog: v1.59.8...v1.59.8-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.8-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 230.0 | 253.74562668371757 | 6.073890684010945 | 0.0 | 1818 | 0 | 198.74819999995452 | 1957.5085989999934 |
| Aggregated | Passed ✅ | 230.0 | 253.74562668371757 | 6.073890684010945 | 0.0 | 1818 | 0 | 198.74819999995452 | 1957.5085989999934 |
Contributors
krrishdholakia, ishaan-jaff, and 3 other contributors
Assets 4
v1.59.8
What's Changed
- refactor: cleanup dead codeblock by @krrishdholakia in #7936
- add type annotation for litellm.api_base (#7980) by @krrishdholakia in #7994
- (QA / testing) - Add unit testing for key model access checks by @ishaan-jaff in #7999
- (Prometheus) - emit key budget metrics on startup by @ishaan-jaff in #8002
- (Feat) set guardrails per team by @ishaan-jaff in #7993
- Supported nested json schema on anthropic calls via proxy + fix langfuse sync sdk issues by @krrishdholakia in #8003
- Bug fix - [Bug]: If you create a key tied to a user that does not belong to a team, and then edit the key to add it to a team (the user is still not a part of a team), using that key results in an unexpected error by @ishaan-jaff in #8008
- (QA / testing) - Add e2e tests for key model access auth checks by @ishaan-jaff in #8000
- (Fix) langfuse - setting
LANGFUSE_FLUSH_INTERVALby @ishaan-jaff in #8007
Full Changelog: v1.59.7...v1.59.8
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.8
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 280.0 | 325.48398318207154 | 6.003526201462839 | 0.0 | 1796 | 0 | 234.56590200004257 | 3690.442290999954 |
| Aggregated | Failed ❌ | 280.0 | 325.48398318207154 | 6.003526201462839 | 0.0 | 1796 | 0 | 234.56590200004257 | 3690.442290999954 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.59.7
What's Changed
- Add datadog health check support + fix bedrock converse cost tracking w/ region name specified by @krrishdholakia in #7958
- Retry for replicate completion response of status=processing (#7901) by @krrishdholakia in #7965
- Ollama ssl verify = False + Spend Logs reliability fixes by @krrishdholakia in #7931
- (Feat) - allow setting
default_onguardrails by @ishaan-jaff in #7973 - (Testing) e2e testing for team budget enforcement checks by @ishaan-jaff in #7988
- (UI) - Usage page show days when spend is 0 and round spend figures on charts to 2 sig figs by @ishaan-jaff in #7991
- (Feat) - Add GCS Pub/Sub Logging integration for sending DB
SpendLogsto BigQuery by @ishaan-jaff in #7976 - fix(spend_tracking_utils.py): revert api key pass through fix by @krrishdholakia in #7977
- Ensure base_model cost tracking works across all endpoints by @krrishdholakia in #7989
- (UI) Allow admin to expose teams for joining by @krrishdholakia in #7992
Full Changelog: v1.59.6...v1.59.7
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.59.7
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 260.0 | 294.5630730660492 | 6.1254059494010225 | 0.0 | 1832 | 0 | 231.04980300001898 | 2728.9633709999634 |
| Aggregated | Passed ✅ | 260.0 | 294.5630730660492 | 6.1254059494010225 | 0.0 | 1832 | 0 | 231.04980300001898 | 2728.9633709999634 |
Contributors
krrishdholakia and ishaan-jaff