Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix custom pricing - separate provider info from model info #7990

Merged
merged 8 commits into from
Jan 26, 2025
Merged

Fix custom pricing - separate provider info from model info #7990

merged 8 commits into from
Jan 26, 2025

Conversation

krrishdholakia
Copy link
Contributor

@krrishdholakia krrishdholakia commented Jan 25, 2025 •
edited
Loading

Title

Fix custom pricing

Relevant issues

Fixes issues where custom pricing was not working when set under litellm params

Issue caused by addition of provider info inside model info (provider_key_info).

[TODO] investigate why this was not being correctly raised an issue when using mock_response

Type

🐛 Bug Fix
🧹 Refactoring

Changes

  • separate provider info from model info

[REQUIRED] Testing - Attach a screenshot of any new tests passing locally

If UI changes, send a screenshot/GIF of working UI fixes

@krrishdholakia
fix(utils.py): initial commit fixing custom cost tracking
5a4c304 
refactors out provider specific model info from `get_model_info` - this was causing custom costs to be registered incorrectly
@krrishdholakia
fix(utils.py): cleanup _supports_factory to check provider info, if…
aebdb58 
… model info is None

some providers support features like vision across all models
@vercel Vercel
Copy link

vercel bot commented Jan 25, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
litellm ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 25, 2025 6:37am

github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 25, 2025
View reviewed changes
litellm/cost_calculator.py
@@ -559,6 +559,10 @@
base_model=base_model,
)

verbose_logger.debug(
f"completion_response _select_model_name_for_cost_calc: {model}"

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expression logs
sensitive data (secret)
Loading
as clear text.
This expressi

Copilot Autofix AI 15 days ago

To fix the problem, we should avoid logging sensitive information directly. Instead, we can log a sanitized version of the model variable or avoid logging it altogether if it contains sensitive data. The best way to fix this without changing existing functionality is to sanitize the model variable before logging it. We can create a helper function to sanitize the model variable by removing or masking any sensitive information.

Suggested changeset 1
litellm/cost_calculator.py

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/litellm/cost_calculator.py b/litellm/cost_calculator.py
--- a/litellm/cost_calculator.py
+++ b/litellm/cost_calculator.py
@@ -72,2 +72,8 @@
 
+def sanitize_model(model: Optional[str]) -> Optional[str]:
+    if model is None:
+        return None
+    # Add logic to sanitize the model string, e.g., mask sensitive parts
+    # For simplicity, let's mask any part that looks like an API key
+    return model.replace("os.environ/", "****")
 
@@ -562,4 +568,5 @@
 
+        sanitized_model = sanitize_model(model)
         verbose_logger.debug(
-            f"completion_response _select_model_name_for_cost_calc: {model}"
+            f"completion_response _select_model_name_for_cost_calc: {sanitized_model}"
         )
EOF
@@ -72,2 +72,8 @@

def sanitize_model(model: Optional[str]) -> Optional[str]:
if model is None:
return None
# Add logic to sanitize the model string, e.g., mask sensitive parts
# For simplicity, let's mask any part that looks like an API key
return model.replace("os.environ/", "****")

@@ -562,4 +568,5 @@

sanitized_model = sanitize_model(model)
verbose_logger.debug(
f"completion_response _select_model_name_for_cost_calc: {model}"
f"completion_response _select_model_name_for_cost_calc: {sanitized_model}"
)
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
@codecov Codecov
Copy link

codecov bot commented Jan 25, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@krrishdholakia krrishdholakia merged commit 03eef5a into main Jan 26, 2025
32 of 33 checks passed
@krrishdholakia krrishdholakia deleted the litellm_dev_01_24_2025_p3 branch January 26, 2025 05:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@krrishdholakia