Litellm dev 01 31 2025 p2 (#8164)

* docs(token_auth.md): clarify title

* refactor(handle_jwt.py): add jwt auth manager + refactor to handle groups

allows user to call model if user belongs to group with model access

* refactor(handle_jwt.py): refactor to first check if service call then check user call

* feat(handle_jwt.py): new `enforce_team_access` param

only allows user to call model if a team they belong to has model access

allows controlling user model access by team

* fix(handle_jwt.py): fix error string, remove unecessary param

* docs(token_auth.md): add controlling model access for jwt tokens via teams to docs

* test: fix tests post refactor

* fix: fix linting errors

* fix: fix linting error

* test: fix import error

docs/my-website/docs/proxy/token_auth.md

@@ -1,7 +1,7 @@
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
-# SSO - JWT-based Auth 
+# OIDC - JWT-based Auth 
 
 Use JWT's to auth admins / projects into the proxy.
 
@@ -156,50 +156,75 @@ scope: ["litellm-proxy-admin",...]
 scope: "litellm-proxy-admin ..."
 ```
 
-## Enforce Role-Based Access Control (RBAC)
+## Control Model Access with Roles
 
 Reject a JWT token if it's valid but doesn't have the required scopes / fields.
 
 Only tokens which with valid Admin (`admin_jwt_scope`), User (`user_id_jwt_field`), Team (`team_id_jwt_field`) are allowed.
 
 ```yaml
 general_settings:
-  master_key: sk-1234
-  enable_jwt_auth: True
+  enable_jwt_auth: True 
   litellm_jwtauth:
-    admin_jwt_scope: "litellm_proxy_endpoints_access"
-    admin_allowed_routes:
-      - openai_routes
-      - info_routes
-    public_key_ttl: 600
-    enforce_rbac: true # 👈 Enforce RBAC
+    user_roles_jwt_field: "resource_access.litellm-test-client-id.roles"
+    user_allowed_roles: ["basic_user"] # roles that map to an 'internal_user' role on LiteLLM 
+    enforce_rbac: true # if true, will check if the user has the correct role to access the model + endpoint
+  
+  role_permissions: # control what models + endpointsare allowed for each role
+    - role: internal_user
+      models: ["anthropic-claude"]
 ```
 
-Expected Scope in JWT: 
+**[Architecture Diagram (Control Model Access)](./jwt_auth_arch)**
+
+## Control model access with Teams
+
+
+1. Specify the JWT field that contains the team ids, that the user belongs to. 
+
+```yaml
+general_settings:
+  master_key: sk-1234
+  litellm_jwtauth:
+    user_id_jwt_field: "sub"
+    team_ids_jwt_field: "groups" 
+```
 
+This is assuming your token looks like this:
 ```
 {
-  "scope": "litellm_proxy_endpoints_access"
+  ...,
+  "sub": "my-unique-user",
+  "groups": ["team_id_1", "team_id_2"]
 }
 ```
 
-### Control Model Access 
+2. Create the teams on LiteLLM 
 
-```yaml
-general_settings:
-  enable_jwt_auth: True 
-  litellm_jwtauth:
-    user_roles_jwt_field: "resource_access.litellm-test-client-id.roles"
-    user_allowed_roles: ["basic_user"] # roles that map to an 'internal_user' role on LiteLLM 
-    enforce_rbac: true # if true, will check if the user has the correct role to access the model + endpoint
-
-  role_permissions: # control what models + endpointsare allowed for each role
-    - role: internal_user
-      models: ["anthropic-claude"]
+```bash
+curl -X POST '<PROXY_BASE_URL>/team/new' \
+-H 'Authorization: Bearer <PROXY_MASTER_KEY>' \
+-H 'Content-Type: application/json' \
+-D '{
+    "team_alias": "team_1",
+    "team_id": "team_id_1" # 👈 MUST BE THE SAME AS THE SSO GROUP ID
+}'
 ```
 
+3. Test the flow
+
+SSO for UI: [**See Walkthrough**](https://www.loom.com/share/8959be458edf41fd85937452c29a33f3?sid=7ebd6d37-569a-4023-866e-e0cde67cb23e)
+
+OIDC Auth for API: [**See Walkthrough**](https://www.loom.com/share/00fe2deab59a426183a46b1e2b522200?sid=4ed6d497-ead6-47f9-80c0-ca1c4b6b4814)
+
+
+### Flow
+
+- Validate if user id is in the DB (LiteLLM_UserTable)
+- Validate if any of the groups are in the DB (LiteLLM_TeamTable)
+- Validate if any group has model access
+- If all checks pass, allow the request
 
-**[Architecture Diagram (Control Model Access)](./jwt_auth_arch)**
 
 ## Advanced - Allowed Routes 
 

litellm/proxy/_new_secret_config.yaml

@@ -3,6 +3,10 @@ model_list:
     litellm_params:
       model: gpt-3.5-turbo
       rpm: 3
+  - model_name: o3-mini
+    litellm_params:
+      model: o3-mini
+      rpm: 3
   - model_name: anthropic-claude
     litellm_params:
       model: claude-3-5-haiku-20241022
@@ -18,4 +22,11 @@ model_list:
 
 litellm_settings:
   callbacks: ["langsmith"]
-  disable_no_log_param: true
+  disable_no_log_param: true
+
+general_settings:
+  enable_jwt_auth: True
+  litellm_jwtauth:
+    user_id_jwt_field: "sub"
+    user_email_jwt_field: "email"
+    team_ids_jwt_field: "groups" # 👈 CAN BE ANY FIELD