fix(reasoning): compatible with the return data of the reasoning mod…

[kuloud]

Title

compatible with the return data of the reasoning model

Relevant issues

Fixes #8193

Type

🐛 Bug Fix

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locally

If UI changes, send a screenshot/GIF of working UI fixes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 5, 2025 5:17pm

[kuloud]

@krrishdholakia Do you think this PR needs to be merged? there aren't many customization scenarios for reasoning_content, and adding a global param doesn't seem necessary for now.

[qtnx]

Waiting for this one to be merged to have reasoning output in OpenRouter. Quite long...

[krrishdholakia]

Hi @kuloud can you share a screenshot of this passing your test?

Once done, it should be good to merge

[kuloud]

Hi @kuloud can you share a screenshot of this passing your test?

Once done, it should be good to merge

I found that I had some misunderstandings about the implementation within LiteLLM. I tried it and although Delta did handle the mapping of reasoning_content, I don't quite understand where in streaming_handler.py it is processed and the result is ignored.

INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': 'Sure', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     root:test_streaming.py:4104 chunk ----: ModelResponseStream(id='chatcmpl-200e1aaf-b64a-4c3a-bc3c-9e31ca6cfadf', created=1738997695, model='deepseek/deepseek-r1', object='chat.completion.chunk', system_fingerprint=None, choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(provider_specific_fields=None, content='Sure', role='assistant', function_call=None, tool_calls=None, audio=None), logprobs=None)], stream_options=None)
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '!', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     root:test_streaming.py:4104 chunk ----: ModelResponseStream(id='chatcmpl-200e1aaf-b64a-4c3a-bc3c-9e31ca6cfadf', created=1738997695, model='deepseek/deepseek-r1', object='chat.completion.chunk', system_fingerprint=None, choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(provider_specific_fields=None, content='!', role=None, function_call=None, tool_calls=None, audio=None), logprobs=None)], stream_options=None)
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': ' Here', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     root:test_streaming.py:4104 chunk ----: ModelResponseStream(id='chatcmpl-200e1aaf-b64a-4c3a-bc3c-9e31ca6cfadf', created=1738997695, model='deepseek/deepseek-r1', object='chat.completion.chunk', system_fingerprint=None, choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(provider_specific_fields=None, content=' Here', role=None, function_call=None, tool_calls=None, audio=None), logprobs=None)], stream_options=None)
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': "'s", 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     root:test_streaming.py:4104 chunk ----: ModelResponseStream(id='chatcmpl-200e1aaf-b64a-4c3a-bc3c-9e31ca6cfadf', created=1738997695, model='deepseek/deepseek-r1', object='chat.completion.chunk', system_fingerprint=None, choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(provider_specific_fields=None, content="'s", role=None, function_call=None, tool_calls=None, audio=None), logprobs=None)], stream_options=None)
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': '', 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0, 'tool_use': None}
INFO     LiteLLM:streaming_handler.py:871 1--------{'text': ' a', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}
INFO     root:test_streaming.py:4104 chunk ----: ModelResponseStream(id='chatcmpl-200e1aaf-b64a-4c3a-bc3c-9e31ca6cfadf', created=1738997695, model='deepseek/deepseek-r1', object='chat.completion.chunk', system_fingerprint=None, choices=[StreamingChoices(finish_reason=None, index=0, delta=Delta(provider_specific_fields=None, content=' a', role=None, function_call=None, tool_calls=None, audio=None), logprobs=None)], stream_options=None)

When processing the reasoning information, it reaches is_async_iterable as shown in the figure, but the chunk information is:

{'text': ' a', 'tool_use': None, 'is_finished': False, 'finish_reason': '', 'usage': None, 'index': 0}

[kuloud]

This PR doesn't solve the actual problem. It still requires your support @krrishdholakia

[kuloud]

This PR doesn't solve the actual problem. It still requires your support @krrishdholakia

I fixed the issue, and maybe I will make an new pr today or tomottow