ChatOllama stream method raises warn_deprecated NotImplementedError

[v-byte-cpu]

System Info

langchain version: v0.0.352
python version: 3.11

Hi there! After that PR #14713 was merged, I started getting errors in stream() method:

File .../lib/python3.11/site-packages/langchain_core/_api/deprecation.py:295, in warn_deprecated(since, message, name, alternative, pending, obj_type, addendum, removal)
    293 if not removal:
    294     removal = f"in {removal}" if removal else "within ?? minor releases"
--> 295     raise NotImplementedError(
    296         f"Need to determine which default deprecation schedule to use. "
    297         f"{removal}"
    298     )
    299 else:
    300     removal = f"in {removal}"

NotImplementedError: Need to determine which default deprecation schedule to use. within ?? minor releases

I guess this decorator must have a pending=True argument.

Who can help?

@hwchase17 @agola11

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

from langchain.chat_models import ChatOllama
llm = ChatOllama(
    model="openchat:7b-v3.5-1210-q4_K_M",
)
for chunk in llm.stream("Where were the Olympics held?"):
    print(chunk, end="", flush=True)

Expected behavior

successful streaming output from llm

[dosubot]

🤖

It's great to see that you've identified a potential issue and a possible solution related to the stream() method in LangChain. Your observation about adding a pending=True argument to the decorator seems like a step in the right direction.

If you're interested in contributing to LangChain, I would encourage you to go ahead and create a pull request with your proposed fix. The maintainers will review your changes and determine if it aligns with the project's goals.

Thank you for your interest in improving LangChain!

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[tylertitsworth]

I'm getting the same issue in my app. Chainlit offers a generic langchain provider that streams the model backend request provided by an llm input. When I add support for updating slider inputs in a playground request like so:

result = await make_async(self.llm.stream)(
    input=messages, # User Prompt
    **request.prompt.settings # User Inputs like temperature, top_K, etc.
)

I get the error above.

When I downgrade to v0.0.350 I also receive the same error, so I am unsure if the issue is in the specific PR you mentioned.

The out-of-the-box GenericLangchainProvider in chainlit for v0.0.352 doesn't work because of this issue.

Edit: On followup testing, it appears that v0.0.349 is the release that is working for me.

langchain_ollama_stream_error.mp4

[romellfudi]

The issue still persists in the latest version. Please provide guidance on how to handle streaming in ChatOllama. @v-byte-cpu do you have any updates?

[tylertitsworth]

Issue was fixed in #15104

[kesavan22]

I'm facing this issue for GooglePalm in the latest version