Add Support for /v1/chat/completions Endpoint in langchain_community.llms.llamafile

[hfahrudin]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

I am requesting the addition of support for the /v1/chat/completions endpoint within the langchain_community.llms.llamafile, integrated directly into the in generation function like .invoke, .stream etc. This would complement the existing /completion endpoint, allowing users to choose between the two endpoints depending on their specific needs or preferences.

Motivation

Currently, the .invoke or .stream function defaults to using the /completion endpoint, which can result in unexpected behavior in certain use cases. I have found that manually interacting with the /v1/chat/completions endpoint provides more reliable results for my workflow. Additionally, ensuring compatibility with PromptTemplate would maintain the flexibility and ease of use that langchain users expect.

Adding support for /v1/chat/completions would:

• Provide flexibility in endpoint selection, allowing users to choose the endpoint that best fits their specific use case.
• Ensure consistent response generation for workflows where /completion may yield suboptimal results.
• Broaden support for additional use cases by allowing users to take advantage of the prompt templating features while interacting with the alternative endpoint, providing a more cohesive user experience within langchain.

Proposal (If applicable)

I would be happy to contribute to this feature by implementing the following changes:

1. Endpoint Option in generation function:

   • Introduce an option in the .invoke and .stream function to specify the endpoint, allowing users to choose between /completion (the current default) and /v1/chat/completions.

2. Dynamic Payload Formatting:

   • Ensure that the payload format dynamically adapts based on the selected endpoint to guarantee compatibility with Llamafile's API requirements.

3. Compatibility with PromptTemplate:

   • Ensure that this alternative endpoint works seamlessly with PromptTemplate from langchain_core.prompts. This would allow users to continue using prompt templating functionality with either endpoint selection, providing flexibility while maintaining ease of integration with the broader langchain ecosystem.

4. Configuration Flexibility:

   • Make the endpoint selection configurable, either as an optional parameter in the .invoke method or as a global configuration setting, enabling users to set their preferred endpoint for all calls or on a per-call basis.