feat: add local model support and examples

[vacekj]

this PR adds a client impl for calling local models, such as ollama and lm studio. it also duplicates some examples, such as simple chat, embeddings and tools calls. it also adds a collab example with local models, that demonstrates models working together to find a good prompt for themselves.

I have verified all of the examples work for me, on an M1 Max w/ ollama latest.

[cvauclair]

Hey @vacekj thank you for your contribution and interest in Rig, this is seriously awesome to see! We'll review this ASAP!

In the meantime, I've enabled CI on this PR so if you could address the couple linting/styling issues that might pop up that would be awesome. Cheers!

[vacekj]

I belive my last commit should address the singular lint issue. Please feel free to re-run the CI to see if everything's alright.

[cvauclair]

FYI @vacekj the currently failing tests are known to fail when a PR is made from an outside contributor, so no need to worry about those

[akashicMarga]

@vacekj @cvauclair instead of local.rs in providers we can have ollama.rs, lm_studio.rs, as there are other servers too which run locally like llama.cpp, LLMedge, ?

[vacekj]

@akashicMarga All of the projects you mentioned use the standard OpenAI API. I don't think it's necessary to duplicate the client., as it would be the same interface for LM Studio, ollama and LLMEdge.

[akashicMarga]

@akashicMarga All of the projects you mentioned use the standard OpenAI API. I don't think it's necessary to duplicate the client., as it would be the same interface for LM Studio, ollama and LLMEdge.

thought so, I was just thinking let's say in future if any local server that follows some other format where it will be integrated. so local_opena_ai_compatible.rs, or something on this line?

[cvauclair]

@akashicMarga @vacekj I think it's fine to have a local.rs for local openai-compatible servers. However, if say for example it turns out that using the ollama base API (and not the openai compatible one) works better, then we could make a dedicated client for it in ollama.rs.

[cvauclair]

@vacekj are there differences between your local.rs provider client and the openai.rs one? The latter allows the user to set the url with from_url which you could use to point the client to a local server, so perhaps a lot of code can be reused 🤔

[0xMochan]

@vacekj are there differences between your local.rs provider client and the openai.rs one? The latter allows the user to set the url with from_url which you could use to point the client to a local server, so perhaps a lot of code can be reused 🤔

Yea, the OpenAI compatibility server from ollama works fine using just the OpenAI client which is also what the documentation recommends when using the official openai pypi library

use rig::{completion::Prompt, providers};

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    // Create an Ollama client using the OpenAI compatibility layer
    let client = providers::openai::Client::from_url("ollama", "http://localhost:11434");

    // Create agent with a single context prompt
    let comedian_agent = client
        .agent("llama3.2:latest")
        .preamble("You are a comedian here to entertain the user using humour and jokes.")
        .build();

    // Prompt the agent and print the response
    let response = comedian_agent.prompt("Entertain me!").await?;
    println!("{}", response);

    Ok(())
}

There is the normal Ollama endpoint that would be worth to create an ollama.rs file for. For this PR to continue, it would be suitable to go for a direct ollama integration to use those specific features!

[vacekj]

Great, then I will implement the API ollama lists here: https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion

That could unlock some cool new capabilities, like an agent pulling a new model and spawning it on demand. Agents spawning other agents, like recruiting for an army hehe.

[cvauclair]

@vacekj I'll close the PR for now to reduce clutter and because we're talking about a new whole new client implementation here. We can re-open this one later or create a new one once the new integration will be ready for review.

As always, thanks for contributing :)