Provide implementations for HuggingFace 🤗 IChatCompletionService and TextEmbedding via Semantic Kernel?

[jkone27]

would be nice and easy to use to add hugging face models via semantic kernel extensions Microsoft.SemanticKernel.Connectors.HuggingFace so people can experiment more with it

which interfaces should be implemented to provide an huggingFace alternative to default openAI implementations?

this project seems really awesome, thank you

[dluc]

hi @jkone27 that's already possible, see WithSemanticKernelTextGenerationService and WithSemanticKernelTextEmbeddingGenerationService

[jkone27]

getting some errors, here is my sample

ref. errors here: #659

add ionide ext to run in vscode, test.fsx... keep text size small else ionide can throw exception if text exists the screen

// https://microsoft.github.io/kernel-memory/serverless
// https://www.nuget.org/packages/Microsoft.KernelMemory.SemanticKernelPlugin
// https://www.nuget.org/packages/Microsoft.KernelMemory.Core
// https://github.com/microsoft/kernel-memory/blob/main/examples/107-dotnet-SemanticKernel-TextCompletion/Program.cs

#r "nuget: Microsoft.KernelMemory.SemanticKernelPlugin, 0.71.240820.1"
#r "nuget: Microsoft.KernelMemory.Core, 0.71.240820.1"
#r "nuget: Microsoft.SemanticKernel.Connectors.HuggingFace, 1.15.0-preview"
#r "nuget: Microsoft.SemanticKernel.Plugins.Memory, 1.15.0-alpha"
#r "nuget: SmartComponents.LocalEmbeddings.SemanticKernel, 0.1.0-preview10148"
open Microsoft.KernelMemory
open Microsoft.KernelMemory.SemanticKernelPlugin
open Microsoft.KernelMemory.Configuration
open Microsoft.KernelMemory.SemanticKernel
open Microsoft.SemanticKernel
open Microsoft.SemanticKernel.Connectors.HuggingFace
open System
open System.Threading.Tasks
open System.Linq

let config = new SemanticKernelConfig()
config.MaxTokenTotal <- 1000

let TEXT_MODEL = "microsoft/Phi-3-mini-4k-instruct"
let EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
let API_KEY = "TEST_KEY"
let ENDPOINT = "https://api-inference.huggingface.co/" |> Uri

let memory = 

    let chatConfig = 
        new HuggingFaceTextGenerationService(
            TEXT_MODEL, 
            ENDPOINT, 
            API_KEY)

    let embeddingConfig = 
        new HuggingFaceTextEmbeddingGenerationService(
            EMBEDDING_MODEL, 
            ENDPOINT, 
            API_KEY)

    (new KernelMemoryBuilder())
        .WithSemanticKernelTextGenerationService(
                service=chatConfig, config=config)
        .WithSemanticKernelTextEmbeddingGenerationService(
                service=embeddingConfig, config=config)
        //.WithMemory(searchEndpoint, searchApiKey)
        .Build<MemoryServerless>()

let results = 
    task {
        
        let! _ = memory.ImportWebPageAsync(
            "https://raw.githubusercontent.com/microsoft/kernel-memory/main/COMMUNITY.md", 
            documentId= "doc001")

        let question = "How can I join Kernel Memory's Discord?"
        Console.WriteLine($"\n\nQuestion: {question}")

        let! answer = memory.AskAsync(question)
        Console.WriteLine($"\nAnswer: {answer.Result}")

        Console.WriteLine("\n\n  Sources:\n")

        for x in answer.RelevantSources do
            Console.WriteLine(
                $"  - {x.SourceName}  - {x.Link} [{x.Partitions.First().LastUpdate:D}]")
        
    }
    |> Async.AwaitTask
    |> Async.RunSynchronously

[dluc]

are you getting errors or warnings? warnings are not blocking, only highlighting something to be aware of.
could you paste the logs here?

[dluc]

@jkone27 looks like you shared a private key before editing the comment above - the key is still visible by checking the original message. I'd recommend invalidating that API key as soon as possible to avoid unexpected usage and bills