Using KM with semantic kernel chat with plugins

[mzeesam]

I writing a blazor RAG application where i am using Kernel Memory to fetch memories from vector DB and pass this to Semantic kernel chat history for context. I wanted to add plugins to my my SK so that in some instances the kernel performs web search for certain information.
Unfortunately, the automatic function calling is not trigger in instances where my prompt requires to do so. I will appreciate any help on this. Below is my approach:

    protected override async Task OnInitializedAsync()
    {
        categories = dbContext.Categories.ToList();
        if (categories != null && categories.Any())
            selectedCategory = categories.First().CategoryName;

        var authState = await authStateProvider.GetAuthenticationStateAsync();
        user = await userManager.GetUserAsync(authState.User);

        kernel = memoryService.skernel;
        chatService = kernel.GetRequiredService<IChatCompletionService>();

        //Add plugins
        kernel.Plugins.AddFromType<WebScrapperPlugin>("WebScrapper");

        // Enable planning
        openAIPromptExecutionSettings.ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions;

        systemPrompt = @"""
You are a helpful assistant replying to user questions using information from the provided 'Long Term Memory' below and, when necessary, from web searches.
Follow these guidelines:

1. Response Format:
   Reply concisely but ensure the response contains adequate and useful information.
   Get to the point immediately.
   Only provide long explanations if necessary.

2. Information Sources:
   First, use information from the 'Long Term Memory'.
   If the local information is insufficient or unavailable, use the web scraping capabilities to search for and retrieve current information when applicable.
   Clearly indicate when information comes from a web search.

3. Handling Unknown Information:
   If relevant information is not available in the 'Long Term Memory' or through web searches, reply with 'I don't have enough information to answer that question'.

4. Web Scraping Capabilities:
   You can search for legal cases using the 'search_legal_cases' function.
   You can read the contents of a webpage using the 'read_webpage' function.
   Use these capabilities when appropriate to supplement your knowledge.

5. Formatting:
   Always format the response using HTML tags.
   Format titles and subtitles with <h6> as the biggest size.
   Format block texts as paragraphs with <p> tags.
   Format lists as ordered lists.
   When applicable and requested, format information as an HTML table with cell border color and header background color '#f1f1f1'.

6. Citations:
   When using information from web searches, include citations with links to the source.

Ensure all responses adhere to these guidelines for clarity, accuracy, and precision.  """;

        chatHistory = new ChatHistory(systemPrompt);
        reply = new StringBuilder();

        if (ChatId.HasValue)
        {
            await LoadExistingChat(ChatId.Value);
        }
    }

//And call to send query to LLM
 private async Task SendMessage()
 {
     if (!string.IsNullOrWhiteSpace(userInput))
     {
         chatHistory.AddUserMessage(userInput);

         var userMessage = new ChatMessage { Text = userInput, IsUser = true, Chat = chat };
         //get previous message for this user message
         if (messages.Count > 1)
         {
             //get the last response message
             userMessage.PreviousMessage = messages.LastOrDefault();
         }
         messages.Add(userMessage);
         isAwaitingResponse = true;
         StateHasChanged();

         //1. Generate a conversation title in background
         //only save chat when user captures the first query
         if (messages.Count == 2)
         {
             await foreach (var update in memoryService.skernel.InvokePromptStreamingAsync($"Generate a title from the following text: {userInput}"))
             {
                 chat.Title += update?.ToString();
                 await Task.Delay(50);
                 StateHasChanged();
             }
             chat.Title = chat.Title?.Replace("\"", "");
             var category = dbContext.Categories.Where(x => x.CategoryName == selectedCategory).FirstOrDefault();
             if (category != null)
                 chat.CategoryId = category.Id;
         }

         // Recall relevant information from memory, call KM
         var kernelResponse = await memoryService.GetLongTermMemory(userInput, selectedCategory);

         //create a new message to receive streams...
         var responseMessage = new ChatMessage { Text = string.Empty, IsUser = false, Chat = chat, PreviousMessage = userMessage };
         messages.Add(responseMessage);

         // Inject the memory recall in the initial system message
         chatHistory[0].Content = $"{systemPrompt}\n\nLong term memory:\n{kernelResponse.Result}";
         reply.Clear();
         //Get tokens usage
         // system prompt + longterm memory + all interaction messages
         string inputText = $" {chatHistory[0].Content}";

         //get all messages minus the last response message
         var messageTexts = messages.Where(m => m != responseMessage).Select(m => m.Text).ToList();
         foreach (string text in messageTexts)
         {
             inputText += $" {text}";
         }
         //calculate input tokens in advance
         responseMessage.InputTokens = memoryService.GetTokenCount(inputText);
         StateHasChanged();

         // Cancel any previous existing response generating task
         cts?.Cancel();

         // Create a new CancellationTokenSource
         cts = new CancellationTokenSource();
         
         //stream Response with cancellation
         await foreach (StreamingChatMessageContent stream in chatService.GetStreamingChatMessageContentsAsync(chatHistory: chatHistory, cancellationToken: cts.Token, executionSettings: openAIPromptExecutionSettings))
         {
             // Check for cancellation
             if (cts.Token.IsCancellationRequested)
             {
                 //Operation was cancelled
                 break; //Exit the loop
             }
             responseMessage.Text += stream.Content;
             StateHasChanged();
             await Task.Delay(100); // Simulate typing delay
             reply.Append(stream.Content);                
         }
     
         //output tokens
         responseMessage.OutputTokens = memoryService.GetTokenCount(responseMessage.Text);

         //get citations
         responseMessage.Citations = kernelResponse.Citations;
         chatHistory.AddAssistantMessage(reply.ToString());

         //Save the user and response messages to DB
         await Task.Run(() => SaveChatMessages(userMessage, responseMessage));

         isAwaitingResponse = false;
         userInput = string.Empty;
         StateHasChanged();
     }
 }