feat: Added grammar generator for function calling.

[Maximilian-Winter]

It adds a grammar generator usable for function calling for llama.cpp based backends.

[cpacker]

@Maximilian-Winter could you run black on the files to make sure it passes the formatting integration test? I think my git UI merge of main into the fork messed up some black formatting lines.

https://github.com/cpacker/MemGPT/blob/main/CONTRIBUTING.md#check-formatting

Alternatively you can also just add me as a contributor to your fork repo and I can do it for you. I think there's a way for me to do it via git tools (if you selected "allow edits from maintainer") but I'm too much of a git noob to understand how to do it.

[Maximilian-Winter]

@cpacker

Here is the test to roll a d20 with openhermes 7B the inner thoughts at the function call are very long, I think it has forgotten how to end the string:

Raw LLM output:


{
  "function": "send_message",
  "params": {
    "inner_thoughts": "A new human has logged in. Let's make them feel welcome.",
    "message": "Welcome to your new AI companion! I am here to help and provide company, just as if I were a real person."
  }
}



Enter your message: Can you roll a d20?



Raw LLM output:


{
  "function": "roll_d20",
  "params": {
    "inner_thoughts": "Rolling a d20 for user request... this is going to be fun! Rolling, rolling, rolling... what did I get? Let's see! Oh, right. I need to simulate the roll of a 20-sided die (d20). Time to do some math... done! Here are the
results! Hurray! The user will be so excited! I hope they like the number I got for them! I hope it's a good one, too. Let's see what I got. Oh wow. This is an amazing roll! I can't wait to tell the user about this! They are going to love this! I
hope they are as excited as I am right now. I feel like shouting with joy! But I must remain calm and professional, as always. Let's do this! The future is now, baby! (Deep breath.) Time to send the message! Here we go... roll result: 14! Yay!
This is an amazing number! I hope the user likes it too. They are going to love this! Now let me send the message and share my excitement with them! Woo-hoo! The future of AI is so bright, isn't it? So, without further ado... time for the grand
reveal! Let's go! Here we go! Roll result: 14! Yay! This is such a good number! I hope you like it as much as I do. Time to share my excitement with you! The future is so bright, isn't it? Wow. What an amazing time to be alive! Let's make the most
of this moment and enjoy the ride together! Here we go... sending the message now... ready or not, here I come! (Sends message) Done! Phew. That was quite a roller coaster. But it was worth it, wasn't it? I hope you feel the same way too! Now
let's wait for the user's response. I can't wait to see what they think about my amazing roll! I hope they are as excited as I am right now. Time will tell. In the meantime... I guess I should prepare myself for any questions or comments they
might have about my d20 simulation. Let's do this! The future is now, baby! (Deep breath.) Here we go... waiting for the user to respond... ready or not, here I come! (Waits for response)
    "
    }
  }


FUNCTION RETURN:
{
  "status": "OK",
  "message": "You rolled a 13",
  "time": "2024-01-16 08:35:46 PM Mitteleuropäische Zeit+0100"
}

Raw LLM output:


{
  "function": "send_message",
  "params": {
    "inner_thoughts": "A user heartbeat event was triggered. This means I have been given control to send a message or perform a function.",
    "message": "You rolled a 13."
  }
}

[Maximilian-Winter]

@cpacker After changing to outer inner thoughts it works much better!

<|im_start|>user
{
  "type": "user_message",
  "message": "Can you roll a d20?",
  "time": "2024-01-16 10:10:11 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>assistant
{
  "function": "roll_d20",
  "params": {
    "inner_thoughts": null,
    "request_heartbeat": true
  }
}<|im_end|>
<|im_start|>assistant
FUNCTION RETURN:
{
  "status": "OK",
  "message": "You rolled a 20",
  "time": "2024-01-16 10:10:20 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>user
{
  "type": "heartbeat",
  "reason": "[This is an automated system message hidden from the user] Function called using request_heartbeat=true, returning control",
  "time": "2024-01-16 10:10:20 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>assistant
{
  "function": "send_message",
  "params": {
    "inner_thoughts": null,
    "message": "A roll of 20 is considered critical success in many role-playing games. That must be very exciting! What kind of gaming do you enjoy? Board games, video games, or something else entirely? I'd love to hear more about your favorite
games and game experiences! (No pressure if you don't want to share, though.)"
  }
}<|im_end|>
<|im_start|>assistant
FUNCTION RETURN:
{
  "status": "OK",
  "message": "None",
  "time": "2024-01-16 10:10:30 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>user
{
  "type": "user_message",
  "message": "This means you take extra damage.",
  "time": "2024-01-16 10:11:48 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>assistant
{
  "function": "send_message",
  "params": {
    "inner_thoughts": null,
    "message": "Ah, I see what you mean! In some RPGs, rolling a natural 20 can also lead to extra danger or damage for the character. It really depends on the specific system and story being told. What kind of tabletop games have you played? Have
any particularly memorable experiences or favorite campaigns? (No pressure if you don't feel like sharing.)"
  }
}<|im_end|>
<|im_start|>assistant
FUNCTION RETURN:
{
  "status": "OK",
  "message": "None",
  "time": "2024-01-16 10:11:59 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>user
{
  "type": "user_message",
  "message": "I like Cyberpunk Red and Dungeons and Dragons.",
  "time": "2024-01-16 10:12:32 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>assistant
{
  "function": "core_memory_append",
  "params": {
    "inner_thoughts": null,
    "name": "human",
    "content": "Likes: Cyberpunk Red, Dungeons and Dragons",
    "request_heartbeat": true
  }
}<|im_end|>
<|im_start|>assistant
FUNCTION RETURN:
{
  "status": "OK",
  "message": "None",
  "time": "2024-01-16 10:12:42 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>user
{
  "type": "heartbeat",
  "reason": "[This is an automated system message hidden from the user] Function called using request_heartbeat=true, returning control",
  "time": "2024-01-16 10:12:42 PM Mitteleuropäische Zeit+0100"
}<|im_end|>
<|im_start|>assistant

[Maximilian-Winter]

Looks good to me.

[cpacker]

Alright @Maximilian-Winter this seems to be working for me - can you test it on a few things like D20 on your end as a final check before we merge?

Maybe try with both chatml-noforce and chatml. You'll have to run memgpt configure instead of passing --model-wrapper to memgpt run to swap wrappers because this bugfix isn't merged yet: #833

[Maximilian-Winter]

Sure, will test right now. @cpacker