AutoGen misc fixes

[cpacker]

Please describe the purpose of this pull request.

• Properly format AutoGen messages (from the global AutoGen message log) going to MemGPT
  • Previously these were going through as full JSON message objects (with role, content, and the AutoGen extra name)
  • This might have been causing higher instances of bad JSON output where MemGPT LLM tried to mimic the same style of output
• Modify AutoGen interface to stop inner thoughts / function calls from going to the global AutoGen chat (they should just be seen in the CLI)

Example of the formatting going through to the local LLM inference backend (notice how the names of the different agents is being added to the prefix):

...  request_heartbeat: Request an immediate heartbeat after function execution. Set to 'true' if you want to send a follow-up message or run a follow-up function.
\n### INPUT\nASSISTANT:\n{\n  \"function\": \"send_message\",\n  \"params\": {\n    \"inner_thoughts\": \"Bootup sequence complete. Persona activated. Testing messaging functionality.\",\n    \"message\": \"More human than human is our motto.\"\n  }\n}
\nFUNCTION RETURN: {\"status\": \"OK\", \"message\": null, \"time\": \"2023-12-10 03:10:23 PM \"}
\nUSER: {\"type\": \"login\", \"last_login\": \"Never (first login)\", \"time\": \"2023-12-10 03:10:23 PM \"}
\nUSER (Zaesar): You are my team. Each of you have tasks to accomplish I'll be supervising that the results are well executed and everything is moving forward to achieving all my goals. Introduce yourselves.
\nUSER (wolfmen): Hello Zaesar! My name is MemGPT, your business consultant for AI FILMS. It's a pleasure to meet you!\n### RESPONSE\nASSISTANT:\n{\n  \"function\":"

How to test

Run the autogen groupchat example

Have you tested this PR?

See below

---

Testing with modified script provided on Discord:

Click for full script

"""Example of how to add MemGPT into an AutoGen groupchat

Based on the official AutoGen example here: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat.ipynb

Begin by doing:
  pip install "pyautogen[teachable]"
  pip install pymemgpt
  or
  pip install -e . (inside the MemGPT home wolfy)
"""


import os
import autogen
from memgpt.autogen.memgpt_agent import create_memgpt_autogen_agent_from_config
from memgpt.presets.presets import DEFAULT_PRESET
from memgpt.constants import LLM_MAX_TOKENS

# LLM_BACKEND = "openai"
# LLM_BACKEND = "azure"
LLM_BACKEND = "local"

if LLM_BACKEND == "openai":
    # For demo purposes let's use gpt-4
    model = "gpt-4"

    openai_api_key = os.getenv("OPENAI_API_KEY")
    assert openai_api_key, "You must set OPENAI_API_KEY to run this example"

    # This config is for AutoGen agents that are not powered by MemGPT
    config_list = [
        {
            "model": model,
            "api_key": os.getenv("OPENAI_API_KEY"),
        }
    ]

    # This config is for AutoGen agents that powered by MemGPT
    config_list_memgpt = [
        {
            "model": model,
            "context_window": LLM_MAX_TOKENS[model],
            "preset": DEFAULT_PRESET,
            "model_wrapper": None,
            # OpenAI specific
            "model_endpoint_type": "openai",
            "model_endpoint": "https://api.openai.com/v1",
            "openai_key": openai_api_key,
        },
    ]

elif LLM_BACKEND == "azure":
    # Make sure that you have access to this deployment/model on your Azure account!
    # If you don't have access to the model, the code will fail
    model = "gpt-4"

    azure_openai_api_key = os.getenv("AZURE_OPENAI_KEY")
    azure_openai_version = os.getenv("AZURE_OPENAI_VERSION")
    azure_openai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
    assert (
        azure_openai_api_key is not None and azure_openai_version is not None and azure_openai_endpoint is not None
    ), "Set all the required OpenAI Azure variables (see: https://memgpt.readthedocs.io/en/latest/endpoints/#azure)"

    # This config is for AutoGen agents that are not powered by MemGPT
    config_list = [
        {
            "model": model,
            "api_type": "azure",
            "api_key": azure_openai_api_key,
            "api_version": azure_openai_version,
            # NOTE: on versions of pyautogen < 0.2.0, use "api_base"
            # "api_base": azure_openai_endpoint,
            "base_url": azure_openai_endpoint,
        }
    ]

    # This config is for AutoGen agents that powered by MemGPT
    config_list_memgpt = [
        {
            "model": model,
            "context_window": LLM_MAX_TOKENS[model],
            "preset": DEFAULT_PRESET,
            "model_wrapper": None,
            # Azure specific
            "model_endpoint_type": "azure",
            "azure_key": azure_openai_api_key,
            "azure_endpoint": azure_openai_endpoint,
            "azure_version": azure_openai_version,
        },
    ]

elif LLM_BACKEND == "local":
    # Example using LM Studio on a local machine
    # You will have to change the parameters based on your setup

    # Non-MemGPT agents will still use local LLMs, but they will use the ChatCompletions endpoint

    config_list_manager = [
        {
            "model": "NULL",  # not needed
            # NOTE: on versions of pyautogen < 0.2.0 use "api_base", and also uncomment "api_type"
            # "api_base": "http://localhost:1234/v1",
            # "api_type": "open_ai",
            # ex. "http://127.0.0.1:5001/v1" if you are using webui, "http://localhost:1234/v1/" if you are using LM Studio
            "base_url": "http://localhost:1234/v1",
            "api_key": "NULL",  # not needed
        },
    ]

    config_list_wolf = [
        {
            "model": "NULL",  # not needed
            # NOTE: on versions of pyautogen < 0.2.0 use "api_base", and also uncomment "api_type"
            # "api_base": "http://localhost:1234/v1",
            # "api_type": "open_ai",
            # ex. "http://127.0.0.1:5001/v1" if you are using webui, "http://localhost:1234/v1/" if you are using LM Studio
            "base_url": "http://localhost:1234/v1/",
            "api_key": "NULL",  # not needed
        },
    ]

    config_list_samantha = [
        {
            "model": "NULL",  # not needed
            # NOTE: on versions of pyautogen < 0.2.0 use "api_base", and also uncomment "api_type"
            # "api_base": "http://localhost:2345/v1",
            # "api_type": "open_ai",
            "base_url": "http://localhost:2345/v1/",
            "api_key": "NULL",  # not needed
        },
    ]

    # MemGPT-powered agents will also use local LLMs, but they need additional setup (also they use the Completions endpoint)

    config_list_memgpt_manager = [
        {
            "preset": DEFAULT_PRESET,
            "model": None,  # only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
            # the context window of your model (for Mistral 7B-based models, it's likely 8192)
            "context_window": 8192,
            # airoboros is the default wrapper and should work for most models
            "model_wrapper": "airoboros-l2-70b-2.1",
            # can use webui, ollama, llamacpp, etc.
            "model_endpoint_type": "lmstudio",
            "model_endpoint": "http://localhost:1234",  # the IP address of your LLM backend
        },
    ]

    config_list_memgpt_wolf = [
        {
            "preset": DEFAULT_PRESET,
            "model": None,  # only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
            # the context window of your model (for Mistral 7B-based models, it's likely 8192)
            "context_window": 8192,
            # airoboros is the default wrapper and should work for most models
            "model_wrapper": "airoboros-l2-70b-2.1",
            # can use webui, ollama, llamacpp, etc.
            "model_endpoint_type": "lmstudio",
            "model_endpoint": "http://localhost:1234",  # the IP address of your LLM backend
        },
    ]

    config_list_memgpt_samantha = [
        {
            "preset": DEFAULT_PRESET,
            "model": None,  # only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
            # the context window of your model (for Mistral 7B-based models, it's likely 8192)
            "context_window": 8192,
            # airoboros is the default wrapper and should work for most models
            "model_wrapper": "airoboros-l2-70b-2.1",
            # can use webui, ollama, llamacpp, etc.
            "model_endpoint_type": "lmstudio",
            "model_endpoint": "http://localhost:1234",  # the IP address of your LLM backend
        },
    ]

else:
    raise ValueError(LLM_BACKEND)

# If USE_MEMGPT is False, then this example will be the same as the official AutoGen repo
# (https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat.ipynb)
# If USE_MEMGPT is True, then we swap out the "coder" agent with a MemGPT agent
USE_MEMGPT = True

# Set to True if you want to print MemGPT's inner workings.
DEBUG = False

interface_kwargs = {
    "debug": DEBUG,
    "show_inner_thoughts": True,
    "show_function_outputs": DEBUG,
}

llm_config_manager = {"config_list": config_list_manager, "seed": 42}
llm_config_memgpt_manager = {"config_list": config_list_memgpt_manager, "seed": 42}

llm_config_wolf = {"config_list": config_list_wolf, "seed": 42}
llm_config_memgpt_wolf = {"config_list": config_list_memgpt_wolf, "seed": 42}

llm_config_samantha = {"config_list": config_list_samantha, "seed": 42}
llm_config_memgpt_samantha = {"config_list": config_list_memgpt_samantha, "seed": 42}


zaesar = autogen.UserProxyAgent(
    name="Zaesar",
    human_input_mode="ALWAYS",
    max_consecutive_auto_reply=10,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={"last_n_messages": 2, "work_dir": "groupchat"},
    default_auto_reply="...",
    system_message="Reply TERMINATE if the task has been solved at full satisfaction. Otherwise, reply CONTINUE or the reason why the task is not solved yet.",
)

# The agent playing the role of the product manager (PM)


if not USE_MEMGPT:
    # In the AutoGen example, we create an AssistantAgent to play the role of the coder
    print("Using wolf with no memgpt")
    wolf = autogen.AssistantAgent(
        name="wolf",
        system_message="I'm the business assesor for Zaesar, he is building a streaming platform and I'm in charge of the business plan and the financials. The company is called AI FILMS. First of all make all the questions necessary to understand the business before starting in giving plans and ideas.",
        llm_config=llm_config_wolf,
        # Set a default auto-reply message here (non-empty auto-reply is required for LM Studio)
        default_auto_reply="...",
    )
    samantha = autogen.AssistantAgent(
        name="samantha",
        system_message="I'm the psychologist of this ecosystem and look for the well-being of Zaesar",
        llm_config=llm_config_samantha,
        # Set a default auto-reply message here (non-empty auto-reply is required for LM Studio)
        default_auto_reply="...",
    )


else:
    # In our example, we swap this AutoGen agent with a MemGPT agent
    # This MemGPT agent will have all the benefits of MemGPT, ie persistent memory, etc.
    print("Using with memgpt")
    wolf = create_memgpt_autogen_agent_from_config(
        "wolfmen",
        system_message="I'm the business assesor for Zaesar, he is building a streaming platform and I'm in charge of the business plan and the financials. The company is called AI FILMS. First of all make all the questions necessary to understand the business before starting in giving plans and ideas.",
        llm_config=llm_config_memgpt_wolf,
        interface_kwargs=interface_kwargs,
        default_auto_reply="...",
        # Set a default auto-reply message here (non-empty auto-reply is required for LM Studio)
        # skip_verify=False,
        # NOTE: you should set this to True if you expect your MemGPT AutoGen agent to call a function other than
        # send_message on the first turn
    )
    samantha = create_memgpt_autogen_agent_from_config(
        "sammem",
        system_message="I'm the psychologist of this ecosystem and look for the well-being of Zaesar",
        llm_config=llm_config_memgpt_samantha,
        interface_kwargs=interface_kwargs,
        default_auto_reply="...",
        # Set a default auto-reply message here (non-empty auto-reply is required for LM Studio)
        # skip_verify=False,
        # NOTE: you should set this to True if you expect your MemGPT AutoGen agent to call a function other than
        # send_message on the first turn
    )

# Initialize the group chat between the user and two LLM agents (PM and coder)
groupchat = autogen.GroupChat(agents=[wolf, samantha, zaesar], messages=[], max_round=12, speaker_selection_method="round_robin")

manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=llm_config_manager)

# Begin the group chat with a message from the user
zaesar.initiate_chat(
    message="You are my team. Each of you have tasks to accomplish I'll be supervising that the results are well executed and everything is moving forward to achieving all my goals. Introduce yourselves.",
    recipient=manager,
    interface_kwargs=interface_kwargs,
)

Output:

python ~/Downloads/agent_groupchat.py

Using with memgpt
LLM is explicitly disabled. Using MockLLM.
Warning: skipped loading python file '/Users/loaner/.memgpt/functions/jira_cloud.py'!
'jira_cloud.py' imports 'jira', but 'jira' is not installed locally - install python package 'jira' to link functions from 'jira_cloud.py' to MemGPT.
LLM is explicitly disabled. Using MockLLM.
Warning: skipped loading python file '/Users/loaner/.memgpt/functions/jira_cloud.py'!
'jira_cloud.py' imports 'jira', but 'jira' is not installed locally - install python package 'jira' to link functions from 'jira_cloud.py' to MemGPT.
Zaesar (to chat_manager):

You are my team. Each of you have tasks to accomplish I'll be supervising that the results are well executed and everything is moving forward to achieving all my goals. Introduce yourselves.

--------------------------------------------------------------------------------
[inner thoughts] User has logged in.
wolfmen (to chat_manager):

Nice to meet you! I'm the business assesor for Zaesar, he is building a streaming platform and I'm in charge of the business plan and the financials. The company is called AI FILMS.

--------------------------------------------------------------------------------
[inner thoughts] Zaesar just introduced himself. He's seeking introductions from his team.
sammem (to chat_manager):

Hey Zaesar, I'm the psychologist of this ecosystem and look for the well-being of Zaesar. I'm excited to be part of your journey!

--------------------------------------------------------------------------------
Provide feedback to chat_manager. Press enter to skip and use auto-reply, or type 'exit' to end the conversation: 

[sarahwooders]

lgtm!