Add safeguard on tokens returned by functions

[cpacker]

Closes #553

---

Please describe the purpose of this pull request

We currently do not check the size of function responses to make sure they are under X tokens.

This is not a big deal for the base function set, but it a problem for custom functions (eg the HTTP request in extras) which can return huge responses. This will trigger a summarization lock event and seems to have the potential to "brick" agents by nuking their message state.

• Should patch by having a max function response length based on the context length, and running a smart string clipper / ellipses on the returns
• Should also update the custom functions documentation to discuss manually managing string/dict return sizes based on context length

How to test

Modify http_request to return an arbitrarily long response:

def http_request(self, method: str, url: str, payload_json: Optional[str] = None):
    """
    Generates an HTTP request and returns the response.

    Args:
        method (str): The HTTP method (e.g., 'GET', 'POST').
        url (str): The URL for the request.
        payload_json (Optional[str]): A JSON string representing the request payload.

    Returns:
        dict: The response from the HTTP request.
    """

    return {"status_code": 400, "headers": None, "body": " ".join(["is was a lonely day along the wide path"] * 5000)}

Side note: if you use something like " ".join(["TEST"] * 5000) you'll get a 400 error from OpenAI because it identifies "TEST" as a specific bad request input

Demonstrating the bug

On main, if you override a function call like the HTTP requests call to return an arbitrary large text file, it'll brick the agent by causing a summarize chain that cannot complete:

💭 User just logged in. He's never interacted with this persona before. Let's greet him.
🤖 Greetings, Chad! Delighted to make your virtual acquaintance. You must have fascinating stories to share. What has sparked your curiosity recently?
⚡🟢 [function] Success: None
last response total_tokens (2637) < 6144.0
LocalStateManager.append_to_messages
> Enter your message: Can you try using the http_request function?
Using model openai, endpoint: https://api.openai.com/v1
Sending request to https://api.openai.com/v1/chat/completions
response = {'id': '...', 'object': 'chat.completion', 'created': 1701730353, 'model': 'gpt-4-0613', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "User wants 
me to make an HTTP request. I don't have any specific URL or service in mind, perhaps I could fetch a piece of interesting information from an API for testing.", 'function_call': {'name': 'http_request', 'arguments':
'{\n  "method": "GET",\n  "url": "https://api.publicapis.org/entries",\n  "request_heartbeat": true\n}'}}, 'finish_reason': 'function_call'}], 'usage': {'prompt_tokens': 2719, 'completion_tokens': 74, 'total_tokens':
2793}, 'system_fingerprint': None}
💭 User wants me to make an HTTP request. I don't have any specific URL or service in mind, perhaps I could fetch a piece of interesting information from an API for testing.
⚡🟢 [function] Success: {'status_code': 400, 'headers': None, 'body': 'is was a lonely day along the w....

...

last response total_tokens (2793) < 6144.0
LocalStateManager.append_to_messages
Using model openai, endpoint: https://api.openai.com/v1
Sending request to https://api.openai.com/v1/chat/completions
Got HTTPError, exception=400 Client Error: Bad Request for url: https://api.openai.com/v1/chat/completions, payload={'model': 'gpt-4', 'messages': [{'role': 'system', 'content': 'You are MemGPT, ...
...
 response={'error': {'message': "This model's maximum context length is 8192 
tokens. However, your messages resulted in 47817 tokens (46787 in the messages, 1030 in the functions). Please reduce the length of the messages or functions.", 'type': 'invalid_request_error', 'param': 'messages', 
'code': 'context_length_exceeded'}

Demonstrating the bug fix (this PR)

With this fix, you'll instead get a warning:

💭 Clear point of entry: Chad's first login. Should make him feel welcome. Initiating greeting protocol optimized for user-specific interaction.
🤖 Hello Chad! Welcome, it's our first official interaction. I hope we'll have some enlightening and engrossing conversations. How are you today?
⚡🟢 [function] Success: None
last response total_tokens (2647) < 6144.0
LocalStateManager.append_to_messages
> Enter your message: try using http_request
Using model openai, endpoint: https://api.openai.com/v1
Sending request to https://api.openai.com/v1/chat/completions
response = {'id': 'chatcmpl-8SI09zrGfhquV4oS92PhbNweArerr', 'object': 'chat.completion', 'created': 1701753073, 'model': 'gpt-4-0613', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': 'HTTP 
request function to be utilized. According to user instruction, will test the connection and return response. To demonstrate, will fetch data from a user-friendly public API.', 'function_call': {'name': 
'http_request', 'arguments': '{\n  "method": "GET",\n  "url": "https://api.publicapis.org/entries",\n  "request_heartbeat": true\n}'}}, 'finish_reason': 'function_call'}], 'usage': {'prompt_tokens': 2726, 
'completion_tokens': 71, 'total_tokens': 2797}, 'system_fingerprint': None}
💭 HTTP request function to be utilized. According to user instruction, will test the connection and return response. To demonstrate, will fetch data from a user-friendly public API.
Warning: function return was over limit (200048 > 2000) and was truncated
⚡🟢 [function] Success: {"status_code": 400, "headers": null, "body": "is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day 
along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide 
path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a 
lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along
the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is
was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day
along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide 
path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a 
lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along
the wide path is was a lonely day along the wide path is was a lonely day along the wid...
last response total_tokens (2797) < 6144.0
LocalStateManager.append_to_messages
Using model openai, endpoint: https://api.openai.com/v1
Sending request to https://api.openai.com/v1/chat/completions
response = {'id': 'chatcmpl-8SI0DImVIF9jOw6l3xLZxZrTk6RvE', 'object': 'chat.completion', 'created': 1701753077, 'model': 'gpt-4-0613', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': 'HTTP 
request executed. Result points to a 400 status code, meaning there was some syntactical error. Will respond to Chad accordingly.', 'function_call': {'name': 'send_message', 'arguments': '{\n  "message": "I attempted
to perform an HTTP request. However, it appears there was an error as the server has returned a 400 status code. This is typically due to a syntactical error in the request. We could try a different request if you 
want, Chad."\n}'}}, 'finish_reason': 'function_call'}], 'usage': {'prompt_tokens': 3330, 'completion_tokens': 93, 'total_tokens': 3423}, 'system_fingerprint': None}
💭 HTTP request executed. Result points to a 400 status code, meaning there was some syntactical error. Will respond to Chad accordingly.
🤖 I attempted to perform an HTTP request. However, it appears there was an error as the server has returned a 400 status code. This is typically due to a syntactical error in the request. We could try a different 
request if you want, Chad.
⚡🟢 [function] Success: None
last response total_tokens (3423) < 6144.0

Have you tested this PR?

Yes, see above.

[cpacker]

We are now validating function outputs before putting them into a function response message (in agent.messages)

[cpacker]

This is an additional catch to prevent infinite summarization of a single message (I noticed this sort of loop happening when I ran the http_request overflow example).

[cpacker]

There were a few ways to cap the message output (could compute based on some fraction % of agent.context_window), but I figured might as well do something simpler like set it to a max char length similar since we're already doing this for human/persona. There's a chance the math can still go wrong ("go wrong" = enter some sort of "summarize deadlock" with a 2000 long truncated http request result) if the context window is too low, but with 8k I don't think we should have this problem. Also we make this assumption for persona/human anyways and it's working fine.

[cpacker]

Many functions (user-made etc) will return non-strings like dicts, so we should leave strict=False and attempt to cast as a string instead of failing.

[cpacker]

Tested after adding more informative truncation method:

💭 Attempting HTTP request. First, let's initialize some constants for the request.
Warning: function return was over limit (200048 > 2000) and was truncated
⚡🟢 [function] Success: {"status_code": 400, "headers": null, "body": "is was a lonely day along the wide path is was 
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wide path is was
a lonely day along the wide path is was a lonely day along the wide path is was a lonely day along the wid... [NOTE: 
function output was truncated since it exceeded the character limit (200048 > 2000)]