feat: allow templated system messages

[cpacker]

Please describe the purpose of this pull request.

Refactors the way system messages are compiled to set the foundation for system message templating.

This PR allows for system messages that have the reserved variable {CORE_MEMORY} in them.

1. If {CORE_MEMORY} exists in the system prompt, the system prompt will be formatted such that the core memory block is injected there
2. If {CORE_MEMORY} does not exist in the system prompt, it'll automatically be appended to the end of the system prompt (this makes this PR backwards compatible)

Future PRs should enable users to define their own variables via user_defined_variables (TBD where this should be stored in the DB), which can be used to

Also, a future PR should enable other templating styles other than just f-strings, e.g. Mustache / jinja2 (or Handlebars via pybars). I've added this as a reminder:

    template_format: Literal["f-string", "mustache", "jinja2"] = "f-string",

How to test

Try modifying memgpt_chat.txt to have {CORE_MEMORY} at the end:

...
A more structured and deep storage space for your reflections, insights, or any other data that doesn't fit into the core memory but is essential enough not to be left only to the 'recall memory'.
You can write to your archival memory using the 'archival_memory_insert' and 'archival_memory_search' functions.
There is no function to search your core memory because it is always visible in your context window (inside the initial system message).

Base instructions finished.
From now on, you are going to act as your persona.

{CORE_MEMORY}

Then create a new agent (memgpt run) and check the memory block in system (/dumpraw).

Have you tested this PR?

Yes

Related issues or PRs

#880 #895

[sarahwooders]

lgtm!