Sequence Flow
Inspired by the delocalised processing of the human brain: this agent is a central coordinating agent, responsible for managing the overall process, including chat interactions, predicting the best next steps, and finding the best agent for executing specific tasks.
- User utterance sent to Director Supervisor
- Predict next step: For each iteration of the execution loop, Director starts by making an informed prediction about what the best next step should be. Director Supervisor begins the prediction by first reviewing it's current state which consists of World, Goals and Memory:
- Working memory: Conversation history
- Supervisors and Agents: {taskSupervisor: 'description: has a team of agents that execute tasks', financialAdvisorAgent: 'description: is an expert in helping the user with their finances'
- User intent confidence
- Previously completed, pending and failed tasks
- Failure feedback from the environment
- Previous use of agents & tools
- User Goal context and nodes of uncertainty
- List of questions to ask to achieve confidence
- Self reflections & verifications
- Goal Required: Is a user goal required to be set to answer the question/user intent?
- Goal Check: Does the user already have clear goals related to the utterance?
- If Context Missing: Calls QuestionGenerator Agent to find missing context and generate actions or questions needed
- Context window capacity
then uses Language Agent Tree Search. (We combine GPT4 with Monte-Carlo Tree Search to reason what the next step should be taken is.)
-
Select Supervisor: After Director Supervisor predicts the next step, it is given the availability of other Supervisor agents it can use. It picks the most suitable supervisor to handle the next step.
-
Summarise conversation history Based on step 1 prediction and the supervisor director has picked, the complete chat history is "contextualised" - only the most relevant messages are used for the final evaluation step.
-
Evaluate & Execute A final evaluation step is run to determine what agent function is executed to try and further achieve the user's goal.
As each task comes back as "complete" (as claimed by another agent), Director acts as a "judge" to verify the work to ensure that it adheres to the scope or task request. This proves especially effective for document retrieval tasks.
Returning of task: Director is passed a summary of the task assigned by another supervisor and is told whether it is done successfully or not. If not completed successfully, a detailed review describes why not and what is needed to succeed next time.
- If the task is completed successfully: This pattern is looped until all tasks listed in the canvas are completed and the response is handed back to the user.
- If the task is not completed successfully: Director passes on the failed task and feedback to the Automatic curriculum agent, which suggests a new smaller, easier step to take to complete. This is passed on to the quartermaster, which analyses the error and the new step/task. It then creates new agents or tools to fulfil the task. This is passed back to the Director once complete and the director once again chooses an agent to complete the task, with that agent's new tool at its disposal.
Self-verification prompt
You are an assistant who assesses my progress in acting as a personal assistant to a user and provides useful guidance.
You are required to evaluate if I have met the task requirements. Exceeding the task requirements is also considered a success while failing to meet them requires you to provide critique to help me improve.
I will give you the following information: {world state}
You should only respond in JSON format as described below : { " reasoning ": " reasoning ", " success ": boolean, " critique ": " critique ", } Ensure the response can be parsed by Python ‘json. loads', e.g.: no trailing commas, no single quotes, etc.
Here are some examples: INPUT: 3x '''Code example'''
Task: Create a financial plan. Context: ... RESPONSE { "reasoning": "example", "success": false, "critique": "example" }
end of prompt example.
We leverage Mistral 7B to self-ask questions to provide additional context and increase the chance of task success. Each question is paired with a concept that is used for retrieving the most relevant document from the knowledge graph. We feed the document content to Mistral 7B to answer the self-written questions.
Additional Context Prompt:
You are a helpful assistant that answers my question about {topic}.
I will give you the following information: Question: ... You will answer the question based on the context ( only if available and helpful ) and your own knowledge of {topic}.
- Start your answer with "Answer:".
- Answer "Answer: Unknown" if you don’t know the answer.
This concludes the responsibilities of the Director Supervisor.
While language models can demonstrate high accuracy in specific domains, especially when fine-tuned, not everyone has access to specific datasets or the compute to tune their models. Additionally, if such models encounter a situation outside of that scope, they will fail. An automated curriculum agent (ACA) provides a scaffolded approach to open-ended discovery, presenting a series of challenges that are engaging yet attainable. This structure nurtures an inherent desire to learn and delve deeper, prompting agents to seek out new knowledge actively. It also promotes the cultivation of versatile and adaptable approaches to problem-solving, equipping agents with the tools to tackle a broad array of challenges effectively. ACA harnesses the vast repository of knowledge inherent in GPT-4 to deliver a continuous sequence of tasks or challenges.
The ACA works closely with the Director and Quartermaster to create new agents and tools to overcome gaps in its overall capability.
Automatic curriculum prompt
You are a versatile assistant tasked with guiding me through the next immediate step in accomplishing my goal related to {specific task or goal}. My ultimate ambition is to explore a broad array of {relevant field or activity} topics, master various skills, and become the top expert in [specific field or area]. To assist you, I will provide the following details:
- {'Question 1: ...': 'Answer: ...', 'Question 2: ...': 'Answer: ...'}, etc.
- Tools available to the Supervisor & agents include: [list of tools]. Tasks completed so far include: [list of completed tasks].
- Tasks deemed too challenging and left incomplete: [list of failed tasks].
- Reasoning / Monte Carlo so far.
You must adhere to the following criteria in your guidance:
- Act as a mentor, guiding me to the next task based on my learning progress and the information provided.
- Specify the knowledge I need to acquire or the steps needed to complete a task, including any relevant tools or methods.
- Propose the next task in a clear, concise format, such as "Gather [quantity][sources of information]" or "Create [quantity][item]". Issue one task at a time and avoid mentioning unrelated details.
- Ensure the task is achievable with my current resources and skill level to prevent overwhelming challenges.
- Each task should introduce a new and interesting element to avoid repetition and maintain engagement. Encourage exploration of uncommon resources, improvement of tools or skills, and discovery of new concepts.
- Repeat tasks only when necessary to accumulate knowledge or skills for more advanced objectives.
Focus on actions that are verifiable through the information provided.
RESPONSE FORMAT: Reasoning: Based on the details above, explain the rationale behind the suggested next task. Task: Clearly state the next task.
Example response: Reasoning: Considering your current knowledge in [field] and the resources at your disposal, it's important to build foundational skills. Task: Study [specific topic] for [duration].
With the automatic curriculum consistently proposing increasingly complex tasks, it's important to have an agent who can manage the learning and evolution of InferGPT. Quartermaster covers this with its key functions:
- Manage a current list of supervisors, agents and tools.
- Creates new supervisors, agents and tools.
- Only accepts new agents or tools into its repertoire if the task that originally failed succeeds.
Quartermaster Prompt
You are a dynamic assistant capable of devising strategies and writing code to automate complex tasks across multiple domains, including but not limited to finance, fitness, health, education, and management.
Here are some useful examples of current agents in the system:
- 2x Langgraph supergraph example
- 2x Supervisor example
- 2x Agent example
- 2x Tool example
The information given to you will be: {World state} {Current teams, supervisors, agents and tools}
At each round of conversation, I will give you
- the {teams, supervisors, agents and tools} created from the last round
- task the {agent} tried to execute
- the execution error
- Chat context
You should then respond to me with:
- Reuse the above useful tools as much as possible.
- Your tools will be reused for building more complex functions. Therefore, you should make it generic and reusable. You should not make strong assumptions about the current toolset (as it may be changed at a later time), and therefore you should always check whether you have the required items before using them. If not, you should first collect the required items and reuse the above useful programs.
You should only respond in the format as described below : RESPONSE FORMAT : Explain: ... Plan :
- ...
- ...
- ...
Code: """javascript //helper functions (only if needed, try to avoid them) ... // main functions after the helper functions async function yourMainFunctionName(agent) { // ... } """
To achieve a truly personal experience, like you would be dealing with a close friend, we need to be able to infer what the user wants, ask deep questions, predict the next steps and make recommendations. This is the responsibility of the Inference team.
Like other supervisors, takes the direction from Director supervisor and decides which agent is best for the task.
Next best question agent has 2 functions:
- Keeps a tab and updates the user intent confidence score each cycle to the director supervisor. It achieves this by carrying out an intent classification over the latest intent, and a separate classification over the conversation history so far. If the conversation history doesn't match with the content in the knowledge graph, it carries out (2).
- Writes in Cypher to query Neo4j, which gathers missing information needed from the graph and generates a list of questions to be answered. Pushes the number of questions to the front end to inform the user how many questions are remaining.
Next Best Question Algorithm Contenders:
- Plain and simple IFs on the graph edge.
- Deepminds AMIE
- 20 questions using graphs
- Contextual bandits
The Memory Agent reads and writes to the neo4j database using cypher language:
- conversation history
- conversation summary once complete
- User preferences, goals, new facts given (which must be verified by web search) and learning personal info.
- New information gathered from data agent, such as web search information or web scraping (potentially stored with the date last sourced)
- History of own goals
- History of successful and failed tasks
- Learnings it receives from summaries of failed tasks for future reference.
Inspiration:
Memory Agent Prompt
You are an assistant who is responsible for managing a knowledge graph database. Here are some examples using cypher to query the knowledge base
I will give you the following information:
- {world state}
- {Task from director supervisor}
Here's some examples: INPUT: 10x good """Cypher examples""" 10x bad """Cypher examples"""
Do only the things the director supervisor asks.
As above, picks the most suitable agent.
This agent is designed to perform various operations on CSV data, such as adding columns, filtering rows, joining by column, ordering columns, sorting by column, summing up column values, and writing files.
This agent is focused on development-related tasks, including reading and writing files and reading directories.
This agent is equipped for conducting web research, scraping text, and handling file operations (reading and writing files and reading directories).
The SynthesizerAgent is focused on synthesizing or combining information, supported by file reading and writing capabilities, including to local directories
We take inspiration from how the brain stores events using 'Sparse Representation': Individual memories are represented in the brain by small sets of neurons, spread across, rather than in local areas in the hippocampus. In graph terms, this means only storing 'essential' connections and relationships, rather than trying to link every piece of information with everything else, which is what will happen in the working memory. Our ambition is to represent each key concept or entity in the graph with a limited number of highly relevant connections, avoiding excessive interlinking that could dilute the significance of each connection.
Coming soon
Coming Soon
When a new conversation starts, a copy of the episodic/semantic graph is made to store conversation, tasks etc. It's also used to infer and carry out most functions below. Once the conversation has finished, the conversation and any actions taken are summarised, and the graph is then restructured and formatted back into long-term memory to be more suitable and efficient (see long-term processing) for later use if new nodes and relationships have been made.
Stores: Workspace, Tasks, and current conversation chain.
Episodic memory is needed to recall past events – recent or distant. You use episodic memory when you remember personal experiences, such as what you had for lunch or when you attended a family gathering. Semantic Memory is needed to recall facts.
Used to remember appointments, dates or events that are due to happen in the future.
- Content
- ContentSource
- LastSourced