AI Short-Term Memory (Agent Context) Management

When creating an agent instance and implementing role-playing conversations, the agent can remember recent chat interactions (commonly known as short-term memory), which is implemented through the context of LLM (Large Language Model). During voice conversations, memory has the following stages:

Initial Memory: Whether chat history is needed when creating an instance.
- You can use In-app chat conversation message history as initial memory;
- You can customize the initial memory with external context. If empty, the AI agent has no memory.
Memory During Voice Call: The chat history generated during voice calls and cached on the AI Agent server.
- To get the real-time memory, read Get Agent Instance Context List.
- To clear the existing memory, read Reset Agent Instance Context List.
Memory Archival After Voice Call:
- You can archive the chat history in In-app chat. Read Archive Memory After Voice Call.

Warning

Context Length Limit: LLMs typically have a maximum length limit, exceeding which will result in an error. For example, "skylark-1.5-pro-32k" allows no more than 32k tokens.
Context Length vs. Inference Time: Generally, longer context results in slower LLM output.
Context Length vs. Cost: LLM pricing is usually based on context length for input billing - longer length means higher cost.

Implementation Steps

Set Initial Memory

When creating an agent instance, you can set the initial memory (initial context) for the agent instance. Memory sources can be either "In-app chat conversation history" or "custom external context", controlled through the MessageHistory parameter of the create agent instance API. The detailed parameter structure is as follows:

MessageHistory

MessageHistory.Messages

MessageHistory.In-app chat

Load Initial Memory from In-app chat conversation History

Ensure you have enabled the In-app chat service.
When creating an agent instance, set SyncMode to 0 and fill in the corresponding In-app chat parameters. Example:

Note

In-app chat.RobotId is the UserInfo.UserId used when calling the In-app chat API to register a bot.

Untitled

 "MessageHistory": {
    "SyncMode": 0,
    "In-app chat": {
        "RobotId": "@RBT#123",
        "LoadMessageCount": 10
    }
 }

Copied!

Load Initial Memory from Custom External Context

Business needs to store context information itself.
When creating an agent instance, set SyncMode to 1 and fill in the Messages parameter. Example:

Untitled

 "MessageHistory":{
    "SyncMode": 1,
    "Messages": [
        {
            "Role": "user",
            "Content": "What's your name?"
        },
        {
            "Role": "assistant",
            "Content": "My name is Skylark."
        },
        {
            "Role": "user",
            "Content": "Tell me a story."
        },
        {
            "Role": "assistant",
            "Content": "Sure, let me tell you the story of 'The Three Little Pigs'."
        }
    ]
}

Copied!

Manage Memory During Voice Call

Managing memory during voice calls means managing the agent instance context. You can get the context or reset it.

Get Agent Instance Context List

Call the GetAgentInstanceMsgList API with the AgentInstanceId returned after creating an agent instance. The server will return the context of the AI agent instance, with messages sorted by chat time in ascending order.

Sample

{
    "Code": 0,
    "Message": "success",
    "RequestId": "2537521374375652066",
    "Data": {
        "Total": 4,
        "MessageList": [
            {
                "Role": "user",
                "Content": "What's your name?"
            },
            {
                "Role": "assistant",
                "Content": "My name is Skylark."
            },
            {
                "Role": "user",
                "Content": "Tell me a story."
            },
            {
                "Role": "assistant",
                "Content": "Sure, let me tell you the story of 'The Three Little Pigs'."
            }
        ]
    }
}

Copied!

Reset Agent Instance Context List

Call the ResetAgentInstanceMsgList API with the AgentInstanceId returned after creating an agent instance. The server will reset the context of the AI agent instance.

Archive Memory After Voice Call

Simply set SyncMode to 0 and provide a valid In-app chat bot ID as In-app chat.RobotId when creating an agent instance, and the voice call conversation records will be stored in the In-app Chat service.

These stored chat history messages can be used as initial memory for subsequent conversations. Read Load Initial Memory from In-app chat conversation History. You can also maintain memory archives in other ways according to your business needs.

Sample

"MessageHistory": {
    "SyncMode": 0,
    "In-app chat": {
        "RobotId": "@RBT#123"
    }
}

Copied!