Display User and Agent Instance Status

During real-time voice calls with AI agents, you may need to display the AI agent instance status and user speaking status changes in real-time on the client interface to enhance user experience. You can obtain these statuses by listening to corresponding client or server callback events.

Status messages include the following types:

• AI agent instance creation and destruction status: AI agent instance created successfully, AI agent instance destroyed successfully.
• AI agent instance status events: Idle, Listening, Thinking, Speaking.
• User speaking status events: Start speaking, Stop speaking.

Quick Implementation

To display user and AI agent instance status, there are two methods:

1. Listen to ZEGOCLOUD Express SDK experimental API callbacks on the client.
2. Listen to AI agent instance status and user speaking status events through server callbacks, then send custom messages to the client through your own signaling channel (such as WebSocket, etc.).

Prerequisites

1. AI Agent service is enabled
2. ZEGOCLOUD Express SDK is initialized and joined the RTC room
3. The server has called the Create Agent Instance API

Listen to ZEGOCLOUD Express SDK Experimental API Callbacks

By listening to ZEGOCLOUD Express SDK experimental API callbacks (onRecvExperimentalAPI or recvExperimentalAPI) on the client SDK, you can handle corresponding status events based on message type. Below are example codes for each platform:

Listen to Agent Instance Status and User Speaking Status Events Through Server Callbacks, Then Send Custom Messages to Clients Through Your Own Signaling Channel

If your application already has its own signaling channel, such as WebSocket or instant messaging system, you can:

• After receiving status event callbacks on the server, forward the status information to relevant clients through your signaling channel.
• Agree on the message format with the client, and the client updates the UI interface based on the received status information (such as displaying speaking indicators, animations, etc.).

The advantage of this method is that you can fully control the message format and transmission logic, which is suitable for applications with mature signaling systems.

Listen for Server Callbacks

Please refer to the Receiving Callback documentation to develop callbacks for receiving AI Agent event notifications, and contact ZEGOCLOUD technical support to configure the callback address.

Callback content samples are as follows:

{
    "AppId": 1234567,
    "AgentInstanceId": "1912124734317838336",
    "Data": {
// !mark
        "Status": "LISTENING",// IDLE: Idle LISTENING: Listening THINKING: Thinking SPEAKING: Speaking
    },
// !mark
    "Event": "AgentInstanceStatus",
    "Nonce": "7450395512627324902",
    "Signature": "fd9c1ce54e85bd92f48b0a805e82a52b0c0c6445",
    "Timestamp": 1745502313000,
    "AgentUserId": "123456789",
    "RoomId": "123456789",
    "Sequence": 123456789,
}

{
    "AppId": 1234567,
    "AgentInstanceId": "1912124734317838336",
    "Data": {
// !mark
        "Status": "LISTENING",// IDLE: Idle LISTENING: Listening THINKING: Thinking SPEAKING: Speaking
    },
// !mark
    "Event": "AgentInstanceStatus",
    "Nonce": "7450395512627324902",
    "Signature": "fd9c1ce54e85bd92f48b0a805e82a52b0c0c6445",
    "Timestamp": 1745502313000,
    "AgentUserId": "123456789",
    "RoomId": "123456789",
    "Sequence": 123456789,
}