Display User and Agent Status

During real-time voice calls with AI agents, you may need to display the speaking status changes of both the AI agent instance and users in real-time on the client interface to enhance user experience. You can obtain these statuses by listening to corresponding server callback events.

Status messages include the following types:

AI agent instance speaking status events: Start speaking, Stop speaking.
User speaking status events: Start speaking, Stop speaking.

Quick Implementation

Listen for Server Callbacks

Please refer to the Receiving Callback documentation to develop callbacks for receiving AI Agent event notifications, and contact ZEGOCLOUD technical support to configure the callback address.

Note

To receive user and agent status callback results, when creating an agent instance, set the CallbackConfig.UserSpeakAction and CallbackConfig.AgentSpeakAction parameters to 1.

Callback content samples:

Agent speaking status callback

User speaking status callback

{
    "AppId": 1234567,
    "AgentInstanceId": "1912124734317838336",
    "Data": {
        "Action": "SPEAK_BEGIN",// SPEAK_BEGIN: Start speaking SPEAK_END: Stop speaking
    },
    "Event": "AgentSpeakAction",
    "Nonce": "7450395512627324902",
    "Signature": "fd9c1ce54e85bd92f48b0a805e82a52b0c0c6445",
    "Timestamp": 1745502313000,
    "AgentUserId": "123456789",
    "RoomId": "123456789",
    "Sequence": 123456789,
}

Copied!

{
    "AppId": 1234567,
    "AgentInstanceId": "1912124734317838336",
    "Data": {
        "Action": "SPEAK_BEGIN",// SPEAK_BEGIN: Start speaking SPEAK_END: Stop speaking
    },
    "Event": "UserSpeakAction",
    "Nonce": "7450395512627324902",
    "Signature": "fd9c1ce54e85bd92f48b0a805e82a52b0c0c6445",
    "Timestamp": 1745502313000,
    "AgentUserId": "123456789",
    "RoomId": "123456789",
    "Sequence": 123456789,
}

Copied!

How to Notify Clients and Display Status

After receiving speaking status events for AI agent instances or users through server callbacks, you can notify clients of these status updates to enable real-time status display. Here are two common notification methods:

Using Your Own Signaling Channel

If your application already has its own signaling channel, such as WebSocket or instant messaging system, you can:

Forward status information to relevant clients through your signaling channel after receiving status event callbacks on the server.
Agree on a message format with clients, so they can update their UI (e.g., display speaking indicators, animations) based on received status information.

The advantage of this approach is complete control over message format and transmission logic, making it suitable for applications with mature signaling systems.

Using ZEGOCLOUD RTC Room Message Channel for Custom Messages

If you don't have your own business signaling channel, you can utilize ZEGOCLOUD RTC's room messaging feature:

Call ZEGOCLOUD RTC Server API to send custom messages after receiving status event callbacks on the server
Agree on a message format with clients, so they can listen for custom messages via ZEGOCLOUD RTC SDK and update their UI based on status change notifications (e.g., display speaking indicators, animations)

The advantage of this approach is that no additional signaling system setup is required, as you can directly use ZEGOCLOUD's infrastructure. However, the disadvantage is that room messages are not guaranteed to be completely reliable and have sending frequency limitations, making them unsuitable for scenarios requiring high message reliability.

Implementation examples are as follows:

Server

Client