Quick Start Voice Call
Quick Start Digital Human Video Call
Digital Human Introduction
Quick Start Digital Human Live Broadcast
This document explains how to call AI Agent related backend APIs to implement digital human live broadcast.
Unlike digital human video calls, digital human broadcasting is a one-way viewing scenario — users do not interact with the digital human, but only watch the broadcast content. The client only needs to log in to the RTC room and play stream of the digital human, without capturing or publishing stream of the user's audio and video. After creating a broadcast instance, the server can proactively make the digital human broadcast specified text via the TTS API.
Suitable for scenarios such as digital human live streaming, news broadcasting, event hosting, and scripted broadcast.
Differences from Digital Human Video Calls
| Item | Digital Human Video Call | Digital Human Live Broadcast |
|---|---|---|
| Interaction Mode | Two-way, the digital human responds after the user speaks | One-way, users only watch the digital human broadcast content |
| Use Cases | AI interaction, AI customer service, digital human tutor | Digital human live streaming, news broadcasting |
| Client Requires User Audio Capture | Yes | No |
| Digital Human Driving Mechanism | After receiving user audio, the LLM generates a response and drives the digital human via TTS | Broadcast content is fully controlled by the server; the digital human is driven via TTS |
| Instance Creation API | CreateDigitalHumanAgentInstance | CreateLiveDigitalHumanAgentInstance |
| Requires user_id and user_stream_id | Yes | No |
Prerequisites
- You have created a project in the ZEGOCLOUD Console and obtained a valid AppID and ServerSecret (for server API signing and RTC Token04 generation). For details, see Console - Project Info.
- You have contacted ZEGOCLOUD Technical Support to enable the AI Agent related services and obtained LLM and TTS related configuration information. For details, see Console - AI Agent.
- You have obtained a valid
digital_human_id(for testing, you can use the public ID: 63c3aa64-1d80-4b04-a0be-1c65614eb7eb).
During the test period (within 2 weeks after the AI Agent service is enabled), you can set the LLM and TTS authentication parameters to "zego_test" to use the related services. For details, see Register Agent > BODY Parameter Description.
You can also purchase LLM and TTS services supported by ZEGOCLOUD to obtain authentication information. Alternatively, you can contact ZEGOCLOUD sales to purchase TTS services directly.
Example Code
The following is the example code for the business backend that integrates the real-time interactive AI Agent API. You can refer to the example code to implement your own business logic.
Includes the basic capabilities of obtaining ZEGO Token, registering AI Agent, creating AI Agent instances, and deleting AI Agent instances.
The following is the client sample code. You can refer to the example code to implement your own business logic.
Includes the basic capabilities of logging in, publishing stream, playing stream, and exiting rooms.
Includes the basic capabilities of logging in, publishing stream, playing stream, and exiting rooms.
Includes the basic capabilities of logging in, publishing stream, playing stream, and exiting rooms.
Includes the basic capabilities of logging in, publishing stream, playing stream, and exiting rooms.
Overall Business Process
- Service backend: run the business backend example code and deploy the business backend.
- Integrate the real-time interactive AI Agent API to manage AI Agents.
- Client: refer to the Android Quick Start, iOS Quick Start, or Web Quick Start document to run the client example code.
- Create and manage broadcast digital human instances through the business backend.
- Integrate ZEGO Express SDK for joining rooms and playing streams.
- Android/iOS: Integrate Digital Human SDK for digital human rendering.
- Call the TTS API on demand to proactively broadcast via the digital human.
After completing the above two steps, you can watch the digital human broadcast.
Core Implementation
Register Agent
Register Agent is used to set the basic configuration of the AI Agent, including the agent name, LLM, TTS, ASR, and other related configurations. After registration, the agent can be used as a template to create multiple instances for real-time interaction with multiple real users.
Typically, the agent configuration is relatively fixed. Once the relevant parameters (persona) are set, they do not change frequently. Therefore, it is recommended to register the agent at the appropriate time according to your business flow. The agent will not be automatically destroyed or recycled after registration. Once an agent instance is created, you can interact with the agent via voice.
The following is an example of calling the Register Agent API:
// Please replace the LLM and TTS authentication parameters (ApiKey, appid, token, etc.) in the following example with your actual authentication parameters.
async registerAgent(agentId: string, agentName: string) {
// Request URL: https://aigc-aiagent-api.zegotech.cn?Action=RegisterAgent
const action = 'RegisterAgent';
const body = {
AgentId: agentId,
Name: agentName,
LLM: {
Url: "https://ark.cn-beijing.volces.com/api/v3/chat/completions",
ApiKey: "zego_test",
Model: "doubao-1-5-pro-32k-250115",
SystemPrompt: "You are an AI Agent. Please answer the user's questions."
},
TTS: {
Vendor: "ByteDance",
Params: {
"app": {
"appid": "zego_test",
"token": "zego_test",
"cluster": "volcano_tts"
},
"audio": {
"voice_type": "zh_female_wanwanxiaohe_moon_bigtts"
}
}
}
};
// The sendRequest method encapsulates the request URL and common parameters. For details, see: /aiagent-server/api-reference/accessing-server-apis
return this.sendRequest<any>(action, body);
} - Please ensure all LLM parameters are correctly filled in according to the LLM service provider's official documentation. Otherwise, you may not be able to see the agent's text responses or hear the agent's voice output.
- Please ensure all TTS parameters are correctly filled in according to the TTS service provider's official documentation. Otherwise, you may see the agent's text responses but not hear the agent's voice output.
- If the agent cannot output text or voice, please first check whether the LLM and TTS parameter configurations are completely correct, or refer to Get AI Agent Service Status - Monitor Server Exception Events to identify the specific issue.
Obtain RTC Token
The client needs a Token to log in to the RTC room. The Token04 can be generated by the business backend based on the AppID and ServerSecret.
GET /api/zego-token?user_id=<user_id> Response example:
{
"code": 0,
"token": "***"
} Create Broadcast Digital Human Instance
You can use a registered agent as a template to create a broadcast digital human instance. Unlike digital human video calls, broadcast digital human only needs to join the RTC room and publish the digital human stream, without receiving the real user's stream. Therefore, UserId and UserStreamId are not required when calling the creation API.
The client calls the business backend's POST /api/start-live-digital-human. In RTC mode, the request body example is as follows:
{
"digital_human_id": "digital_human_id",
"config_id": "mobile",
"room_id": "room_id"
} After receiving the request, the business backend calls CreateLiveDigitalHumanAgentInstance to create the instance, and returns the AgentInstanceId, AgentStreamId, AgentUserId, and DigitalHumanConfig to the client after conversion. The client saves the AgentInstanceId for subsequent calls to SendAgentInstanceTTS and deleting the instance.
The following is an example of calling the Create Broadcast Digital Human Instance API:
async createLiveDigitalHumanAgentInstance(agentId: string, rtcInfo: RtcInfo, digitalHuman: DigitalHumanInfo) {
// Request URL: https://aigc-aiagent-api.zegotech.cn?Action=CreateLiveDigitalHumanAgentInstance
const action = 'CreateLiveDigitalHumanAgentInstance';
const body = {
AgentId: agentId,
RTC: rtcInfo, // RTC mode
DigitalHuman: digitalHuman, // For testing, you can use the public ID: 63c3aa64-1d80-4b04-a0be-1c65614eb7eb
};
const result = await this.sendRequest<any>(action, body);
return {
code: 0,
message: 'success',
agent_instance_id: result.AgentInstanceId,
agent_stream_id: result.AgentStreamId,
agent_user_id: result.AgentUserId,
digital_human_config: result.DigitalHumanConfig
};
} Proactively Invoke TTS
After creating a broadcast digital human instance, the business backend can call SendAgentInstanceTTS to proactively send broadcast text.
async sendAgentInstanceTTS(agentInstanceId: string, text: string) {
const action = 'SendAgentInstanceTTS';
const body = {
AgentInstanceId: agentInstanceId,
Text: text
};
return this.sendRequest<any>(action, body);
} If the business backend exposes a quick-start server API, you can use:
POST /api/send-agent-instance-tts
Content-Type: application/json Request example:
{
"agent_instance_id": "1912124734317838336",
"text": "Hello developer, welcome to ZEGO RTC. Let's build the real-time interactive world together."
} - The maximum length of
Textis 300 characters. - Optional parameters:
AddHistory,Priority,SamePriorityOption. For details, see Proactively Invoke LLM or TTS and SendAgentInstanceTTS. - By default, the instance is automatically destroyed after 900 seconds of idle time without calling
SendAgentInstanceTTS. For long live broadcasts, you can adjust this viaAdvancedConfig.MaxIdleTime(30~86400 seconds).
Stop Broadcast and Delete Instance
After the broadcast ends, the business backend calls POST /api/stop to stop and delete the broadcast instance by passing in the agent_instance_id. After deletion, the digital human will automatically exit the room and stop publishing stream.
{
"agent_instance_id": "1912124734317838336"
} Display User and Agent Status
To display user and agent status, please refer to Display User and Agent Status.
Status descriptions for broadcast digital human:
- Idle: The digital human instance has been created successfully and is waiting for the server to input driving content.
- Speaking: The digital human has received driving content from the server and is broadcasting in real time. It automatically switches back to "Idle" after the broadcast ends.
Integrate Client SDK
Please refer to the following documents to complete client integration development:
Quick Start
Quick Start
Quick Start
Congratulations 🎉! After completing this step, you have successfully integrated the client SDK and can watch the digital human broadcast. You can also proactively trigger digital human broadcast via the TTS API.
Delete Agent Instance
After deleting the agent instance, the broadcast digital human will automatically exit the room and stop publishing stream. After the client stops playing stream and exits the room, a complete broadcast session ends.
The following is an example of calling the Delete Agent Instance API:
async deleteAgentInstance(agentInstanceId: string) {
// Request URL: https://aigc-aiagent-api.zegotech.cn?Action=DeleteAgentInstance
const action = 'DeleteAgentInstance';
const body = {
AgentInstanceId: agentInstanceId
};
// The sendRequest method encapsulates the request URL and common parameters. For details, see: /aiagent-server/api-reference/accessing-server-apis
return this.sendRequest(action, body);
} The above is the complete core flow for implementing digital human live broadcast.
Callback Monitoring
Click to view the callback monitoring guide. Monitor events where Event is Exception in the callbacks. You can quickly locate issues through Data.Code and Data.Message.
Click to view the exception error code list.
Broadcast Troubleshooting Checklist
- Frozen screen: Check whether
digital_human_configis valid, whetheragent_stream_idis correct, whether custom rendering is enabled beforestartPlayingStream(Android/iOS), and whether video frames and SEI data are passed to the Digital Human SDK (Android/iOS). - TTS not broadcasting: Check whether the Code returned by
SendAgentInstanceTTSis 0, whetherMaxIdleTimehas expired, and whetherDisableTTSis set. - Failed to create instance: Check whether
digital_human_idis valid, and whetherAppIDandServerSecretare correct.
