AI Agent(iOS) Quick Start Digital Human Live Broadcast

This document describes how to quickly integrate the client SDKs (ZEGO Express SDK and Digital Human SDK) to implement live digital human broadcasting.
Unlike digital human video calls, digital human broadcasting is a one-way viewing scenario - users do not interact with the digital human, they simply play stream the digital human's broadcast content for viewing.
The client only needs to log in to an RTC room and play stream; it does not need to capture or publish stream the user's audio and video. After creating a broadcast instance, the server can proactively make the digital human broadcast specified text via the TTS API.
Suitable for scenarios such as digital human live streaming, news broadcasting, event hosting, and scripted broadcasts.

This document describes how to quickly integrate the client SDK (ZEGO Express SDK) to implement live digital human broadcasting.
Unlike digital human video calls, digital human broadcasting is a one-way viewing scenario - users do not interact with the digital human, they simply play stream the digital human's broadcast content for viewing.
The client only needs to log in to an RTC room and play stream; it does not need to capture or publish stream the user's audio and video. After creating a broadcast instance, the server can proactively make the digital human broadcast specified text via the TTS API.
Suitable for scenarios such as digital human live streaming, news broadcasting, event hosting, and scripted broadcasts.

Item	Digital Human Video Call	Live Digital Human Broadcasting
Interaction mode	Two-way: the digital human responds after the user speaks	One-way: users only view the digital human's broadcast content
Use cases	Conversational AI, AI customer service, digital human tutor	Digital human live streaming, news broadcasting
Does the client need to capture user audio?	Yes	No
Digital human driving mechanism	After receiving user audio, LLM outputs a response and drives the digital human via TTS	Broadcast content is fully controlled by the server, driving the digital human via TTS
Instance creation API	CreateDigitalHumanAgentInstance	CreateLiveDigitalHumanAgentInstance
Requires user_id, user_stream_id	Yes	No

You have downloaded the ZEGO Express SDK optimized for AI Agent from the download page and integrated it into your project.

You have contacted ZEGOCLOUD Technical Support to obtain the ZEGO Express SDK optimized for AI Agent and integrated it into your project.

The following video demonstrates how to run through the server and client (Web) sample code and interact with the agent via voice.

The following video demonstrates how to run through the server and client (iOS) sample code and interact with the agent via voice.

On the client side, run through the sample code.
- Create and manage agents through the business backend.
- Integrate ZEGO Express SDK for real-time communication.

On the client side, run through the sample code.
- Create and manage agents through the business backend.
- Integrate ZEGO Express SDK and the Digital Human SDK for real-time communication.

Please refer to Integrate SDK > 2.2 > Method 2 to manually integrate the SDK. The sample project uses ZegoExpressEngine and ZegoDigitalMobile.

Add the Digital Human SDK in app/build.gradle:

app/build.gradle

dependencies {
    implementation 'im.zego:digitalmobile:1.3.0.43'
}

app/build.gradle

dependencies {
    implementation 'im.zego:digitalmobile:1.3.0.43'
}

Declare network permissions in AndroidManifest.xml. The broadcast digital human scenario does not require declaring or requesting RECORD_AUDIO:

AndroidManifest.xml

<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.INTERNET" />

AndroidManifest.xml

<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.INTERNET" />

The broadcast scenario does not require runtime microphone permission. After entering the page, you can directly initialize the ZEGO Express SDK:

ZegoEngineProfile profile = new ZegoEngineProfile();
profile.appID = appID; // Obtain from ZEGOCLOUD Console
// !mark
profile.scenario = ZegoScenario.HIGH_QUALITY_CHATROOM;
profile.application = getApplication();
ZegoExpressEngine.createEngine(profile, null);

ZegoEngineProfile profile = new ZegoEngineProfile();
profile.appID = appID; // Obtain from ZEGOCLOUD Console
// !mark
profile.scenario = ZegoScenario.HIGH_QUALITY_CHATROOM;
profile.application = getApplication();
ZegoExpressEngine.createEngine(profile, null);

The iOS quickstart integrates the SDK via CocoaPods:

Podfile

target 'ai_agent_quickstart' do
  use_frameworks! :linkage => :static
  use_modular_headers!

  pod 'ZegoExpressEngine', :path => 'libs/Express'
  pod 'Masonry', '1.1.0'
  pod 'ZegoDigitalMobile', '>= 1.3.0'
end

Podfile

target 'ai_agent_quickstart' do
  use_frameworks! :linkage => :static
  use_modular_headers!

  pod 'ZegoExpressEngine', :path => 'libs/Express'
  pod 'Masonry', '1.1.0'
  pod 'ZegoDigitalMobile', '>= 1.3.0'
end

Digital human broadcasting is a one-way viewing scenario. You do not need to add NSMicrophoneUsageDescription in Info.plist or call requestRecordPermission:. If the same app also supports digital human video calls, you can keep the microphone permission declaration required for video calls.

When initializing the broadcast page, call ZegoExpressEngine directly without requesting microphone permission:

- (void)initZegoExpressEngine {
    ZegoEngineProfile *profile = [[ZegoEngineProfile alloc] init];
    profile.appID = appID; // Obtain from ZEGOCLOUD Console
    profile.scenario = ZegoScenarioHighQualityChatroom;
    [ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];
}

- (void)initZegoExpressEngine {
    ZegoEngineProfile *profile = [[ZegoEngineProfile alloc] init];
    profile.appID = appID; // Obtain from ZEGOCLOUD Console
    profile.scenario = ZegoScenarioHighQualityChatroom;
    [ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];
}

The Web quickstart uses zego-express-engine-webrtc. The broadcast entry does not create audio streams or call startPublishingStream, so no microphone permission is needed:

npm install zego-express-engine-webrtc

npm install zego-express-engine-webrtc

import { ZegoExpressEngine } from "zego-express-engine-webrtc";

// appID: number, obtained from ZEGOCLOUD Console project information
// server: signaling server address; for ZEGO Express SDK 3.7.0 and above, you can use the Server address from the console or leave it as an empty string
const zg = new ZegoExpressEngine(appID, "");

import { ZegoExpressEngine } from "zego-express-engine-webrtc";

// appID: number, obtained from ZEGOCLOUD Console project information
// server: signaling server address; for ZEGO Express SDK 3.7.0 and above, you can use the Server address from the console or leave it as an empty string
const zg = new ZegoExpressEngine(appID, "");

Below are commonly used variable declarations/configurations for the Web client:

// Obtain from ZEGOCLOUD Console
const APP_ID = 1234567890; // AppID, number type
const SERVER = ""; // Signaling server address, can be empty string for 3.7.0+
// Obtain from business backend
const BASE_URL = "http://your-server-host:3000"; // Business backend address
const ROOM_ID = "room_xxx"; // RTC room ID
const USER_ID = "user_xxx"; // User ID
const DIGITAL_HUMAN_ID = "63c3aa64-1d80-4b04-a0be-1c65614eb7eb"; // Digital human ID
const CONFIG_ID = "web"; // Use "web" for Web client


// Module-level variables, used by roomStreamUpdate / remoteCameraStatusUpdate / exit
let agentStreamId = "";
let agentInstanceId = "";

// Obtain from ZEGOCLOUD Console
const APP_ID = 1234567890; // AppID, number type
const SERVER = ""; // Signaling server address, can be empty string for 3.7.0+
// Obtain from business backend
const BASE_URL = "http://your-server-host:3000"; // Business backend address
const ROOM_ID = "room_xxx"; // RTC room ID
const USER_ID = "user_xxx"; // User ID
const DIGITAL_HUMAN_ID = "63c3aa64-1d80-4b04-a0be-1c65614eb7eb"; // Digital human ID
const CONFIG_ID = "web"; // Use "web" for Web client


// Module-level variables, used by roomStreamUpdate / remoteCameraStatusUpdate / exit
let agentStreamId = "";
let agentInstanceId = "";

const loginResult = await zg.loginRoom(roomID, token, {
    userID,
    userName,
});

if (!loginResult) {
  throw new Error("Failed to log in to RTC room");
}

const loginResult = await zg.loginRoom(roomID, token, {
    userID,
    userName,
});

if (!loginResult) {
  throw new Error("Failed to log in to RTC room");
}

The current quickstart's common initialization logic still calls checkSystemRequirements to check WebRTC and microphone capabilities for compatibility with the voice call entry. If your Web app only implements digital human broadcasting, you can remove the microphone check.

The Digital Human SDK has been published to the Maven repository. Refer to the following steps to integrate it into your project.

1

Add `maven` configuration

Choose the appropriate steps based on your Android Gradle plugin version.

2

Modify your app-level build.gradle file

dependencies {
    ...
    // Digital Human SDK dependency
    implementation "im.zego:digitalmobile:1.3.0.43"
}

dependencies {
    ...
    // Digital Human SDK dependency
    implementation "im.zego:digitalmobile:1.3.0.43"
}

Note

Supports Android 6.0 (API 23) and above.

Note

Supports iOS 12 and above.

Field	Description
`digital_human_id`	Digital human ID, used to specify the digital human appearance.
`encode_code`	Encoding type. Only supports `mobile` (Android/iOS) and `web` (Web). Android and iOS use `mobile`, Web uses `web`.

Field	Purpose
`agent_instance_id`	For calling the proactive TTS and stop instance APIs
`agent_stream_id`	The ID of the digital human stream in the RTC room
`agent_user_id`	The digital human's user ID in the RTC room
`digital_human_config`	Configuration for initializing the Android/iOS Digital Human SDK

Android uses OkHttp to directly request the business backend:

private void startLiveDigitalHuman(String baseUrl, String digitalHumanId,
                                   String configId, String roomId) {
    JSONObject bodyJson = new JSONObject();
    bodyJson.put("digital_human_id", digitalHumanId);
    bodyJson.put("config_id", configId);
    bodyJson.put("room_id", roomId);

    RequestBody body = RequestBody.create(
        bodyJson.toString(), MediaType.parse("application/json; charset=utf-8"));
    Request request = new Request.Builder()
        .url(baseUrl + "/api/start-live-digital-human")
        .post(body)
        .build();

    new OkHttpClient().newCall(request).enqueue(new Callback() {
        @Override
        public void onFailure(@NonNull Call call, @NonNull IOException e) {
            // Handle network error
        }

        @Override
        public void onResponse(@NonNull Call call, @NonNull Response response)
            throws IOException {
            JSONObject result = new JSONObject(response.body().string());
            if (result.getInt("code") == 0) {
                String agentInstanceId = result.getString("agent_instance_id");
                String agentStreamId = result.getString("agent_stream_id");
                String digitalHumanConfig = result.getString("digital_human_config");
                // Save agentInstanceId for subsequent TTS and stop calls
                startPlayingStream(agentStreamId);
                initDigitalMobileSDK(digitalHumanConfig);
            }
        }
    });
}

private void startLiveDigitalHuman(String baseUrl, String digitalHumanId,
                                   String configId, String roomId) {
    JSONObject bodyJson = new JSONObject();
    bodyJson.put("digital_human_id", digitalHumanId);
    bodyJson.put("config_id", configId);
    bodyJson.put("room_id", roomId);

    RequestBody body = RequestBody.create(
        bodyJson.toString(), MediaType.parse("application/json; charset=utf-8"));
    Request request = new Request.Builder()
        .url(baseUrl + "/api/start-live-digital-human")
        .post(body)
        .build();

    new OkHttpClient().newCall(request).enqueue(new Callback() {
        @Override
        public void onFailure(@NonNull Call call, @NonNull IOException e) {
            // Handle network error
        }

        @Override
        public void onResponse(@NonNull Call call, @NonNull Response response)
            throws IOException {
            JSONObject result = new JSONObject(response.body().string());
            if (result.getInt("code") == 0) {
                String agentInstanceId = result.getString("agent_instance_id");
                String agentStreamId = result.getString("agent_stream_id");
                String digitalHumanConfig = result.getString("digital_human_config");
                // Save agentInstanceId for subsequent TTS and stop calls
                startPlayingStream(agentStreamId);
                initDigitalMobileSDK(digitalHumanConfig);
            }
        }
    });
}

iOS uses NSURLSession to directly request the business backend and saves the instance ID, stream ID, and digital human config from the response:

- (void)startLiveDigitalHuman {
    NSURL *url = [NSURL URLWithString:
        [baseURL stringByAppendingString:@"/api/start-live-digital-human"]];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
    request.HTTPMethod = @"POST";
    [request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];

    NSDictionary *params = @{
        @"digital_human_id": digitalHumanId,
        @"config_id": @"mobile",
        @"room_id": roomID,
    };
    request.HTTPBody = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];

    [[[NSURLSession sharedSession] dataTaskWithRequest:request
        completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        NSDictionary *result = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
        if (error == nil && [result[@"code"] integerValue] == 0) {
            agentInstanceId = result[@"agent_instance_id"];
            agentStreamId = result[@"agent_stream_id"];
            NSString *config = result[@"digital_human_config"];
            [self initDigitalMobileSDK:config];
        }
    }] resume];
}

- (void)startLiveDigitalHuman {
    NSURL *url = [NSURL URLWithString:
        [baseURL stringByAppendingString:@"/api/start-live-digital-human"]];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
    request.HTTPMethod = @"POST";
    [request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];

    NSDictionary *params = @{
        @"digital_human_id": digitalHumanId,
        @"config_id": @"mobile",
        @"room_id": roomID,
    };
    request.HTTPBody = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];

    [[[NSURLSession sharedSession] dataTaskWithRequest:request
        completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        NSDictionary *result = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
        if (error == nil && [result[@"code"] integerValue] == 0) {
            agentInstanceId = result[@"agent_instance_id"];
            agentStreamId = result[@"agent_stream_id"];
            NSString *config = result[@"digital_human_config"];
            [self initDigitalMobileSDK:config];
        }
    }] resume];
}

Web uses fetch to directly request the business backend. The example only passes the RTC room ID, so no local user or local stream information is sent:

async function startLiveDigitalHuman(roomId: string) {
  const response = await fetch(`${baseURL}/api/start-live-digital-human`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
    digital_human_id: config.digitalHuman.id,
    config_id: config.digitalHuman.configId,
    room_id: roomId,
    }),
  });
  const result = await response.json();
  if (result.code !== 0) throw new Error(result.message);
  return result;
}

async function startLiveDigitalHuman(roomId: string) {
  const response = await fetch(`${baseURL}/api/start-live-digital-human`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
    digital_human_id: config.digitalHuman.id,
    config_id: config.digitalHuman.configId,
    room_id: roomId,
    }),
  });
  const result = await response.json();
  if (result.code !== 0) throw new Error(result.message);
  return result;
}

Android needs to first obtain the Token from the business backend using OkHttp, then call the ZEGO Express SDK to log in to the room. After successful login, enable custom video rendering, then create a broadcast digital human instance:

String tokenUrl = baseUrl + "/api/zego-token?userId=" + userId;
Request tokenRequest = new Request.Builder().url(tokenUrl).get().build();
new OkHttpClient().newCall(tokenRequest).enqueue(new Callback() {
    @Override
    public void onResponse(@NonNull Call call, @NonNull Response response)
        throws IOException {
        String token = new JSONObject(response.body().string()).getString("token");

        ZegoEngineConfig engineConfig = new ZegoEngineConfig();
        engineConfig.advancedConfig = new HashMap<String, String>() {{
            //===== Digital human specific ====//
            put("set_audio_volume_ducking_mode", "1");           // Audio volume ducking toggle, enabled by default
            put("enable_rnd_volume_adaptive", "true");           // Playback volume adaptive toggle, enabled by default
            put("sideinfo_callback_version", "3");               // Make SEI and frame correspond one-to-one, for ZEGO Express SDK 3.17
            put("sideinfo_bound_to_video_decoder", "true");      // Make SEI and frame correspond one-to-one, for ZEGO Express SDK 3.18
        }};
        ZegoExpressEngine.setEngineConfig(engineConfig);

        ZegoRoomConfig roomConfig = new ZegoRoomConfig();
        roomConfig.isUserStatusNotify = true;
        roomConfig.token = token;
        ZegoExpressEngine.getEngine().loginRoom(
            roomId, new ZegoUser(userId, userId), roomConfig,
            (errorCode, extendedData) -> {
                if (errorCode == 0) {
                    openExpressCustomRender();
                    startLiveDigitalHuman(baseUrl, digitalHumanId, "mobile", roomId);
                }
            });
    }

    @Override
    public void onFailure(@NonNull Call call, @NonNull IOException e) {
        // Handle Token retrieval failure
    }
});

String tokenUrl = baseUrl + "/api/zego-token?userId=" + userId;
Request tokenRequest = new Request.Builder().url(tokenUrl).get().build();
new OkHttpClient().newCall(tokenRequest).enqueue(new Callback() {
    @Override
    public void onResponse(@NonNull Call call, @NonNull Response response)
        throws IOException {
        String token = new JSONObject(response.body().string()).getString("token");

        ZegoEngineConfig engineConfig = new ZegoEngineConfig();
        engineConfig.advancedConfig = new HashMap<String, String>() {{
            //===== Digital human specific ====//
            put("set_audio_volume_ducking_mode", "1");           // Audio volume ducking toggle, enabled by default
            put("enable_rnd_volume_adaptive", "true");           // Playback volume adaptive toggle, enabled by default
            put("sideinfo_callback_version", "3");               // Make SEI and frame correspond one-to-one, for ZEGO Express SDK 3.17
            put("sideinfo_bound_to_video_decoder", "true");      // Make SEI and frame correspond one-to-one, for ZEGO Express SDK 3.18
        }};
        ZegoExpressEngine.setEngineConfig(engineConfig);

        ZegoRoomConfig roomConfig = new ZegoRoomConfig();
        roomConfig.isUserStatusNotify = true;
        roomConfig.token = token;
        ZegoExpressEngine.getEngine().loginRoom(
            roomId, new ZegoUser(userId, userId), roomConfig,
            (errorCode, extendedData) -> {
                if (errorCode == 0) {
                    openExpressCustomRender();
                    startLiveDigitalHuman(baseUrl, digitalHumanId, "mobile", roomId);
                }
            });
    }

    @Override
    public void onFailure(@NonNull Call call, @NonNull IOException e) {
        // Handle Token retrieval failure
    }
});

iOS uses NSURLSession to obtain the Token, then calls loginRoom directly. The broadcast scenario does not call startPublishingStream:

- (void)loginRoomForBroadcast:(NSString *)token {
    ZegoRoomConfig *roomConfig = [[ZegoRoomConfig alloc] init];
    roomConfig.isUserStatusNotify = YES;
    roomConfig.token = token;

    ZegoUser *user = [[ZegoUser alloc] initWithUserID:userID userName:userID];
    [[ZegoExpressEngine sharedEngine] loginRoom:roomID
                                           user:user
                                         config:roomConfig
                                        callback:^(int errorCode, NSDictionary *extendedData) {
        if (errorCode == 0) {
            [self enableCustomVideoRender];
            [self startLiveDigitalHuman];
        }
    }];
}

- (void)loginRoomForBroadcast:(NSString *)token {
    ZegoRoomConfig *roomConfig = [[ZegoRoomConfig alloc] init];
    roomConfig.isUserStatusNotify = YES;
    roomConfig.token = token;

    ZegoUser *user = [[ZegoUser alloc] initWithUserID:userID userName:userID];
    [[ZegoExpressEngine sharedEngine] loginRoom:roomID
                                           user:user
                                         config:roomConfig
                                        callback:^(int errorCode, NSDictionary *extendedData) {
        if (errorCode == 0) {
            [self enableCustomVideoRender];
            [self startLiveDigitalHuman];
        }
    }];
}

Web logs in to the room directly. The broadcast scenario does not create audio streams or publish local streams:

// 1. Obtain Token from business backend
const tokenRes = await fetch(`${BASE_URL}/api/zego-token?userId=${USER_ID}&roomId=${ROOM_ID}`);
const { token } = await tokenRes.json();

// 2. Log in to RTC room
await zg.loginRoom(ROOM_ID, token, { userID: USER_ID, userName: USER_ID });

// 3. Create broadcast digital human instance
const result = await startLiveDigitalHuman(ROOM_ID);
// 4. Extract and save agent_stream_id and agent_instance_id for subsequent play stream, TTS, and exit
agentStreamId = result.agent_stream_id;
agentInstanceId = result.agent_instance_id;

// 1. Obtain Token from business backend
const tokenRes = await fetch(`${BASE_URL}/api/zego-token?userId=${USER_ID}&roomId=${ROOM_ID}`);
const { token } = await tokenRes.json();

// 2. Log in to RTC room
await zg.loginRoom(ROOM_ID, token, { userID: USER_ID, userName: USER_ID });

// 3. Create broadcast digital human instance
const result = await startLiveDigitalHuman(ROOM_ID);
// 4. Extract and save agent_stream_id and agent_instance_id for subsequent play stream, TTS, and exit
agentStreamId = result.agent_stream_id;
agentInstanceId = result.agent_instance_id;

Android initializes the Digital Human SDK. The example involves three view member variables that need to be declared and initialized in the layout beforehand:

digitalView: The container for the Digital Human SDK rendering output. The digital human is ultimately drawn on this View. It is passed to the Digital Human SDK via attach during initialization.
loadingView: A loading placeholder view displayed before the digital human's first frame is rendered. It is hidden in the first frame draw callback.
digitalPic: A static cover placeholder image. Similar to loadingView, it is hidden in the first frame draw callback to prevent the screen from staying on a static image.

private void initDigitalMobileSDK(String digitalHumanConfig) {
    digitalMobileSDK = ZegoDigitalHuman.create(this);
    digitalMobileSDK.start(digitalHumanConfig,
        new IZegoDigitalMobile.ZegoDigitalMobileListener() {
            @Override
            public void onSurfaceFirstFrameDraw() {
                loadingView.setVisibility(View.GONE);
                digitalPic.setVisibility(View.GONE);
            }
        });
    digitalMobileSDK.attach(digitalView);
}

private void initDigitalMobileSDK(String digitalHumanConfig) {
    digitalMobileSDK = ZegoDigitalHuman.create(this);
    digitalMobileSDK.start(digitalHumanConfig,
        new IZegoDigitalMobile.ZegoDigitalMobileListener() {
            @Override
            public void onSurfaceFirstFrameDraw() {
                loadingView.setVisibility(View.GONE);
                digitalPic.setVisibility(View.GONE);
            }
        });
    digitalMobileSDK.attach(digitalView);
}

In openExpressCustomRender, configure RAW_DATA and forward the callback data to the Digital Human SDK. onPlayerSyncRecvSEI is used to receive the SEI (Supplemental Enhancement Information) data parsed by the ZEGO Express SDK and forward it to the Digital Human SDK. The SEI carries additional information such as lip-sync and expressions needed for driving the digital human, which is critical for accurate lip-sync. For details on SEI, see SEI Advanced Features.

private void openExpressCustomRender() {
    ZegoCustomVideoRenderConfig renderConfig = new ZegoCustomVideoRenderConfig();
    renderConfig.bufferType = ZegoVideoBufferType.RAW_DATA;
    renderConfig.frameFormatSeries = ZegoVideoFrameFormatSeries.RGB;
    renderConfig.enableEngineRender = false;
    ZegoExpressEngine.getEngine().enableCustomVideoRender(true, renderConfig);

    ZegoExpressEngine.getEngine().setCustomVideoRenderHandler(
        new IZegoCustomVideoRenderHandler() {
            @Override
            public void onRemoteVideoFrameRawData(
                ByteBuffer[] data, int[] dataLength, ZegoVideoFrameParam param,
                String streamID) {
                IZegoDigitalMobile.ZegoVideoFrameParam digitalParam =
                    new IZegoDigitalMobile.ZegoVideoFrameParam();
                digitalParam.format =
                    IZegoDigitalMobile.ZegoVideoFrameFormat.getZegoVideoFrameFormat(
                        param.format.value());
                digitalParam.height = param.height;
                digitalParam.width = param.width;
                digitalParam.rotation = param.rotation;
                for (int i = 0; i < 4; i++) {
                    digitalParam.strides[i] = param.strides[i];
                }

                if (digitalMobileSDK != null) {
                    digitalMobileSDK.onRemoteVideoFrameRawData(
                        data, dataLength, digitalParam, streamID);
                }
            }
        });

    ZegoExpressEngine.getEngine().setEventHandler(new IZegoEventHandler() {
        @Override
        public void onPlayerSyncRecvSEI(String streamID, byte[] data) {
            if (digitalMobileSDK != null) {
                digitalMobileSDK.onPlayerSyncRecvSEI(streamID, data);
            }
        }
    });
}

private void openExpressCustomRender() {
    ZegoCustomVideoRenderConfig renderConfig = new ZegoCustomVideoRenderConfig();
    renderConfig.bufferType = ZegoVideoBufferType.RAW_DATA;
    renderConfig.frameFormatSeries = ZegoVideoFrameFormatSeries.RGB;
    renderConfig.enableEngineRender = false;
    ZegoExpressEngine.getEngine().enableCustomVideoRender(true, renderConfig);

    ZegoExpressEngine.getEngine().setCustomVideoRenderHandler(
        new IZegoCustomVideoRenderHandler() {
            @Override
            public void onRemoteVideoFrameRawData(
                ByteBuffer[] data, int[] dataLength, ZegoVideoFrameParam param,
                String streamID) {
                IZegoDigitalMobile.ZegoVideoFrameParam digitalParam =
                    new IZegoDigitalMobile.ZegoVideoFrameParam();
                digitalParam.format =
                    IZegoDigitalMobile.ZegoVideoFrameFormat.getZegoVideoFrameFormat(
                        param.format.value());
                digitalParam.height = param.height;
                digitalParam.width = param.width;
                digitalParam.rotation = param.rotation;
                for (int i = 0; i < 4; i++) {
                    digitalParam.strides[i] = param.strides[i];
                }

                if (digitalMobileSDK != null) {
                    digitalMobileSDK.onRemoteVideoFrameRawData(
                        data, dataLength, digitalParam, streamID);
                }
            }
        });

    ZegoExpressEngine.getEngine().setEventHandler(new IZegoEventHandler() {
        @Override
        public void onPlayerSyncRecvSEI(String streamID, byte[] data) {
            if (digitalMobileSDK != null) {
                digitalMobileSDK.onPlayerSyncRecvSEI(streamID, data);
            }
        }
    });
}

iOS calls enableCustomVideoRender before startPlayingStream and forwards video frames and SEI in the digital human event handler:

- (BOOL)enableCustomVideoRender {
    ZegoCustomVideoRenderConfig *renderConfig =
        [[ZegoCustomVideoRenderConfig alloc] init];
    renderConfig.bufferType = ZegoVideoBufferTypeRawData;
    renderConfig.frameFormatSeries = ZegoVideoFrameFormatSeriesRGB;
    renderConfig.enableEngineRender = NO;

    ZegoExpressEngine *engine = [ZegoExpressEngine sharedEngine];
    if (!engine) {
        return NO;
    }

    [engine enableCustomVideoRender:YES config:renderConfig];
    [engine setCustomVideoRenderHandler:self];
    return YES;
}

- (void)onRemoteVideoFrameRawData:(unsigned char **)data
                       dataLength:(unsigned int *)dataLength
                            param:(ZegoVideoFrameParam *)param
                         streamID:(NSString *)streamID {
    ZDMVideoFrameParam *digitalParam = [[ZDMVideoFrameParam alloc] init];
    digitalParam.format = (ZDMVideoFrameFormat)param.format;
    digitalParam.width = param.size.width;
    digitalParam.height = param.size.height;
    digitalParam.rotation = param.rotation;

    for (int i = 0; i < 4; i++) {
        [digitalParam setStride:param.strides[i] atIndex:i];
    }

    if (self.digitalMobile) {
        [self.digitalMobile onRemoteVideoFrameRawData:data
                                           dataLength:dataLength
                                                param:digitalParam
                                             streamID:streamID];
    }
}

- (void)onPlayerSyncRecvSEI:(NSData *)data streamID:(NSString *)streamID {
    if (self.digitalMobile) {
        [self.digitalMobile onPlayerSyncRecvSEI:streamID data:data];
    }
}

- (BOOL)enableCustomVideoRender {
    ZegoCustomVideoRenderConfig *renderConfig =
        [[ZegoCustomVideoRenderConfig alloc] init];
    renderConfig.bufferType = ZegoVideoBufferTypeRawData;
    renderConfig.frameFormatSeries = ZegoVideoFrameFormatSeriesRGB;
    renderConfig.enableEngineRender = NO;

    ZegoExpressEngine *engine = [ZegoExpressEngine sharedEngine];
    if (!engine) {
        return NO;
    }

    [engine enableCustomVideoRender:YES config:renderConfig];
    [engine setCustomVideoRenderHandler:self];
    return YES;
}

- (void)onRemoteVideoFrameRawData:(unsigned char **)data
                       dataLength:(unsigned int *)dataLength
                            param:(ZegoVideoFrameParam *)param
                         streamID:(NSString *)streamID {
    ZDMVideoFrameParam *digitalParam = [[ZDMVideoFrameParam alloc] init];
    digitalParam.format = (ZDMVideoFrameFormat)param.format;
    digitalParam.width = param.size.width;
    digitalParam.height = param.size.height;
    digitalParam.rotation = param.rotation;

    for (int i = 0; i < 4; i++) {
        [digitalParam setStride:param.strides[i] atIndex:i];
    }

    if (self.digitalMobile) {
        [self.digitalMobile onRemoteVideoFrameRawData:data
                                           dataLength:dataLength
                                                param:digitalParam
                                             streamID:streamID];
    }
}

- (void)onPlayerSyncRecvSEI:(NSData *)data streamID:(NSString *)streamID {
    if (self.digitalMobile) {
        [self.digitalMobile onPlayerSyncRecvSEI:streamID data:data];
    }
}

The Web client does not integrate the Digital Human SDK and does not require custom rendering configuration. It directly uses the ZEGO Express SDK to play the digital human video stream.

The Android quickstart directly plays the agent_stream_id after the broadcast digital human instance is successfully created, without using onRoomStreamUpdate:

// Set play stream buffer interval range to mitigate stuttering caused by network jitter; min is the initial buffer (ms), max is the maximum buffer (ms)
ZegoExpressEngine.getEngine()
    .setPlayStreamBufferIntervalRange(agent_stream_id, 100, 2000);
ZegoExpressEngine.getEngine().startPlayingStream(agent_stream_id);

// Set play stream buffer interval range to mitigate stuttering caused by network jitter; min is the initial buffer (ms), max is the maximum buffer (ms)
ZegoExpressEngine.getEngine()
    .setPlayStreamBufferIntervalRange(agent_stream_id, 100, 2000);
ZegoExpressEngine.getEngine().startPlayingStream(agent_stream_id);

Listen for onRoomStreamUpdate and start playing the stream only when the agentStreamId returned by the server appears in the room:

- (void)startPlayStream:(NSString *)streamId {
    [[ZegoExpressEngine sharedEngine]
        setPlayStreamBufferIntervalRange:streamId min:0 max:2000];
    [[ZegoExpressEngine sharedEngine] startPlayingStream:streamId];
}

- (void)onRoomStreamUpdate:(ZegoUpdateType)updateType
                streamList:(NSArray<ZegoStream *> *)streamList
              extendedData:(nullable NSDictionary *)extendedData
                    roomID:(NSString *)roomID {
    if (updateType == ZegoUpdateTypeAdd) {
        for (ZegoStream *stream in streamList) {
            if ([stream.streamID isEqualToString:self.agentStreamId]) {
                [self startPlayStream:self.agentStreamId];
                break;
            }
        }
    } else if (updateType == ZegoUpdateTypeDelete) {
        for (ZegoStream *stream in streamList) {
            if ([stream.streamID isEqualToString:self.agentStreamId]) {
                [[ZegoExpressEngine sharedEngine] stopPlayingStream:stream.streamID];
            }
        }
    }
}

- (void)startPlayStream:(NSString *)streamId {
    [[ZegoExpressEngine sharedEngine]
        setPlayStreamBufferIntervalRange:streamId min:0 max:2000];
    [[ZegoExpressEngine sharedEngine] startPlayingStream:streamId];
}

- (void)onRoomStreamUpdate:(ZegoUpdateType)updateType
                streamList:(NSArray<ZegoStream *> *)streamList
              extendedData:(nullable NSDictionary *)extendedData
                    roomID:(NSString *)roomID {
    if (updateType == ZegoUpdateTypeAdd) {
        for (ZegoStream *stream in streamList) {
            if ([stream.streamID isEqualToString:self.agentStreamId]) {
                [self startPlayStream:self.agentStreamId];
                break;
            }
        }
    } else if (updateType == ZegoUpdateTypeDelete) {
        for (ZegoStream *stream in streamList) {
            if ([stream.streamID isEqualToString:self.agentStreamId]) {
                [[ZegoExpressEngine sharedEngine] stopPlayingStream:stream.streamID];
            }
        }
    }
}

Listen for roomStreamUpdate and create a remote stream view after matching the agentStreamId returned by the server. The agentStreamId comes from the agent_stream_id returned by the create instance API, which was extracted from the response after logging in to the room:

let remoteView: any = null;

zg.on(
  "roomStreamUpdate",
  async (
    roomID: string,
    updateType: "DELETE" | "ADD",
    streamList: ZegoStreamList[],
  ) => {
    if (updateType === "ADD" && streamList.length > 0) {
      for (const stream of streamList) {
        // Only play the digital human stream, filter out other streams in the room
        if (stream.streamID !== agentStreamId) continue;
        const mediaStream = await zg.startPlayingStream(stream.streamID);
        remoteView = await zg.createRemoteStreamView(mediaStream);
        remoteView?.playAudio();
        break;
      }
    }
  },
);

// The digital human video stream is published by the server. Camera status changing to OPEN indicates the video stream is ready, at which point video playback begins
zg.on(
  "remoteCameraStatusUpdate",
  (streamID: string, status: "OPEN" | "MUTE") => {
    if (streamID === agentStreamId && status === "OPEN") {
      remoteView?.playVideo("remoteStreamView");
    }
  }
);

let remoteView: any = null;

zg.on(
  "roomStreamUpdate",
  async (
    roomID: string,
    updateType: "DELETE" | "ADD",
    streamList: ZegoStreamList[],
  ) => {
    if (updateType === "ADD" && streamList.length > 0) {
      for (const stream of streamList) {
        // Only play the digital human stream, filter out other streams in the room
        if (stream.streamID !== agentStreamId) continue;
        const mediaStream = await zg.startPlayingStream(stream.streamID);
        remoteView = await zg.createRemoteStreamView(mediaStream);
        remoteView?.playAudio();
        break;
      }
    }
  },
);

// The digital human video stream is published by the server. Camera status changing to OPEN indicates the video stream is ready, at which point video playback begins
zg.on(
  "remoteCameraStatusUpdate",
  (streamID: string, status: "OPEN" | "MUTE") => {
    if (streamID === agentStreamId && status === "OPEN") {
      remoteView?.playVideo("remoteStreamView");
    }
  }
);

Android uses OkHttp to directly call the TTS API:

JSONObject bodyJson = new JSONObject();
bodyJson.put("agent_instance_id", agentInstanceId);
bodyJson.put("text", text);

Request request = new Request.Builder()
    .url(baseUrl + "/api/send-agent-instance-tts")
    .post(RequestBody.create(bodyJson.toString(),
        MediaType.parse("application/json; charset=utf-8")))
    .build();
new OkHttpClient().newCall(request).enqueue(new Callback() {
    @Override
    public void onResponse(@NonNull Call call, @NonNull Response response)
        throws IOException {
        JSONObject result = new JSONObject(response.body().string());
        if (result.getInt("code") != 0) {
            // Handle TTS request failure
        }
    }

    @Override
    public void onFailure(@NonNull Call call, @NonNull IOException e) {
        // Handle network error
    }
});

JSONObject bodyJson = new JSONObject();
bodyJson.put("agent_instance_id", agentInstanceId);
bodyJson.put("text", text);

Request request = new Request.Builder()
    .url(baseUrl + "/api/send-agent-instance-tts")
    .post(RequestBody.create(bodyJson.toString(),
        MediaType.parse("application/json; charset=utf-8")))
    .build();
new OkHttpClient().newCall(request).enqueue(new Callback() {
    @Override
    public void onResponse(@NonNull Call call, @NonNull Response response)
        throws IOException {
        JSONObject result = new JSONObject(response.body().string());
        if (result.getInt("code") != 0) {
            // Handle TTS request failure
        }
    }

    @Override
    public void onFailure(@NonNull Call call, @NonNull IOException e) {
        // Handle network error
    }
});

NSDictionary *params = @{
    @"agent_instance_id": agentInstanceId,
    @"text": text,
};
NSURL *url = [NSURL URLWithString:
    [baseURL stringByAppendingString:@"/api/send-agent-instance-tts"]];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
request.HTTPMethod = @"POST";
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
request.HTTPBody = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];

[[NSURLSession.sharedSession dataTaskWithRequest:request
    completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
    NSDictionary *result = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
    if (error == nil && [result[@"code"] integerValue] == 0) {
        NSLog(@"Broadcast sent successfully");
    }
}] resume];

NSDictionary *params = @{
    @"agent_instance_id": agentInstanceId,
    @"text": text,
};
NSURL *url = [NSURL URLWithString:
    [baseURL stringByAppendingString:@"/api/send-agent-instance-tts"]];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
request.HTTPMethod = @"POST";
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
request.HTTPBody = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];

[[NSURLSession.sharedSession dataTaskWithRequest:request
    completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
    NSDictionary *result = [NSJSONSerialization JSONObjectWithData:data options:0 error:nil];
    if (error == nil && [result[@"code"] integerValue] == 0) {
        NSLog(@"Broadcast sent successfully");
    }
}] resume];

const response = await fetch(`${baseURL}/api/send-agent-instance-tts`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    agent_instance_id: agentInstanceId,
    text,
  }),
});
const result = await response.json();
if (result.code !== 0) throw new Error(result.message);

const response = await fetch(`${baseURL}/api/send-agent-instance-tts`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    agent_instance_id: agentInstanceId,
    text,
  }),
});
const result = await response.json();
if (result.code !== 0) throw new Error(result.message);

The Web quickstart displays a TTS input box in broadcast mode and saves the agent_instance_id returned by the server to the page state.

@Override
protected void onDestroy() {
    super.onDestroy();
    JSONObject bodyJson = new JSONObject();
    bodyJson.put("agent_instance_id", agentInstanceId);
    Request request = new Request.Builder()
        .url(baseUrl + "/api/stop")
        .post(RequestBody.create(bodyJson.toString(),
            MediaType.parse("application/json; charset=utf-8")))
        .build();
    new OkHttpClient().newCall(request).enqueue(new Callback() {
        @Override
        public void onResponse(@NonNull Call call, @NonNull Response response) {
            ZegoExpressEngine.getEngine().stopPlayingStream(agentStreamId);
            ZegoExpressEngine.getEngine().logoutRoom();
            digitalMobile.stop();
            ZegoExpressEngine.destroyEngine(null);
        }

        @Override
        public void onFailure(@NonNull Call call, @NonNull IOException e) {
            // Even if the request fails, release local RTC and Digital Human SDK resources
            ZegoExpressEngine.getEngine().logoutRoom();
            digitalMobile.stop();
            ZegoExpressEngine.destroyEngine(null);
        }
    });
}

@Override
protected void onDestroy() {
    super.onDestroy();
    JSONObject bodyJson = new JSONObject();
    bodyJson.put("agent_instance_id", agentInstanceId);
    Request request = new Request.Builder()
        .url(baseUrl + "/api/stop")
        .post(RequestBody.create(bodyJson.toString(),
            MediaType.parse("application/json; charset=utf-8")))
        .build();
    new OkHttpClient().newCall(request).enqueue(new Callback() {
        @Override
        public void onResponse(@NonNull Call call, @NonNull Response response) {
            ZegoExpressEngine.getEngine().stopPlayingStream(agentStreamId);
            ZegoExpressEngine.getEngine().logoutRoom();
            digitalMobile.stop();
            ZegoExpressEngine.destroyEngine(null);
        }

        @Override
        public void onFailure(@NonNull Call call, @NonNull IOException e) {
            // Even if the request fails, release local RTC and Digital Human SDK resources
            ZegoExpressEngine.getEngine().logoutRoom();
            digitalMobile.stop();
            ZegoExpressEngine.destroyEngine(null);
        }
    });
}

- (void)stopLiveDigitalHuman {
    NSDictionary *params = @{ @"agent_instance_id": agentInstanceId };
    NSURL *url = [NSURL URLWithString:
        [baseURL stringByAppendingString:@"/api/stop"]];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
    request.HTTPMethod = @"POST";
    [request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
    request.HTTPBody = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];

    [[[NSURLSession sharedSession] dataTaskWithRequest:request
        completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        [[ZegoExpressEngine sharedEngine] stopPlayingStream:agentStreamId];
        [[ZegoExpressEngine sharedEngine] logoutRoom];
        [digitalMobile stop];
        [ZegoExpressEngine destroyEngine:nil];
    }] resume];
}

- (void)stopLiveDigitalHuman {
    NSDictionary *params = @{ @"agent_instance_id": agentInstanceId };
    NSURL *url = [NSURL URLWithString:
        [baseURL stringByAppendingString:@"/api/stop"]];
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:url];
    request.HTTPMethod = @"POST";
    [request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
    request.HTTPBody = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];

    [[[NSURLSession sharedSession] dataTaskWithRequest:request
        completionHandler:^(NSData *data, NSURLResponse *response, NSError *error) {
        [[ZegoExpressEngine sharedEngine] stopPlayingStream:agentStreamId];
        [[ZegoExpressEngine sharedEngine] logoutRoom];
        [digitalMobile stop];
        [ZegoExpressEngine destroyEngine:nil];
    }] resume];
}

async function logoutRoom() {
  await fetch(`${baseURL}/api/stop`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ agent_instance_id: agentInstanceId }),
  });
  zg.stopPlayingStream(agentStreamId);
  zg.logoutRoom(roomID);
  agentInstanceId = "";
}

async function logoutRoom() {
  await fetch(`${baseURL}/api/stop`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ agent_instance_id: agentInstanceId }),
  });
  zg.stopPlayingStream(agentStreamId);
  zg.logoutRoom(roomID);
  agentInstanceId = "";
}

The broadcast scenario has no local audio stream, so there is no need to destroy audio capture streams.

Mode	Instance Creation Parameters	Client Playback Method	Use Cases
RTC	`room_id`	ZEGO Express SDK to play RTC stream	Low latency, small-scale interaction
CDN	`cdn_url`	Use a player to play CDN stream	Large-scale live streaming

Digital humans support both RTC and CDN stream playback. If you need a large number of users to watch digital human broadcasts, such as in e-commerce live streaming or live classroom scenarios, you can use CDN live streaming mode. In CDN mode, the Web client does not need to integrate the ZEGO Express SDK. Simply use any Web player that supports HLS/FLV (such as hls.js or video.js) to play the cdn_url.

Android Troubleshooting Checklist

Symptom	Troubleshooting Direction
Room login failed	Check if the Token is valid, if `AppID` matches, and if the network can reach ZEGOCLOUD services.
Instance creation failed	Confirm that `digital_human_id`, `config_id` (Android uses `mobile`), and `room_id` are correct, and that the business backend signature is valid.
No stream / No video	Confirm `agent_stream_id` matches the server response; check if `enableCustomVideoRender` was called before `startPlayingStream`.
Display stays on static image	Check if `digitalView`/`loadingView`/`digitalPic` are correctly `attach`ed; confirm video frames and SEI are forwarded to `digitalMobileSDK`; run on a real device, not an emulator.
Inaccurate lip-sync	Confirm SEI data is forwarded via `onPlayerSyncRecvSEI`; confirm SEI-related parameters in `advanceConfig` match the ZEGO Express SDK version.
TTS not broadcasting	Confirm `agent_instance_id` is correct, `text` ≤ 300 characters, and the instance has not been destroyed due to 900 seconds of idle time.

iOS Troubleshooting Checklist

Symptom	Troubleshooting Direction
Room login failed	Check if the Token is valid, if `AppID` matches, and if the network can reach ZEGOCLOUD services.
Instance creation failed	Confirm that `digital_human_id`, `config_id` (iOS uses `mobile`), and `room_id` are correct, and that the business backend signature is valid.
No stream / No video	Match `agentStreamId` in `onRoomStreamUpdate` before playing the stream; confirm `enableCustomVideoRender` is called before `startPlayingStream`.
Display stays on static image	Confirm video frames and SEI are forwarded to `digitalMobile`; Info.plist does not need microphone permission but must have network permission.
TTS not broadcasting	Confirm `agent_instance_id` is correct, `text` ≤ 300 characters, and the instance has not been destroyed due to 900 seconds of idle time.

Web Troubleshooting Checklist

Symptom	Troubleshooting Direction
Room login failed	Check if the Token is valid, if `AppID` matches; for version 3.7.0 and above, `server` can be an empty string.
Instance creation failed	Confirm that `digital_human_id`, `config_id` (Web uses `web`), and `room_id` are correct, and that the business backend signature is valid.
No video / No audio	Filter by `agentStreamId` in `roomStreamUpdate` before calling `startPlayingStream`; wait for `remoteCameraStatusUpdate` to be `OPEN` before calling `playVideo`.
Video container not displaying	Confirm the container ID passed to `playVideo` is `remoteStreamView` (check spelling).
TTS not broadcasting	Confirm `agent_instance_id` is correct, `text` ≤ 300 characters, and the instance has not been destroyed due to 900 seconds of idle time.

Implement Live Digital Human Broadcasting

Differences from Digital Human Video Calls

Prerequisites

Sample Code

Overall Business Flow

Core Feature Implementation

Integrating ZEGO Express SDK

app/build.gradle

AndroidManifest.xml

Podfile

Integrating the Digital Human SDK

Integrating the Digital Human SDK

Notify the Business Backend to Create a Broadcast Digital Human Instance

User Enters the Room (No Stream Publishing)

Initializing the Digital Human SDK and Custom Rendering

Playing the Digital Human Stream

Proactively Making the Digital Human Broadcast Text

Leaving the Room and Ending the Broadcast

RTC and CDN Modes

Listening to Callbacks

Troubleshooting Checklist