The following is the example code for the business backend that integrates the real-time interactive AI Agent API. You can refer to the example code to implement your own business logic.
Business Backend Example Code
Includes the basic capabilities of obtaining ZEGOCLOUD Token, registering AI Agent, creating AI Agent instances, and deleting AI Agent instances.
Below are the client sample codes, you can refer to these sample codes to implement your own business logic.
Android Client Sample Code
Android client sample code. It includes basic capabilities such as logging into and out of RTC rooms, and publishing and playing streams.
iOS Client Sample Code
iOS client sample code. It includes basic capabilities such as logging into and out of RTC rooms, and publishing and playing streams.
Flutter Client Sample Code
Flutter client sample code. It includes basic capabilities such as logging into and out of RTC rooms, and publishing and playing streams.
Web Client Sample Code
Web client sample code. It includes basic capabilities such as logging into and out of RTC rooms, and publishing and playing streams.
The following video demonstrates how to run the server and client (Web) sample code and interact with an AI agent by voice.
The following video demonstrates how to run the server and client (iOS) sample code and interact with an AI agent by voice.
Overall Business Process
Server side: Follow the Server Quick Start guide to run the server sample code and deploy your server
Integrate ZEGOCLOUD AI Agent APIs to manage AI agents.
After completing these two steps, you can add an AI agent to a room for real-time interaction with real users.
Core Capability Implementation
Integrate ZEGO Express SDK
Please refer to Integrate the SDK > 2.2 > Method 2 to manually integrate the SDK. After integrating the SDK, follow these steps to initialize ZegoExpressEngine.
1
Add Permission Declaration
Navigate to the "app/src/main" directory, open the "AndroidManifest.xml" file, and add permissions.
private final ActivityResultLauncher<String> requestPermissionLauncher = registerForActivityResult(
new ActivityResultContracts.RequestPermission(), new ActivityResultCallback<Boolean>() {
@Override
public void onActivityResult(Boolean isGranted) {
if (isGranted) {
// Permission granted
}
}
});
// Initiate request
requestPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO);
private final ActivityResultLauncher<String> requestPermissionLauncher = registerForActivityResult(
new ActivityResultContracts.RequestPermission(), new ActivityResultCallback<Boolean>() {
@Override
public void onActivityResult(Boolean isGranted) {
if (isGranted) {
// Permission granted
}
}
});
// Initiate request
requestPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO);
3
Create and Initialize ZegoExpressEngine
ZegoEngineProfile zegoEngineProfile = new ZegoEngineProfile();
zegoEngineProfile.appID = ; // Obtain from ZEGOCLOUD Console
// !mark
zegoEngineProfile.scenario = ZegoScenario.HIGH_QUALITY_CHATROOM; // Setting this scenario can avoid requesting camera permissions, and the integrator should set specific values according to their own business scenarios
zegoEngineProfile.application = getApplication();
ZegoExpressEngine.createEngine(zegoEngineProfile, null);
ZegoEngineProfile zegoEngineProfile = new ZegoEngineProfile();
zegoEngineProfile.appID = ; // Obtain from ZEGOCLOUD Console
// !mark
zegoEngineProfile.scenario = ZegoScenario.HIGH_QUALITY_CHATROOM; // Setting this scenario can avoid requesting camera permissions, and the integrator should set specific values according to their own business scenarios
zegoEngineProfile.application = getApplication();
ZegoExpressEngine.createEngine(zegoEngineProfile, null);
Please refer to Import the SDK > 2.2 > Method 3 to manually integrate the SDK. After integrating the SDK, follow these steps to initialize ZegoExpressEngine.
- (void)requestAudioPermission:(void(^)(BOOL granted))completion {
/// Need to add a description of microphone usage in the project's Info.plist file
AVAudioSession *audioSession = [AVAudioSession sharedInstance];
[audioSession requestRecordPermission:^(BOOL granted) {
dispatch_async(dispatch_get_main_queue(), ^{
completion(granted);
});
}];
}
- (void)requestAudioPermission:(void(^)(BOOL granted))completion {
/// Need to add a description of microphone usage in the project's Info.plist file
AVAudioSession *audioSession = [AVAudioSession sharedInstance];
[audioSession requestRecordPermission:^(BOOL granted) {
dispatch_async(dispatch_get_main_queue(), ^{
completion(granted);
});
}];
}
3
Create and Initialize ZegoExpressEngine
-(void)initZegoExpressEngine{
ZegoEngineProfile* profile = [[ZegoEngineProfile alloc]init];
profile.appID = kZegoPassAppId;
// !mark
profile.scenario = ZegoScenarioHighQualityChatroom; // Setting this scenario can avoid requesting camera permissions, and the integrator should set specific values according to their own business scenarios
[ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];
}
-(void)initZegoExpressEngine{
ZegoEngineProfile* profile = [[ZegoEngineProfile alloc]init];
profile.appID = kZegoPassAppId;
// !mark
profile.scenario = ZegoScenarioHighQualityChatroom; // Setting this scenario can avoid requesting camera permissions, and the integrator should set specific values according to their own business scenarios
[ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];
}
Please refer to Integrate the SDK > 2.2 > Method 1 to manually integrate the SDK. After integrating the SDK, follow these steps to initialize ZegoExpressEngine.
Go to ios directory, open Podfile file, and add permissions
Podfile
post_install do |installer|
installer.pods_project.targets.each do |target|
flutter_additional_ios_build_settings(target)
<!-- !mark(1:8) -->
# Start of the permission_handler configuration
target.build_configurations.each do |config|
config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [
'$(inherited)',
'PERMISSION_MICROPHONE=1',
]
end
# End of the permission_handler configuration
end
end
Podfile
post_install do |installer|
installer.pods_project.targets.each do |target|
flutter_additional_ios_build_settings(target)
<!-- !mark(1:8) -->
# Start of the permission_handler configuration
target.build_configurations.each do |config|
config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [
'$(inherited)',
'PERMISSION_MICROPHONE=1',
]
end
# End of the permission_handler configuration
end
end
Please refer to Integrate SDK > Method 2 to use npm to integrate SDK v3.9.123 or above. After integrating the SDK, initialize ZegoExpressEngine as follows.
Instantiate ZegoExpressEngine
Check system requirements (WebRTC support and microphone permissions)
import { ZegoExpressEngine } from "zego-express-engine-webrtc";
const appID = 1234567 // Obtain from ZEGOCLOUD Console
const server = 'xxx' // Obtain from ZEGOCLOUD Console
// Instantiate ZegoExpressEngine with appId and server configurations
// !mark
const zg = new ZegoExpressEngine(appID, server);
// Check system requirements
// !mark
const checkSystemRequirements = async () => {
// Detect WebRTC support
const rtc_sup = await zg.checkSystemRequirements("webRTC");
if (!rtc_sup.result) {
// Browser does not support WebRTC
}
// Detect microphone permission status
const mic_sup = await zg.checkSystemRequirements("microphone");
if (!mic_sup.result) {
// Microphone permission is not enabled
}
}
checkSystemRequirements()
import { ZegoExpressEngine } from "zego-express-engine-webrtc";
const appID = 1234567 // Obtain from ZEGOCLOUD Console
const server = 'xxx' // Obtain from ZEGOCLOUD Console
// Instantiate ZegoExpressEngine with appId and server configurations
// !mark
const zg = new ZegoExpressEngine(appID, server);
// Check system requirements
// !mark
const checkSystemRequirements = async () => {
// Detect WebRTC support
const rtc_sup = await zg.checkSystemRequirements("webRTC");
if (!rtc_sup.result) {
// Browser does not support WebRTC
}
// Detect microphone permission status
const mic_sup = await zg.checkSystemRequirements("microphone");
if (!mic_sup.result) {
// Microphone permission is not enabled
}
}
checkSystemRequirements()
Notify Your Server to Start Call
You can notify your server to start the call immediately after the real user enters the room on the client side. Asynchronous calls can help reduce call connection time. After receiving the start call notification, your server creates an AI agent instance using the same roomID and associated userID and streamID as the client, so that the AI agent can interact with real users in the same room through mutual stream publishing and playing.
Sample Code for Notifying Your Server
Note
In the following examples, roomID, userID, streamID and other parameters are not passed when notifying your server to start the call because fixed values have been agreed between the client and your server in this example. In actual use, please pass the real parameters according to your business requirements.
// Notify your server to start call
private void start() {
RequestBody body = RequestBody.create("", MediaType.parse("application/json; charset=utf-8"));
Request request = new Request.Builder().url(YOUR_SERVER_URL + "/api/start").post(body).build();
new OkHttpClient.Builder().build().newCall(request).enqueue(new Callback() {
@Override
public void onFailure(@NonNull Call call, @NonNull IOException e) {
}
@Override
public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException {
}
});
}
// Notify your server to start call
private void start() {
RequestBody body = RequestBody.create("", MediaType.parse("application/json; charset=utf-8"));
Request request = new Request.Builder().url(YOUR_SERVER_URL + "/api/start").post(body).build();
new OkHttpClient.Builder().build().newCall(request).enqueue(new Callback() {
@Override
public void onFailure(@NonNull Call call, @NonNull IOException e) {
}
@Override
public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException {
}
});
}
// Notify your server to start call
/**
* Start a call with the AI agent
*
* @param completion Completion callback, returns operation result
* @discussion This method sends a request to the server to start the call, used to initialize the AI agent instance
*/
- (void)doStartCallWithCompletion:(void (^)(NSInteger code, NSString *message, NSDictionary *data))completion {
// Build request URL
NSString *url = [NSString stringWithFormat:@"%@/api/start", self.currentBaseURL];
NSURL *requestURL = [NSURL URLWithString:url];
// Create request
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:requestURL];
request.HTTPMethod = @"POST";
// Set request headers
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
// Create request parameters
NSMutableDictionary *params = [NSMutableDictionary dictionary];
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];
request.HTTPBody = jsonData;
// Create session
NSURLSession *session = [NSURLSession sharedSession];
// Send request
NSURLSessionDataTask *task = [session dataTaskWithRequest:request
completionHandler:^(NSData * _Nullable data,
NSURLResponse * _Nullable response,
NSError * _Nullable error) {
dispatch_async(dispatch_get_main_queue(), ^{
if (error) {
if (completion) {
completion(-1, @"Network request failed", nil);
}
return;
}
NSHTTPURLResponse *httpUrlResponse = (NSHTTPURLResponse *)response;
if (httpUrlResponse.statusCode != 200) {
if (completion) {
completion(httpUrlResponse.statusCode,
[NSString stringWithFormat:@"Server error: %ld", (long)httpUrlResponse.statusCode],
nil);
}
return;
}
NSError *jsonError;
NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:data options:0 error:&jsonError];
if (jsonError) {
if (completion) {
completion(-2, @"Failed to parse response data", nil);
}
return;
}
// Parse response data
NSInteger code = [dict[@"code"] integerValue];
NSString *message = dict[@"message"];
NSDictionary *responseData = dict[@"data"];
if (completion) {
completion(code, message, responseData);
}
});
}];
[task resume];
}
// Notify your server to start call
/**
* Start a call with the AI agent
*
* @param completion Completion callback, returns operation result
* @discussion This method sends a request to the server to start the call, used to initialize the AI agent instance
*/
- (void)doStartCallWithCompletion:(void (^)(NSInteger code, NSString *message, NSDictionary *data))completion {
// Build request URL
NSString *url = [NSString stringWithFormat:@"%@/api/start", self.currentBaseURL];
NSURL *requestURL = [NSURL URLWithString:url];
// Create request
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:requestURL];
request.HTTPMethod = @"POST";
// Set request headers
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
// Create request parameters
NSMutableDictionary *params = [NSMutableDictionary dictionary];
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];
request.HTTPBody = jsonData;
// Create session
NSURLSession *session = [NSURLSession sharedSession];
// Send request
NSURLSessionDataTask *task = [session dataTaskWithRequest:request
completionHandler:^(NSData * _Nullable data,
NSURLResponse * _Nullable response,
NSError * _Nullable error) {
dispatch_async(dispatch_get_main_queue(), ^{
if (error) {
if (completion) {
completion(-1, @"Network request failed", nil);
}
return;
}
NSHTTPURLResponse *httpUrlResponse = (NSHTTPURLResponse *)response;
if (httpUrlResponse.statusCode != 200) {
if (completion) {
completion(httpUrlResponse.statusCode,
[NSString stringWithFormat:@"Server error: %ld", (long)httpUrlResponse.statusCode],
nil);
}
return;
}
NSError *jsonError;
NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:data options:0 error:&jsonError];
if (jsonError) {
if (completion) {
completion(-2, @"Failed to parse response data", nil);
}
return;
}
// Parse response data
NSInteger code = [dict[@"code"] integerValue];
NSString *message = dict[@"message"];
NSDictionary *responseData = dict[@"data"];
if (completion) {
completion(code, message, responseData);
}
});
}];
[task resume];
}
// Notify your server to start call
Future<Map<String, dynamic>> startCall() async {
try {
final response = await http.post(
Uri.parse('$_currentBaseUrl/api/start'),
headers: {'Content-Type': 'application/json'},
);
if (response.statusCode == 200) {
final json = jsonDecode(response.body);
return json;
}
return {'code': -1, 'message': 'Request failed'};
} catch (e) {
return {'code': -1, 'message': e.toString()};
}
}
// Notify your server to start call
Future<Map<String, dynamic>> startCall() async {
try {
final response = await http.post(
Uri.parse('$_currentBaseUrl/api/start'),
headers: {'Content-Type': 'application/json'},
);
if (response.statusCode == 200) {
final json = jsonDecode(response.body);
return json;
}
return {'code': -1, 'message': 'Request failed'};
} catch (e) {
return {'code': -1, 'message': e.toString()};
}
}
// Notify your server to start call
async function startCall() {
try {
const response = await fetch(`${YOUR_SERVER_URL}/api/start`, { // YOUR_SERVER_URL is the address of your Your Server
method: 'POST',
headers: {
'Content-Type': 'application/json',
}
});
const data = await response.json();
console.log('Start call result:', data);
return data;
} catch (error) {
console.error('Failed to start call:', error);
throw error;
}
}
// Notify your server to start call
async function startCall() {
try {
const response = await fetch(`${YOUR_SERVER_URL}/api/start`, { // YOUR_SERVER_URL is the address of your Your Server
method: 'POST',
headers: {
'Content-Type': 'application/json',
}
});
const data = await response.json();
console.log('Start call result:', data);
return data;
} catch (error) {
console.error('Failed to start call:', error);
throw error;
}
}
User logs in a RTC room and starts publishing a stream
After a real user logs into the room, they start publishing streams.
Note
In this scenario, AI echo cancellation should be enabled for better effects.
The token used for login needs to be obtained from your server; please refer to the complete sample code.
Note
Please ensure that the roomID, userID, and streamID are unique under one ZEGOCLOUD APPID.
roomID: Generated by the user according to their own rules, it will be used to log into the Express SDK room. Only numbers, English characters, and '~', '!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '_', '+', '=', '-', '`', ';', ''', ',', '.', '<', '>', '' are supported. If interoperability with the Web SDK is required, do not use '%'.
userID: Length should not exceed 32 bytes. Only numbers, English characters, and '~', '!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '_', '+', '=', '-', '`', ';', ''', ',', '.', '<', '>', '' are supported. If interoperability with the Web SDK is required, do not use '%'.
streamID: Length should not exceed 256 bytes. Only numbers, English characters, and '-', '_' are supported.
Client login to room and publish a stream
private void loginRoom(String userId, String userName, String userName, String token,
IZegoRoomLoginCallback callback) {
ZegoEngineConfig config = new ZegoEngineConfig();
HashMap<String, String> advanceConfig = new HashMap<String, String>();
// !mark(1:2)
advanceConfig.put("set_audio_volume_ducking_mode", "1");
advanceConfig.put("enable_rnd_volume_adaptive", "true");
config.advancedConfig = advanceConfig;
ZegoExpressEngine.setEngineConfig(config);
ZegoExpressEngine.getEngine().setRoomScenario(ZegoScenario.HIGH_QUALITY_CHATROOM);
ZegoExpressEngine.getEngine().setAudioDeviceMode(ZegoAudioDeviceMode.GENERAL);
// !mark(1:6)
ZegoExpressEngine.getEngine().enableAEC(true);
// Please note: To enable AI echo cancellation, please contact ZEGOCLOUD technical support to obtain the corresponding version of ZEGOExpress SDK
ZegoExpressEngine.getEngine().setAECMode(ZegoAECMode.AI_AGGRESSIVE2);
ZegoExpressEngine.getEngine().enableAGC(true);
ZegoExpressEngine.getEngine().enableANS(true);
ZegoExpressEngine.getEngine().setANSMode(ZegoANSMode.MEDIUM);
ZegoRoomConfig roomConfig = new ZegoRoomConfig();
roomConfig.isUserStatusNotify = true;
roomConfig.token = token; // Token authentication is required, obtain it from your server, and refer to ZEGOCLOUD documentation for generation method
String roomId ; // Custom room ID for login, please refer to the format description
String userSteamID // Custom stream ID for publishing, please refer to the format description
// !mark
ZegoExpressEngine.getEngine()
.loginRoom(roomId, new ZegoUser(userId, userName), roomConfig, (errorCode, extendedData) -> {
Timber.d(
"loginRoom() called with: errorCode = [" + errorCode + "], extendedData = [" + extendedData + "]");
if (errorCode == 0) {
// !mark
// Start publishing stream after successful login
ZegoExpressEngine.getEngine().startPublishingStream(userSteamID);
// Set microphone mute status, false means unmuted, true means muted
ZegoExpressEngine.getEngine().muteMicrophone(false);
}
if (callback != null) {
callback.onRoomLoginResult(errorCode, extendedData);
}
});
}
Client login to room and publish a stream
private void loginRoom(String userId, String userName, String userName, String token,
IZegoRoomLoginCallback callback) {
ZegoEngineConfig config = new ZegoEngineConfig();
HashMap<String, String> advanceConfig = new HashMap<String, String>();
// !mark(1:2)
advanceConfig.put("set_audio_volume_ducking_mode", "1");
advanceConfig.put("enable_rnd_volume_adaptive", "true");
config.advancedConfig = advanceConfig;
ZegoExpressEngine.setEngineConfig(config);
ZegoExpressEngine.getEngine().setRoomScenario(ZegoScenario.HIGH_QUALITY_CHATROOM);
ZegoExpressEngine.getEngine().setAudioDeviceMode(ZegoAudioDeviceMode.GENERAL);
// !mark(1:6)
ZegoExpressEngine.getEngine().enableAEC(true);
// Please note: To enable AI echo cancellation, please contact ZEGOCLOUD technical support to obtain the corresponding version of ZEGOExpress SDK
ZegoExpressEngine.getEngine().setAECMode(ZegoAECMode.AI_AGGRESSIVE2);
ZegoExpressEngine.getEngine().enableAGC(true);
ZegoExpressEngine.getEngine().enableANS(true);
ZegoExpressEngine.getEngine().setANSMode(ZegoANSMode.MEDIUM);
ZegoRoomConfig roomConfig = new ZegoRoomConfig();
roomConfig.isUserStatusNotify = true;
roomConfig.token = token; // Token authentication is required, obtain it from your server, and refer to ZEGOCLOUD documentation for generation method
String roomId ; // Custom room ID for login, please refer to the format description
String userSteamID // Custom stream ID for publishing, please refer to the format description
// !mark
ZegoExpressEngine.getEngine()
.loginRoom(roomId, new ZegoUser(userId, userName), roomConfig, (errorCode, extendedData) -> {
Timber.d(
"loginRoom() called with: errorCode = [" + errorCode + "], extendedData = [" + extendedData + "]");
if (errorCode == 0) {
// !mark
// Start publishing stream after successful login
ZegoExpressEngine.getEngine().startPublishingStream(userSteamID);
// Set microphone mute status, false means unmuted, true means muted
ZegoExpressEngine.getEngine().muteMicrophone(false);
}
if (callback != null) {
callback.onRoomLoginResult(errorCode, extendedData);
}
});
}
Client request to login to room and publish a stream
// Record the agent
self.streamToPlay = [self getAgentStreamID];
ZegoEngineConfig* engineConfig = [[ZegoEngineConfig alloc] init];
engineConfig.advancedConfig = @{
// !mark(1:2)
@"set_audio_volume_ducking_mode":@1,/** This configuration is used for volume ducking **/
@"enable_rnd_volume_adaptive":@"true",/** This configuration is used for adaptive playback volume **/
};
// !mark
[ZegoExpressEngine setEngineConfig:engineConfig];
// This setting only affects AEC (echo cancellation). Here we set it to ModeGeneral, which uses our proprietary echo cancellation algorithm, giving us more control.
// If other options are selected, it might use the system's echo cancellation, which may work better on iPhones but could be less effective on some Android devices.
[[ZegoExpressEngine sharedEngine] setAudioDeviceMode:ZegoAudioDeviceModeGeneral];
// Note: Enabling AI echo cancellation requires contacting ZEGOCLOUD technical support to obtain the corresponding ZegoExpressionEngine.xcframework, as versions with these capabilities have not yet been released.
// !mark(1:5)
[[ZegoExpressEngine sharedEngine] enableAGC:TRUE];
[[ZegoExpressEngine sharedEngine] enableAEC:TRUE];
[[ZegoExpressEngine sharedEngine] setAECMode:ZegoAECModeAIAggressive2];
[[ZegoExpressEngine sharedEngine] enableANS:TRUE];
[[ZegoExpressEngine sharedEngine] setANSMode:ZegoANSModeMedium];
// Login to room
// !mark
[self loginRoom:^(int errorCode, NSDictionary *extendedData) {
if (errorCode!=0) {
NSString* errorMsg =[NSString stringWithFormat:@"Failed to enter voice room:%d", errorCode];
completion(NO, errorMsg);
return;
}
// Start publishing stream after entering room
[self startPushlishStream];
}];
Client request to login to room and publish a stream
// Record the agent
self.streamToPlay = [self getAgentStreamID];
ZegoEngineConfig* engineConfig = [[ZegoEngineConfig alloc] init];
engineConfig.advancedConfig = @{
// !mark(1:2)
@"set_audio_volume_ducking_mode":@1,/** This configuration is used for volume ducking **/
@"enable_rnd_volume_adaptive":@"true",/** This configuration is used for adaptive playback volume **/
};
// !mark
[ZegoExpressEngine setEngineConfig:engineConfig];
// This setting only affects AEC (echo cancellation). Here we set it to ModeGeneral, which uses our proprietary echo cancellation algorithm, giving us more control.
// If other options are selected, it might use the system's echo cancellation, which may work better on iPhones but could be less effective on some Android devices.
[[ZegoExpressEngine sharedEngine] setAudioDeviceMode:ZegoAudioDeviceModeGeneral];
// Note: Enabling AI echo cancellation requires contacting ZEGOCLOUD technical support to obtain the corresponding ZegoExpressionEngine.xcframework, as versions with these capabilities have not yet been released.
// !mark(1:5)
[[ZegoExpressEngine sharedEngine] enableAGC:TRUE];
[[ZegoExpressEngine sharedEngine] enableAEC:TRUE];
[[ZegoExpressEngine sharedEngine] setAECMode:ZegoAECModeAIAggressive2];
[[ZegoExpressEngine sharedEngine] enableANS:TRUE];
[[ZegoExpressEngine sharedEngine] setANSMode:ZegoANSModeMedium];
// Login to room
// !mark
[self loginRoom:^(int errorCode, NSDictionary *extendedData) {
if (errorCode!=0) {
NSString* errorMsg =[NSString stringWithFormat:@"Failed to enter voice room:%d", errorCode];
completion(NO, errorMsg);
return;
}
// Start publishing stream after entering room
[self startPushlishStream];
}];
Client request to login to room and publish a stream
final String _userId = 'user_id_1';
final String _roomId = 'room_id_1';
final String _userStreamId = 'user_stream_id_1';
/// Generate RTC Token [Reference Documentation](https://www.zegocloud.com/docs/video-call/token?platform=flutter&language=dart)
final token = await getToken();
if (token.isEmpty) {
return false;
}
// !mark(1:11)
/// The following is used for answering delay optimization, you need to integrate the corresponding version of ZegoExpressEngine sdk, please contact ZEGOCLOUD technical support
ZegoExpressEngine.setEngineConfig(
ZegoEngineConfig(
advancedConfig: {
/**This configuration is used for volume ducking**/
'set_audio_volume_ducking_mode': '1',
/**This configuration is used for adaptive playback volume**/
'enable_rnd_volume_adaptive': 'true'
},
),
);
/// Enable 3A
// !mark(1:3)
ZegoExpressEngine.instance.enableAGC(true);
ZegoExpressEngine.instance.enableAEC(true);
if (!kIsWeb) {
ZegoExpressEngine.instance.setAECMode(ZegoANSMode.AIAGGRESSIVE2);
// !mark(1:4)
/// This setting only affects AEC (echo cancellation). Here we set it to ModeGeneral, which uses our proprietary echo cancellation, which is more controllable.
/// If other options are selected, it might use the system's echo cancellation, which may work better on iPhones but could be less effective on some Android devices.
ZegoExpressEngine.instance.setAudioDeviceMode(
ZegoAudioDeviceMode.General,
);
}
ZegoExpressEngine.instance.enableANS(true);
ZegoExpressEngine.instance.setANSMode(ZegoANSMode.Medium);
/// Login to room
// !mark
final user = ZegoUser(_userId, _userId);
final roomConfig = ZegoRoomConfig.defaultConfig()
..isUserStatusNotify = true
..token = token;
final loginResult = await ZegoExpressEngine.instance.loginRoom(
_roomId,
user,
config: roomConfig,
);
if (0 != loginResult.errorCode && 1002001 != loginResult.errorCode) {
return false;
}
/// Start publishing stream (open microphone)
await ZegoExpressEngine.instance.muteMicrophone(false);
await ZegoExpressEngine.instance.startPublishingStream(_userStreamId);
Client request to login to room and publish a stream
final String _userId = 'user_id_1';
final String _roomId = 'room_id_1';
final String _userStreamId = 'user_stream_id_1';
/// Generate RTC Token [Reference Documentation](https://www.zegocloud.com/docs/video-call/token?platform=flutter&language=dart)
final token = await getToken();
if (token.isEmpty) {
return false;
}
// !mark(1:11)
/// The following is used for answering delay optimization, you need to integrate the corresponding version of ZegoExpressEngine sdk, please contact ZEGOCLOUD technical support
ZegoExpressEngine.setEngineConfig(
ZegoEngineConfig(
advancedConfig: {
/**This configuration is used for volume ducking**/
'set_audio_volume_ducking_mode': '1',
/**This configuration is used for adaptive playback volume**/
'enable_rnd_volume_adaptive': 'true'
},
),
);
/// Enable 3A
// !mark(1:3)
ZegoExpressEngine.instance.enableAGC(true);
ZegoExpressEngine.instance.enableAEC(true);
if (!kIsWeb) {
ZegoExpressEngine.instance.setAECMode(ZegoANSMode.AIAGGRESSIVE2);
// !mark(1:4)
/// This setting only affects AEC (echo cancellation). Here we set it to ModeGeneral, which uses our proprietary echo cancellation, which is more controllable.
/// If other options are selected, it might use the system's echo cancellation, which may work better on iPhones but could be less effective on some Android devices.
ZegoExpressEngine.instance.setAudioDeviceMode(
ZegoAudioDeviceMode.General,
);
}
ZegoExpressEngine.instance.enableANS(true);
ZegoExpressEngine.instance.setANSMode(ZegoANSMode.Medium);
/// Login to room
// !mark
final user = ZegoUser(_userId, _userId);
final roomConfig = ZegoRoomConfig.defaultConfig()
..isUserStatusNotify = true
..token = token;
final loginResult = await ZegoExpressEngine.instance.loginRoom(
_roomId,
user,
config: roomConfig,
);
if (0 != loginResult.errorCode && 1002001 != loginResult.errorCode) {
return false;
}
/// Start publishing stream (open microphone)
await ZegoExpressEngine.instance.muteMicrophone(false);
await ZegoExpressEngine.instance.startPublishingStream(_userStreamId);
Client login to room and publish a stream
const userId = "" // User ID for logging into the Express SDK room
const roomId = "" // RTC Room ID
const userStreamId = "" // User stream push ID
async function enterRoom() {
try {
// Generate RTC Token [Reference Documentation] (https://www.zegocloud.com/docs/video-call/token?platform=web&language=javascript)
const token = await Api.getToken();
// Login to room
await zg.loginRoom(roomId, token, {
userID: userId,
userName: "",
});
// Create local audio stream
const localStream = await zg.createZegoStream({
camera: {
video: false,
audio: true,
},
});
if (localStream) {
// !mark(1:2)
// Push local stream
await zg.startPublishingStream(userStreamId, localStream);
}
} catch (error) {
console.error("Failed to enter room:", error);
throw error;
}
}
enterRoom()
Client login to room and publish a stream
const userId = "" // User ID for logging into the Express SDK room
const roomId = "" // RTC Room ID
const userStreamId = "" // User stream push ID
async function enterRoom() {
try {
// Generate RTC Token [Reference Documentation] (https://www.zegocloud.com/docs/video-call/token?platform=web&language=javascript)
const token = await Api.getToken();
// Login to room
await zg.loginRoom(roomId, token, {
userID: userId,
userName: "",
});
// Create local audio stream
const localStream = await zg.createZegoStream({
camera: {
video: false,
audio: true,
},
});
if (localStream) {
// !mark(1:2)
// Push local stream
await zg.startPublishingStream(userStreamId, localStream);
}
} catch (error) {
console.error("Failed to enter room:", error);
throw error;
}
}
enterRoom()
Play the AI Agent Stream
By default, there is only one real user and one AI agent in the same room, so any new stream added is assumed to be the AI agent stream.
Client request to play the AI agent stream
// Set up the event handler
void setEventHandler() {
ZegoExpressEngine.getEngine().setEventHandler(new IZegoEventHandler() {
@Override
// When other users in the room start/stop publishing streams, you can receive notifications about the corresponding user's audio/video stream changes here
public void onRoomStreamUpdate(String roomID, ZegoUpdateType updateType, ArrayList<ZegoStream> streamList, JSONObject extendedData) {
super.onRoomStreamUpdate(roomID, updateType, streamList, extendedData);
// When updateType is ZegoUpdateType.ADD, it means there is a new audio/video stream, at this time we can call the startPlayingStream interface to pull this audio/video stream
// !mark(1:8)
if (updateType == ZegoUpdateType.ADD) {
ZegoStream stream = streamList.get(0);
// By default, new streams are from the AI agent, so play directly
ZegoExpressEngine.getEngine().startPlayingStream(stream.streamID);
}
}
});
}
Client request to play the AI agent stream
// Set up the event handler
void setEventHandler() {
ZegoExpressEngine.getEngine().setEventHandler(new IZegoEventHandler() {
@Override
// When other users in the room start/stop publishing streams, you can receive notifications about the corresponding user's audio/video stream changes here
public void onRoomStreamUpdate(String roomID, ZegoUpdateType updateType, ArrayList<ZegoStream> streamList, JSONObject extendedData) {
super.onRoomStreamUpdate(roomID, updateType, streamList, extendedData);
// When updateType is ZegoUpdateType.ADD, it means there is a new audio/video stream, at this time we can call the startPlayingStream interface to pull this audio/video stream
// !mark(1:8)
if (updateType == ZegoUpdateType.ADD) {
ZegoStream stream = streamList.get(0);
// By default, new streams are from the AI agent, so play directly
ZegoExpressEngine.getEngine().startPlayingStream(stream.streamID);
}
}
});
}
Client request to play the AI agent stream
// Listen for room stream information update status, and play the AI agent stream
- (void)onRoomStreamUpdate:(ZegoUpdateType)updateType
streamList:(NSArray<ZegoStream *> *)streamList
extendedData:(nullable NSDictionary *)extendedData
roomID:(NSString *)roomID{
if (updateType == ZegoUpdateTypeAdd) {
for (int i=0; i<streamList.count; i++) {
ZegoStream* item = [streamList objectAtIndex:i];
// !mark
[self startPlayStream:item.streamID];
}
} else if(updateType == ZegoUpdateTypeDelete) {
for (int i=0; i<streamList.count; i++) {
ZegoStream* item = [streamList objectAtIndex:i];
[[ZegoExpressEngine sharedEngine] stopPlayingStream:item.streamID];
}
}
}
Client request to play the AI agent stream
// Listen for room stream information update status, and play the AI agent stream
- (void)onRoomStreamUpdate:(ZegoUpdateType)updateType
streamList:(NSArray<ZegoStream *> *)streamList
extendedData:(nullable NSDictionary *)extendedData
roomID:(NSString *)roomID{
if (updateType == ZegoUpdateTypeAdd) {
for (int i=0; i<streamList.count; i++) {
ZegoStream* item = [streamList objectAtIndex:i];
// !mark
[self startPlayStream:item.streamID];
}
} else if(updateType == ZegoUpdateTypeDelete) {
for (int i=0; i<streamList.count; i++) {
ZegoStream* item = [streamList objectAtIndex:i];
[[ZegoExpressEngine sharedEngine] stopPlayingStream:item.streamID];
}
}
}
Client request to play the AI agent stream
ZegoExpressEngine.onRoomStreamUpdate = _onRoomStreamUpdate;
void _onRoomStreamUpdate(
String roomID,
ZegoUpdateType updateType,
List<ZegoStream> streamList,
Map<String, dynamic> extendedData,
) {
if (updateType == ZegoUpdateType.Add) {
for (var stream in streamList) {
ZegoExpressEngine.instance.startPlayingStream(stream.streamID);
}
} else if (updateType == ZegoUpdateType.Delete) {
for (var stream in streamList) {
ZegoExpressEngine.instance.stopPlayingStream(stream.streamID);
}
}
}
Client request to play the AI agent stream
ZegoExpressEngine.onRoomStreamUpdate = _onRoomStreamUpdate;
void _onRoomStreamUpdate(
String roomID,
ZegoUpdateType updateType,
List<ZegoStream> streamList,
Map<String, dynamic> extendedData,
) {
if (updateType == ZegoUpdateType.Add) {
for (var stream in streamList) {
ZegoExpressEngine.instance.startPlayingStream(stream.streamID);
}
} else if (updateType == ZegoUpdateType.Delete) {
for (var stream in streamList) {
ZegoExpressEngine.instance.stopPlayingStream(stream.streamID);
}
}
}
Client request to play the AI agent stream
// Listen to remote stream update events
function setupEvent() {
zg.on("roomStreamUpdate",
async (roomID, updateType, streamList) => {
if (updateType === "ADD" && streamList.length > 0) {
try {
for (const stream of streamList) {
// Play the AI agent stream
// !mark
const mediaStream = await zg.startPlayingStream(stream.streamID);
if (!mediaStream) return;
const remoteView = await zg.createRemoteStreamView(mediaStream);
if (remoteView) {
// A container with the id 'remoteSteamView' is required on the page to receive the AI agent stream [Reference Documentation](https://www.zegocloud.com/article/api?doc=Express_Video_SDK_API~javascript_web~class~ZegoStreamView)
remoteView.play("remoteSteamView", {
enableAutoplayDialog: false,
});
}
}
} catch (error) {
console.error("Failed to pull stream:", error);
}
}
}
);
}
Client request to play the AI agent stream
// Listen to remote stream update events
function setupEvent() {
zg.on("roomStreamUpdate",
async (roomID, updateType, streamList) => {
if (updateType === "ADD" && streamList.length > 0) {
try {
for (const stream of streamList) {
// Play the AI agent stream
// !mark
const mediaStream = await zg.startPlayingStream(stream.streamID);
if (!mediaStream) return;
const remoteView = await zg.createRemoteStreamView(mediaStream);
if (remoteView) {
// A container with the id 'remoteSteamView' is required on the page to receive the AI agent stream [Reference Documentation](https://www.zegocloud.com/article/api?doc=Express_Video_SDK_API~javascript_web~class~ZegoStreamView)
remoteView.play("remoteSteamView", {
enableAutoplayDialog: false,
});
}
}
} catch (error) {
console.error("Failed to pull stream:", error);
}
}
}
);
}
Congratulations🎉! After completing this step, you can ask the AI agent any question by voice, and the AI agent will answer your questions by voice!
Delete the agent instance and the user exits the room
The client calls the logout interface to exit the room and stops publishing and playing streams. At the same time, it notifies your server to end the call. After receiving the end call notification, your server will delete the AI agent instance, and the AI agent instance will automatically exit the room and stop publishing and playing streams. This completes a full interaction.
// Notify your server to end the call
private void stop() {
RequestBody body = RequestBody.create("", MediaType.parse("application/json; charset=utf-8"));
// !mark
Request request = new Request.Builder().url(YOUR_SERVER_URL + "/api/stop").post(body).build();
new OkHttpClient.Builder().build().newCall(request).enqueue(new Callback() {
@Override
public void onFailure(@NonNull Call call, @NonNull IOException e) {
}
@Override
public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException {
if (response.isSuccessful()) {
// Exit room
// !mark
ZegoExpressEngine.getEngine().logoutRoom();
}
}
});
}
// Notify your server to end the call
private void stop() {
RequestBody body = RequestBody.create("", MediaType.parse("application/json; charset=utf-8"));
// !mark
Request request = new Request.Builder().url(YOUR_SERVER_URL + "/api/stop").post(body).build();
new OkHttpClient.Builder().build().newCall(request).enqueue(new Callback() {
@Override
public void onFailure(@NonNull Call call, @NonNull IOException e) {
}
@Override
public void onResponse(@NonNull Call call, @NonNull Response response) throws IOException {
if (response.isSuccessful()) {
// Exit room
// !mark
ZegoExpressEngine.getEngine().logoutRoom();
}
}
});
}
/**
* Notify your server to end the call
*
* @param completion Completion callback, returns operation result
* @discussion This method sends a request to the server to end the call, used to release the AI agent instance
*/
- (void)doStopCallWithCompletion:(void (^)(NSInteger code, NSString *message, NSDictionary *data))completion {
// Build request URL
// !mark
NSString *url = [NSString stringWithFormat:@"%@/api/stop", self.currentBaseURL];
NSURL *requestURL = [NSURL URLWithString:url];
// Create request
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:requestURL];
request.HTTPMethod = @"POST";
// Set request headers
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
// Create request parameters
NSMutableDictionary *params = [NSMutableDictionary dictionary];
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];
request.HTTPBody = jsonData;
// Create session
NSURLSession *session = [NSURLSession sharedSession];
// Send request
NSURLSessionDataTask *task = [session dataTaskWithRequest:request
completionHandler:^(NSData * _Nullable data,
NSURLResponse * _Nullable response,
NSError * _Nullable error) {
dispatch_async(dispatch_get_main_queue(), ^{
if (error) {
if (completion) {
completion(-1, @"Network request failed", nil);
}
return;
}
NSHTTPURLResponse *httpUrlResponse = (NSHTTPURLResponse *)response;
if (httpUrlResponse.statusCode != 200) {
if (completion) {
completion(httpUrlResponse.statusCode,
[NSString stringWithFormat:@"Server error: %ld", (long)httpUrlResponse.statusCode],
nil);
}
return;
}
NSError *jsonError;
NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:data options:0 error:&jsonError];
if (jsonError) {
if (completion) {
completion(-2, @"Failed to parse response data", nil);
}
return;
}
// Parse response data
NSInteger code = [dict[@"code"] integerValue];
NSString *message = dict[@"message"];
NSDictionary *responseData = dict[@"data"];
if (completion) {
completion(code, message, responseData);
}
// Exit room
// !mark
[[ZegoExpressEngine sharedEngine] logoutRoom];
});
}];
[task resume];
}
/**
* Notify your server to end the call
*
* @param completion Completion callback, returns operation result
* @discussion This method sends a request to the server to end the call, used to release the AI agent instance
*/
- (void)doStopCallWithCompletion:(void (^)(NSInteger code, NSString *message, NSDictionary *data))completion {
// Build request URL
// !mark
NSString *url = [NSString stringWithFormat:@"%@/api/stop", self.currentBaseURL];
NSURL *requestURL = [NSURL URLWithString:url];
// Create request
NSMutableURLRequest *request = [[NSMutableURLRequest alloc] initWithURL:requestURL];
request.HTTPMethod = @"POST";
// Set request headers
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
// Create request parameters
NSMutableDictionary *params = [NSMutableDictionary dictionary];
NSData *jsonData = [NSJSONSerialization dataWithJSONObject:params options:0 error:nil];
request.HTTPBody = jsonData;
// Create session
NSURLSession *session = [NSURLSession sharedSession];
// Send request
NSURLSessionDataTask *task = [session dataTaskWithRequest:request
completionHandler:^(NSData * _Nullable data,
NSURLResponse * _Nullable response,
NSError * _Nullable error) {
dispatch_async(dispatch_get_main_queue(), ^{
if (error) {
if (completion) {
completion(-1, @"Network request failed", nil);
}
return;
}
NSHTTPURLResponse *httpUrlResponse = (NSHTTPURLResponse *)response;
if (httpUrlResponse.statusCode != 200) {
if (completion) {
completion(httpUrlResponse.statusCode,
[NSString stringWithFormat:@"Server error: %ld", (long)httpUrlResponse.statusCode],
nil);
}
return;
}
NSError *jsonError;
NSDictionary *dict = [NSJSONSerialization JSONObjectWithData:data options:0 error:&jsonError];
if (jsonError) {
if (completion) {
completion(-2, @"Failed to parse response data", nil);
}
return;
}
// Parse response data
NSInteger code = [dict[@"code"] integerValue];
NSString *message = dict[@"message"];
NSDictionary *responseData = dict[@"data"];
if (completion) {
completion(code, message, responseData);
}
// Exit room
// !mark
[[ZegoExpressEngine sharedEngine] logoutRoom];
});
}];
[task resume];
}
// Notify your server to end the call
Future<Map<String, dynamic>> stopCall() async {
try {
final response = await http.post(
Uri.parse('$_currentBaseUrl/api/stop'),
headers: {'Content-Type': 'application/json'},
);
if (response.statusCode == 200) {
final json = jsonDecode(response.body);
return json;
}
return {'code': -1, 'message': 'Request failed'};
} catch (e) {
return {'code': -1, 'message': e.toString()};
}
}
/// Stop the conversation with the AI agent
Future<bool> stop() async {
stopCall();
final String _roomId = 'room_id_1';
final engine = ZegoExpressEngine.instance;
/// Stop publishing stream
await engine.stopPublishingStream();
/// Log out of the room
await engine.logoutRoom(_roomId);
return true;
}
// Notify your server to end the call
Future<Map<String, dynamic>> stopCall() async {
try {
final response = await http.post(
Uri.parse('$_currentBaseUrl/api/stop'),
headers: {'Content-Type': 'application/json'},
);
if (response.statusCode == 200) {
final json = jsonDecode(response.body);
return json;
}
return {'code': -1, 'message': 'Request failed'};
} catch (e) {
return {'code': -1, 'message': e.toString()};
}
}
/// Stop the conversation with the AI agent
Future<bool> stop() async {
stopCall();
final String _roomId = 'room_id_1';
final engine = ZegoExpressEngine.instance;
/// Stop publishing stream
await engine.stopPublishingStream();
/// Log out of the room
await engine.logoutRoom(_roomId);
return true;
}
// Exit room
async function stopCall() {
try {
// !mark
const response = await fetch(`${YOUR_SERVER_URL}/api/stop`, { // YOUR_SERVER_URL is the address of your Your Server
method: 'POST',
headers: {
'Content-Type': 'application/json',
}
});
const data = await response.json();
console.log('End call result:', data);
return data;
} catch (error) {
console.error('Failed to end call:', error);
throw error;
}
}
stopCall();
zg.destroyLocalStream(localStream);
// !mark
zg.logoutRoom();
// Exit room
async function stopCall() {
try {
// !mark
const response = await fetch(`${YOUR_SERVER_URL}/api/stop`, { // YOUR_SERVER_URL is the address of your Your Server
method: 'POST',
headers: {
'Content-Type': 'application/json',
}
});
const data = await response.json();
console.log('End call result:', data);
return data;
} catch (error) {
console.error('Failed to end call:', error);
throw error;
}
}
stopCall();
zg.destroyLocalStream(localStream);
// !mark
zg.logoutRoom();
This is the complete core process for you to achieve real-time voice interaction with an AI agent.
Best Practices for ZEGO Express SDK Configuration
To achieve the best audio call experience, it is recommended to configure the ZEGO Express SDK according to the following best practices. These configurations can significantly improve the quality of AI agent voice interactions.
Before join room Settings:
Enable traditional audio 3A processing (Acoustic Echo Cancellation AEC, Automatic Gain Control AGC, and Noise Suppression ANS)
Set the room usage scenario to High Quality Chatroom, as the SDK will adopt different optimization strategies for different scenarios
Set the audio device mode to default mode
Enable AI echo cancellation to improve echo cancellation effect (this feature requires contacting ZEGO technical support to obtain the corresponding version of ZEGOExpress SDK)
Configure volume ducking to avoid sound conflicts
Enable adaptive playback volume to enhance user experience
Enable AI noise reduction and set appropriate noise suppression level
ZegoEngineConfig config = new ZegoEngineConfig();
HashMap<String, String> advanceConfig = new HashMap<String, String>();
// Configure volume ducking to avoid sound conflicts
advanceConfig.put("set_audio_volume_ducking_mode", "1");
// Enable adaptive playback volume
advanceConfig.put("enable_rnd_volume_adaptive", "true");
config.advancedConfig = advanceConfig;
ZegoExpressEngine.setEngineConfig(config);
// Set room usage scenario to High Quality Chatroom
ZegoExpressEngine.getEngine().setRoomScenario(ZegoScenario.HIGH_QUALITY_CHATROOM);
// Set audio device mode to default mode
ZegoExpressEngine.getEngine().setAudioDeviceMode(ZegoAudioDeviceMode.GENERAL);
// Enable traditional audio 3A processing
ZegoExpressEngine.getEngine().enableAEC(true);
ZegoExpressEngine.getEngine().enableAGC(true);
ZegoExpressEngine.getEngine().enableANS(true);
// Enable AI echo cancellation, please note: enabling AI echo cancellation requires contacting ZEGO technical support to obtain the corresponding version of ZEGOExpress SDK
ZegoExpressEngine.getEngine().setAECMode(ZegoAECMode.AI_AGGRESSIVE2);
// Enable AI noise reduction with moderate noise suppression
ZegoExpressEngine.getEngine().setANSMode(ZegoANSMode.MEDIUM);
ZegoEngineConfig config = new ZegoEngineConfig();
HashMap<String, String> advanceConfig = new HashMap<String, String>();
// Configure volume ducking to avoid sound conflicts
advanceConfig.put("set_audio_volume_ducking_mode", "1");
// Enable adaptive playback volume
advanceConfig.put("enable_rnd_volume_adaptive", "true");
config.advancedConfig = advanceConfig;
ZegoExpressEngine.setEngineConfig(config);
// Set room usage scenario to High Quality Chatroom
ZegoExpressEngine.getEngine().setRoomScenario(ZegoScenario.HIGH_QUALITY_CHATROOM);
// Set audio device mode to default mode
ZegoExpressEngine.getEngine().setAudioDeviceMode(ZegoAudioDeviceMode.GENERAL);
// Enable traditional audio 3A processing
ZegoExpressEngine.getEngine().enableAEC(true);
ZegoExpressEngine.getEngine().enableAGC(true);
ZegoExpressEngine.getEngine().enableANS(true);
// Enable AI echo cancellation, please note: enabling AI echo cancellation requires contacting ZEGO technical support to obtain the corresponding version of ZEGOExpress SDK
ZegoExpressEngine.getEngine().setAECMode(ZegoAECMode.AI_AGGRESSIVE2);
// Enable AI noise reduction with moderate noise suppression
ZegoExpressEngine.getEngine().setANSMode(ZegoANSMode.MEDIUM);
ZegoEngineProfile* profile = [[ZegoEngineProfile alloc]init];
profile.appID = kZegoAppId;
profile.scenario = ZegoScenarioHighQualityChatroom; // High Quality Chatroom scenario, setting this scenario can avoid requesting camera permissions, integrators should set specific values according to their business scenarios
ZegoEngineConfig* engineConfig = [[ZegoEngineConfig alloc] init];
engineConfig.advancedConfig = @{
@"set_audio_volume_ducking_mode":@1,/** Configure volume ducking to avoid sound conflicts **/
@"enable_rnd_volume_adaptive":@"true",/** Enable adaptive playback volume **/
};
[ZegoExpressEngine setEngineConfig:engineConfig];
[ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];
// Enable traditional audio 3A processing
[[ZegoExpressEngine sharedEngine] enableAGC:TRUE];
[[ZegoExpressEngine sharedEngine] enableAEC:TRUE];
[[ZegoExpressEngine sharedEngine] enableANS:TRUE];
// Enable AI echo cancellation, please note: enabling AI echo cancellation requires contacting ZEGO technical support to obtain the corresponding version of ZEGOExpress SDK
[[ZegoExpressEngine sharedEngine] setAECMode:ZegoAECModeAIAggressive2];
// Enable AI noise reduction with moderate noise suppression
[[ZegoExpressEngine sharedEngine] setANSMode:ZegoANSModeMedium];
ZegoEngineProfile* profile = [[ZegoEngineProfile alloc]init];
profile.appID = kZegoAppId;
profile.scenario = ZegoScenarioHighQualityChatroom; // High Quality Chatroom scenario, setting this scenario can avoid requesting camera permissions, integrators should set specific values according to their business scenarios
ZegoEngineConfig* engineConfig = [[ZegoEngineConfig alloc] init];
engineConfig.advancedConfig = @{
@"set_audio_volume_ducking_mode":@1,/** Configure volume ducking to avoid sound conflicts **/
@"enable_rnd_volume_adaptive":@"true",/** Enable adaptive playback volume **/
};
[ZegoExpressEngine setEngineConfig:engineConfig];
[ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];
// Enable traditional audio 3A processing
[[ZegoExpressEngine sharedEngine] enableAGC:TRUE];
[[ZegoExpressEngine sharedEngine] enableAEC:TRUE];
[[ZegoExpressEngine sharedEngine] enableANS:TRUE];
// Enable AI echo cancellation, please note: enabling AI echo cancellation requires contacting ZEGO technical support to obtain the corresponding version of ZEGOExpress SDK
[[ZegoExpressEngine sharedEngine] setAECMode:ZegoAECModeAIAggressive2];
// Enable AI noise reduction with moderate noise suppression
[[ZegoExpressEngine sharedEngine] setANSMode:ZegoANSModeMedium];
Enable traditional audio 3A processing (Acoustic Echo Cancellation AEC, Automatic Gain Control AGC, and Noise Suppression ANS)
Set the room usage scenario to High Quality Chatroom, as the SDK will adopt different optimization strategies for different scenarios
When pushing streams, configure the push parameters to automatically switch to available videoCodec
// Import necessary modules
import { ZegoExpressEngine } from "zego-express-engine-webrtc";
import { VoiceChanger } from "zego-express-engine-webrtc/voice-changer";
// Load audio processing module, must be called before new ZegoExpressEngine
ZegoExpressEngine.use(VoiceChanger);
// Instantiate ZegoExpressEngine, set room usage scenario to High Quality Chatroom
const zg = new ZegoExpressEngine(appid, server, { scenario: 7 })
// Traditional audio 3A processing is enabled by default in SDK
// Create local media stream
const localStream = await zg.createZegoStream();
// Push local media stream, need to set automatic switching to available videoCodec
await zg.startPublishingStream(userStreamId, localStream, {
enableAutoSwitchVideoCodec: true,
});
// Check system requirements
async function checkSystemRequirements() {
// Check WebRTC support
const rtcSupport = await zg.checkSystemRequirements("webRTC");
if (!rtcSupport.result) {
console.error("Browser does not support WebRTC");
return false;
}
// Check microphone permission
const micSupport = await zg.checkSystemRequirements("microphone");
if (!micSupport.result) {
console.error("Microphone permission not granted");
return false;
}
return true;
}
// Import necessary modules
import { ZegoExpressEngine } from "zego-express-engine-webrtc";
import { VoiceChanger } from "zego-express-engine-webrtc/voice-changer";
// Load audio processing module, must be called before new ZegoExpressEngine
ZegoExpressEngine.use(VoiceChanger);
// Instantiate ZegoExpressEngine, set room usage scenario to High Quality Chatroom
const zg = new ZegoExpressEngine(appid, server, { scenario: 7 })
// Traditional audio 3A processing is enabled by default in SDK
// Create local media stream
const localStream = await zg.createZegoStream();
// Push local media stream, need to set automatic switching to available videoCodec
await zg.startPublishingStream(userStreamId, localStream, {
enableAutoSwitchVideoCodec: true,
});
// Check system requirements
async function checkSystemRequirements() {
// Check WebRTC support
const rtcSupport = await zg.checkSystemRequirements("webRTC");
if (!rtcSupport.result) {
console.error("Browser does not support WebRTC");
return false;
}
// Check microphone permission
const micSupport = await zg.checkSystemRequirements("microphone");
if (!micSupport.result) {
console.error("Microphone permission not granted");
return false;
}
return true;
}
Additional Optimization Recommendations
Browser Compatibility: Recommended to use the latest versions of modern browsers such as Chrome, Firefox, Safari
Network Environment: Ensure stable network connection, recommend using wired network or Wi-Fi with good signal
Audio Equipment: Use high-quality microphones and speakers
Page Optimization: Avoid running too many JavaScript tasks on the same page, which may affect audio processing performance
HTTPS Environment: Use HTTPS protocol in production environment to ensure microphone permission access
Listen for Exception Callback
Note
Due to the large number of parameters for LLM and TTS, it is easy to cause various abnormal problems such as the AI agent not answering or not speaking during the test process due to parameter configuration errors. We strongly recommend that you listen for exception callbacks during the test process and quickly troubleshoot problems based on the callback information.
Receive Callback
Click to view the guide for listening to exception callbacks. The event with Event as Exception can be quickly located through Data.Code and Data.Message.