AI Voice Changer

Function Overview

The "Conan Voice Changer Bowtie" in real-time calls perfectly reproduces the target character's timbre and rhythm, while retaining the user's speech rate, emotion, and tone. Switch timbres at will, and ultra-low latency allows users to enjoy social voice chat, live streaming, game voice, and other scenarios freely.

Function Advantages

• Ultra-high sound quality and ultra-low latency.
• Vivid and realistic, perfectly reproducing the target character's timbre and rhythm while retaining the user's speech rate, emotion, and tone.
• Massive timbres for flexible selection, supporting timbre customization.

Effect Demonstration

Applicable Scenarios

This function can be used in the following real-time scenarios to achieve users' timbre transformation.

• Social voice chat
• Game voice
• Audio/video live streaming
• Virtual human

Prerequisites

Before implementing the AI Voice Changer function, please ensure:

• A project has been created in the ZEGOCLOUD Console and valid AppID and AppSign have been applied. For details, please refer to Console - Project Information.
• ZEGO Express SDK has been integrated in the project and basic audio/video publishing and playing functions have been implemented. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Usage Steps

Developers can complete the AI Voice Changer related settings according to the following steps:

1 Enable Permissions

Please confirm that you have contacted ZEGO technical support for special packaging and enabled AI Voice Changer permissions.

2 Initialize and Login to Room

For the specific process of initializing and logging in to the room, please refer to "Create Engine" and "Login Room" in the Implement Video Call documentation.

3 Initialize AI Voice Changer Engine Instance

1. Call the createAIVoiceChanger interface to create an AI Voice Changer engine instance.

   Currently only supports creating one instance at a time. Calling destroyAIVoiceChanger interface to destroy the instance before creating again will return null.

   // Create AI Voice Changer engine instance
   var changer = await ZegoExpressEngine.instance.createAIVoiceChanger();

   // Create AI Voice Changer engine instance
   var changer = await ZegoExpressEngine.instance.createAIVoiceChanger();

2. Listen to AI Voice Changer engine event callbacks.

   // Set AI Voice Changer engine event callback
   ZegoExpressEngine.onAIVoiceChangerInit = (aiVoiceChanger, errorCode) {
     print('📽️ [onAIVoiceChangerInit] '
         'idx: ${aiVoiceChanger.getIndex()}, errorCode: $errorCode');
   };
   
   ZegoExpressEngine.onAIVoiceChangerUpdate = (aiVoiceChanger, errorCode) {
     print('📽️ [onAIVoiceChangerUpdate] '
         'idx: ${aiVoiceChanger.getIndex()}, errorCode: $errorCode');
   };
   
   ZegoExpressEngine.onAIVoiceChangerGetSpeakerList =
       (aiVoiceChanger, errorCode, speakerList) {
     String content = '';
     int index = 0;
     for (ZegoAIVoiceChangerSpeakerInfo info in speakerList) {
       index += 1;
       content += '$index:[ID:${info.id}][name:${info.name}];';
     }
     print('📽️ [onAIVoiceChangerGetSpeakerList] '
         'idx: ${aiVoiceChanger.getIndex()}, errorCode: $errorCode, speakerList: $content');
   };

   // Set AI Voice Changer engine event callback
   ZegoExpressEngine.onAIVoiceChangerInit = (aiVoiceChanger, errorCode) {
     print('📽️ [onAIVoiceChangerInit] '
         'idx: ${aiVoiceChanger.getIndex()}, errorCode: $errorCode');
   };
   
   ZegoExpressEngine.onAIVoiceChangerUpdate = (aiVoiceChanger, errorCode) {
     print('📽️ [onAIVoiceChangerUpdate] '
         'idx: ${aiVoiceChanger.getIndex()}, errorCode: $errorCode');
   };
   
   ZegoExpressEngine.onAIVoiceChangerGetSpeakerList =
       (aiVoiceChanger, errorCode, speakerList) {
     String content = '';
     int index = 0;
     for (ZegoAIVoiceChangerSpeakerInfo info in speakerList) {
       index += 1;
       content += '$index:[ID:${info.id}][name:${info.name}];';
     }
     print('📽️ [onAIVoiceChangerGetSpeakerList] '
         'idx: ${aiVoiceChanger.getIndex()}, errorCode: $errorCode, speakerList: $content');
   };

3. Call the ZegoAIVoiceChanger.initEngine interface to initialize the AI Voice Changer engine instance.

// Initialize AI Voice Changer engine
await _changer?.initEngine();

// Initialize AI Voice Changer engine
await _changer?.initEngine();

4 Update AI Voice Changer Engine Model

Call the ZegoAIVoiceChanger.update interface to update the AI Voice Changer engine model. The AI Voice Changer engine model file is relatively large, and the first update will take a long time. Please wait patiently.

// Update AI Voice Changer engine model
await _changer?.update();

// Update AI Voice Changer engine model
await _changer?.update();

5 Get Timbre List

Call the ZegoAIVoiceChanger.getSpeakerList interface to get the list of available timbres.

The list of available timbres will be returned through the ZegoExpressEngine.onAIVoiceChangerGetSpeakerList callback interface.

// Get list of available timbres
await _changer?.getSpeakerList();

// Get list of available timbres
await _changer?.getSpeakerList();

6 Set Target Timbre

Call the ZegoAIVoiceChanger.setSpeaker interface to set the timbre. The choice of timbre can be obtained through 5 Get Timbre List.

Setting the timbre ID to 0 means using the original sound.

// Set timbre
int speakerID = 0; // Timbre ID
await _changer?.setSpeaker(speakerID);

// Set timbre
int speakerID = 0; // Timbre ID
await _changer?.setSpeaker(speakerID);

7 Destroy AI Voice Changer Engine Instance

After using the function, call the destroyAIVoiceChanger interface to destroy the AI Voice Changer engine instance and release resources such as the microphone.

// Destroy AI Voice Changer engine instance
await ZegoExpressEngine.instance.destroyAIVoiceChanger(_changer);

// Destroy AI Voice Changer engine instance
await ZegoExpressEngine.instance.destroyAIVoiceChanger(_changer);