AI Voice Changer

Feature Overview

The "Conan Voice Changer Bow Tie" in real-time calls perfectly reproduces the target character's timbre and rhythm, while preserving the user's speech rate, emotion, and intonation. Switch timbres as you wish, with ultra-low latency allowing users to enjoy social chat, live streaming, game voice, and other scenarios freely.

Feature Advantages

• Ultra-high sound quality, ultra-low latency.
• Vivid and real, perfectly reproducing the target character's timbre and rhythm, while preserving the user's speech rate, emotion, and intonation.
• Massive timbres for flexible selection, supporting timbre customization.

Effect Demonstration

Application Scenarios

This feature can be used in the following real-time scenarios to achieve user timbre transformation.

• Social chat
• Game voice
• Audio and video live streaming
• Virtual avatar

Prerequisites

Before implementing the AI Voice Changer feature, please ensure:

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
• You have integrated ZEGO Express SDK in your project and implemented basic audio and video streaming functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Usage Steps

Developers can complete the AI Voice Changer related settings according to the following steps:

1 Enable permissions

Please confirm that you have contacted ZEGO Technical Support for special packaging and enabled AI Voice Changer permissions.

2 Initialize and login to Room

For the specific process of initialization and logging in to Room, please refer to "Create Engine" and "Login to Room" in the implementing video call documentation.

3 Initialize AI Voice Changer engine instance

1. Call the createAIVoiceChanger interface to create an AI Voice Changer engine instance.

   Currently only supports creating one instance at the same time. Calling destroyAIVoiceChanger interface to destroy the instance and creating again will return null.

   // Create AI Voice Changer engine instance
   aiVoiceChanger = ZegoExpressEngine.getEngine().createAIVoiceChanger();

   // Create AI Voice Changer engine instance
   aiVoiceChanger = ZegoExpressEngine.getEngine().createAIVoiceChanger();

2. Call the ZegoAIVoiceChanger.setEventHandler interface to set AI Voice Changer engine event callbacks.

   // Set AI Voice Changer engine event callbacks
   aiVoiceChanger.setEventHandler(new IZegoAIVoiceChangerEventHandler() {
       @Override
       public void onInit(ZegoAIVoiceChanger aiVoiceChanger, int errorCode) {
           super.onInit(aiVoiceChanger, errorCode);
       }
   
       @Override
       public void onUpdate(ZegoAIVoiceChanger aiVoiceChanger, int errorCode) {
           super.onUpdate(aiVoiceChanger, errorCode);
       }
   
       @Override
       public void onGetSpeakerList(ZegoAIVoiceChanger aiVoiceChanger, int errorCode, ArrayList<ZegoAIVoiceChangerSpeakerInfo> speakerList) {
           super.onGetSpeakerList(aiVoiceChanger, errorCode, speakerList);
       }
   });

   // Set AI Voice Changer engine event callbacks
   aiVoiceChanger.setEventHandler(new IZegoAIVoiceChangerEventHandler() {
       @Override
       public void onInit(ZegoAIVoiceChanger aiVoiceChanger, int errorCode) {
           super.onInit(aiVoiceChanger, errorCode);
       }
   
       @Override
       public void onUpdate(ZegoAIVoiceChanger aiVoiceChanger, int errorCode) {
           super.onUpdate(aiVoiceChanger, errorCode);
       }
   
       @Override
       public void onGetSpeakerList(ZegoAIVoiceChanger aiVoiceChanger, int errorCode, ArrayList<ZegoAIVoiceChangerSpeakerInfo> speakerList) {
           super.onGetSpeakerList(aiVoiceChanger, errorCode, speakerList);
       }
   });

3. Call the ZegoAIVoiceChanger.initEngine interface to initialize the AI Voice Changer engine instance.

   Warning

   The ZegoAIVoiceChanger.initEngine interface must be called before the startPublishingStream interface to be effective.

   // Initialize AI Voice Changer engine
   aiVoiceChanger.initEngine();

   // Initialize AI Voice Changer engine
   aiVoiceChanger.initEngine();

4 Update AI Voice Changer engine model

Call the ZegoAIVoiceChanger.update interface to update the AI Voice Changer engine model. The AI Voice Changer engine model file is relatively large, and the first update will take a long time. Please wait patiently.

// Update AI Voice Changer engine model
aiVoiceChanger.update();

// Update AI Voice Changer engine model
aiVoiceChanger.update();

5 Get timbre list

Call the ZegoAIVoiceChanger.getSpeakerList interface to get the list of available timbres.

The list of available timbres will be returned through the IZegoAIVoiceChangerEventHandler.onGetSpeakerList callback interface.

// Get list of available timbres
aiVoiceChanger.getSpeakerList();

// Get list of available timbres
aiVoiceChanger.getSpeakerList();

6 Set target timbre

Call the ZegoAIVoiceChanger.setSpeaker interface to set the timbre. The choice of timbre can be obtained through 5 Get timbre list.

Setting the timbre ID to 0 means using the original voice.

// Set timbre
int speakerID = 0; // Timbre ID
aiVoiceChanger.setSpeaker(speakerID);

// Set timbre
int speakerID = 0; // Timbre ID
aiVoiceChanger.setSpeaker(speakerID);

7 Destroy AI Voice Changer engine instance

After using the feature, call the destroyAIVoiceChanger interface to destroy the AI Voice Changer engine instance and release resources such as the microphone.

// Destroy AI Voice Changer engine instance
ZegoExpressEngine.getEngine().destroyAIVoiceChanger(aiVoiceChanger);

// Destroy AI Voice Changer engine instance
ZegoExpressEngine.getEngine().destroyAIVoiceChanger(aiVoiceChanger);

FAQ

1. How to handle when the following error message appears when building the apk package?

   java.lang.NoClassDefFoundError: Failed resolution of: Lkotlin/jvm/internal/Intrinsics;

   java.lang.NoClassDefFoundError: Failed resolution of: Lkotlin/jvm/internal/Intrinsics;

   When the above error occurs, it is due to missing kotlin-stdlib. You can add the following dependency in the build.gradle file and rebuild the apk.

   dependencies {
       implementation 'org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.3.30'
   }

   dependencies {
       implementation 'org.jetbrains.kotlin:kotlin-stdlib-jdk7:1.3.30'
   }