Custom Audio Processing
Feature Overview
Custom audio processing is generally used to remove interference from speech. Since the SDK has already performed echo cancellation, noise suppression, and other processing on the collected audio raw data, developers usually do not need to process it again. If developers want to implement special functions through custom processing after collecting audio data or before rendering remote audio data (such as voice changing, voice beautification, etc.), they can refer to this document.
The data for custom audio processing is the audio data after the original audio has been processed by 3A (AEC echo cancellation, AGC automatic gain control, ANS noise reduction):
- If developers need to process the raw data, please first call the enableAEC, enableAGC, enableANS interfaces to turn off audio 3A processing. If sound effect processing such as voice changing, reverb, stereo, etc. is enabled (default is disabled), it also needs to be turned off first before raw audio data can be obtained.
- If developers need to obtain both raw data and audio data processed by 3A for processing, please refer to Custom Audio Capture and Rendering.
Usage Steps
1 Create SDK Engine
Call the createEngine interface to create an SDK engine instance. For details, please refer to "Create Engine" in Quick Start - Implementation Flow.
ZegoEngineProfile profile;
profile.appID = ZegoUtilHelper::AppID();
profile.appSign = ZegoUtilHelper::AppSign();
profile.scenario = ZegoScenario::ZEGO_SCENARIO_GENERAL;
ZegoExpressSDK::createEngine(profile, nullptr);2 Set Audio Custom Processing Object and Implement Callback Methods
Call the setCustomAudioProcessHandler interface to set the audio custom processing object, and implement callback methods: custom audio processing local captured PCM audio frame callback onProcessCapturedAudioData and custom audio processing remote played stream PCM audio frame callback onProcessRemoteAudioData. In the callback method, directly processing the passed data can achieve processing of publishing and playing stream audio data.
// Please note, do not call any SDK interfaces in the SDK callback thread, you need to manually switch to another thread, otherwise a deadlock will occur
class MyAudioProcessHandler: public IZegoCustomAudioProcessHandler{
public:
void onProcessCapturedAudioData(unsigned char* data, unsigned int dataLength, ZegoAudioFrameParam* param) override{
}
void onProcessRemoteAudioData(unsigned char* data, unsigned int dataLength, ZegoAudioFrameParam* param, const std::string& streamID) override {
}
};
engine->setCustomAudioProcessHandler(std::make_shared<MyAudioProcessHandler>());3 Custom Audio Processing
-
Before starting publishing stream or starting local preview, call the enableCustomAudioCaptureProcessing interface to enable local capture custom audio processing. After enabling, developers can receive locally captured audio frames through the onProcessCapturedAudioData callback and modify the audio data.
-
Before starting playing stream, call the enableCustomAudioRemoteProcessing interface to enable remote playing stream custom audio processing. After enabling, developers can receive remote playing stream audio frames through onProcessRemoteAudioData and modify the audio data.
ZegoCustomAudioProcessConfig config;
config.channel = ZEGO_AUDIO_CHANNEL_MONO;
config.samples = 2048;
config.sampleRate = ZEGO_AUDIO_SAMPLE_RATE_44K;
engine->enableCustomAudioRemoteProcessing(true, &config);
engine->enableCustomAudioCaptureProcessing(true, &config);