Customize how the audio processes

Introduction

Custom audio processing is generally used to remove interference in the voice. Since the SDK has processed the collected audio raw data with echo cancellation and noise suppression, usually, developers do not need to repeat the processing.

If the developer wants to implement special functions (such as voice change, bel canto, etc.) through custom processing after collecting audio data or before pulling remote audio data for rendering, they can refer to this document.

Prerequisites

Before custom audio processing, please make sure:

• A project has been created in ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - How to view project information .
• ZEGO Express SDK has been integrated into the project to implement basic real-time audio and video functions. For details, please refer to Integrate the SDK and Implement a basic video call.

Set Audio Custom Processing Objects

Call the setCustomAudioProcessHandler method to set the audio custom processing object and implement the event callback. Audio data can be processed by directly processing the passed data in the callback.

[[ZegoExpressEngine sharedEngine] setCustomAudioProcessHandler:self];
-(void)onProcessCapturedAudioData:(unsigned char *)data dataLength:(unsigned int)dataLength param:(ZegoAudioFrameParam *)param {

}

-(void)onProcessRemoteAudioData:(unsigned char *)data dataLength:(unsigned int)dataLength param:(ZegoAudioFrameParam *)param streamID:(NSString *)streamID {

}

Custom Audio Processing

• Call enableCustomAudioCaptureProcessing interface before starting to push streaming or start local preview Local capture custom audio processing. After enabling, developers can use onProcessCapturedAudioData Call back to receive locally collected audio frames, and can modify the audio data.
• Call enableCustomAudioRemoteProcessing interface before starting to stream or start local preview Remote streaming custom audio processing. After enabling, developers can use onProcessRemoteAudioData Receive the audio frame pulled by the remote end, and can modify the audio data.

ZegoCustomAudioProcessConfig *config = [[ZegoCustomAudioProcessConfig alloc] init];
config.channel = ZegoAudioChannelMono;
config.sampleRate = ZegoAudioSampleRate16K;
config.samples = 0;

[[ZegoExpressEngine sharedEngine] enableCustomAudioCaptureProcessing:YES config:config];
[[ZegoExpressEngine sharedEngine] enableCustomAudioRemoteProcessing:YES config:config];