Customize how the audio captures and renders

Introduction

Custom Audio Capture

In the following situations, it is recommended to use the custom audio capture function:

• The customer needs to obtain input after acquisition from the existing audio stream, audio file, or customized acquisition system, and hand it over to the SDK for transmission.
• Customers have their own requirements for special sound processing for PCM input sources. After the sound processing, the input will be sent to the SDK for transmission.

Custom Audio Rendering

When developers have their own rendering requirements, such as special applications or processing and rendering of the original PCM data that are pulled, it is recommended to use the custom audio rendering function of the SDK.

Prerequisites

Before custom audio capture and rendering, please make sure:

• A project has been created in ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - How to view project information .
• ZEGO Express SDK has been integrated into the project to implement basic real-time audio and video functions. For details, please refer to Integrate the SDK and Implement a basic video call.

Enable Custom Audio Capture and Rendering Function

// Set the audio source to custom capture and render
ZegoCustomAudioConfig *audioConfig = [[ZegoCustomAudioConfig alloc] init];
audioConfig.sourceType = ZegoAudioSourceTypeCustom;

[[ZegoExpressEngine sharedEngine] enableCustomAudioIO:YES config:audioConfig];

Collect Audio Data

After publishing or playing a stream, turn on the audio capture device and pass the collected audio data to the engine through sendCustomAudioCaptureAACData or sendCustomAudioCapturePCMData.

Render Audio Data

Use fetchCustomAudioRenderPCMData to get the data to be rendered from the engine, and then use the rendering device to play the audio data.

FAQ

1. When is the time to call related interfaces of custom audio capture and rendering?

   The call timing of each interface is as follows:

   • enableCustomAudioIO: It should be called before the engine starts, that is, before the preview and push-pull streaming.
   • sendCustomAudioCaptureAACData/sendCustomAudioCapturePCMData: It should be called after starting to preview and push streaming. If it is called before starting to preview and push, the SDK will directly discard the received data.
   • fetchCustomAudioRenderPCMData: It should be called after the start streaming is called, and all the invalid mute data is obtained before the streaming starts.

2. How often do you call related interfaces for custom audio capture and rendering?

   The best way is to drive according to the clock of the physical audio device, call sendCustomAudioCaptureAACData and sendCustomAudioCapturePCMData when the physical capture device collects data; call fetchCustomAudioRenderPCMData when the physical rendering device needs data.

   If the developer does not have a specific physical device to drive in the actual scenario, it is recommended to call the above interface every 10 ms ~ 20 ms.

3. When calling fetchCustomAudioRenderPCMData, if the internal data of the SDK is insufficient for "dataLength", how will the SDK handle it?

   Under the condition that the "param" is filled in normally, when the data inside the SDK is insufficient for "dataLength", the insufficient remaining length will be filled in according to the mute data.