Getting Audio Raw Data
Function Overview
The SDK provides developers with the function to Get the Raw Audio Data. The obtained raw audio data format is PCM. Developers can write this data to local devices to implement audio recording.
Prerequisites
Before Get the Raw Audio Data, please ensure:
- You have created a project in ZEGOCLOUD Console and applied for valid AppID and AppSign. For details, please refer to Console - Project Information.
- You have integrated ZEGO Express SDK in the project and implemented basic audio and video streaming functions. For details, please refer to Quick Start - Integration and Quick Start - Implementation.
Usage Steps
1 Initialize SDK
Please refer to "Create Engine" in Quick Start - Implementation.
2 Enable Get the Raw Audio Data Function
Developers can call the startAudioDataObserver interface to enable raw audio data callback monitoring. The callback audio data types are ZegoAudioDataCallbackBitMask, including CAPTURED, PLAYBACK, MIXED, PLAYER. All four callback types need to be enabled.
ZegoAudioFrameParam param = new ZegoAudioFrameParam();
param.channel = ZegoAudioChannel.Stereo;
param.sampleRate = ZegoAudioSampleRate.ZegoAudioSampleRate16K;pap
uint bitmask = 0;
// Add bitmask to enable the callback switch for captured audio data.
// The bitmask values corresponding to capture, playback, mix, and play are: CAPTURED=1, PLAYBACK=2, MIXED=4, PLAYER=8. The final value of bitmask is 15, which means that raw data callbacks for capture, playback, mix, and play will be triggered simultaneously.
bitmask |= (uint)ZegoAudioDataCallbackBitMask.Captured;
bitmask |= (uint)ZegoAudioDataCallbackBitMask.Playback;
bitmask |= (uint)ZegoAudioDataCallbackBitMask.Mixed;
bitmask |= (uint)ZegoAudioDataCallbackBitMask.Player;
engine.StartAudioDataObserver(bitmask, param);Application Scenarios
All numbers in a program are stored in binary form in the computer's memory, while bitwise operations directly operate on the binary bits of integers in memory.
| Symbol | Meaning | Operation Rules | Example |
|---|---|---|---|
| & | AND | The result is 1 only when both bits are 1. | 0&1=0, 1&1=1. |
| | | OR | The result is 0 only when both bits are 0. | 0|0=0, 0|1=1. |
| ^ | XOR | Same bits are 0, different bits are 1. | 1^1=0, 0^1=1. |
| ~ | NOT | 0 becomes 1, 1 becomes 0. | ~1=0, ~0=1. |
| << | Left Shift | Shift all bits to the left by the corresponding number of positions in binary form, high bits are shifted out (discarded), and low bits are filled with zeros. | For example, the binary of integer 3 is 00000011, 3 << 2 left shift by 2 positions is 00001100. |
| >> | Signed Right Shift | Shift all bits to the right by the corresponding number of positions in binary form, low bits are shifted out (discarded), and high bits are filled with the sign bit, i.e., positive numbers are filled with 0, negative numbers are filled with 1. | For example, the binary of integer 3 is 00000011, 3 >> 1 signed right shift by 1 position is 00000001. |
| >>> | Unsigned Right Shift | Shift all bits to the right by the corresponding number of positions in binary form, low bits are shifted out (discarded), and high bits are filled with zeros. For positive numbers, this is the same as signed right shift, but different for negative numbers. | For example, the binary of integer 3 is 00000011, 3 >>> 1 unsigned right shift by 1 position is 00000001. |
The SDK mainly uses the bitmask principle to implement multiple switch operations, that is, when multiple module switches need to be selected in an API, developers need to pass the bitmask result to the SDK.
Examples
Taking the raw audio and video data acquisition function as an example, the bitmask usage examples for different languages are as follows:
enum ZegoAudioDataCallbackBitMask
{
/** This property controls whether the SDK triggers the [onCapturedAudioData] method callback */
ZEGO_AUDIO_DATA_CALLBACK_BIT_MASK_CAPTURED = 1 << 0,
/** This property controls whether the SDK triggers the [onPlaybackAudioData] method callback */
ZEGO_AUDIO_DATA_CALLBACK_BIT_MASK_PLAYBACK = 1 << 1,
/** This property controls whether the SDK triggers the [onMixedAudioData] method callback */
ZEGO_AUDIO_DATA_CALLBACK_BIT_MASK_MIXED = 1 << 2,
/** This property controls whether the SDK triggers the [onPlayerAudioData] method callback */
ZEGO_AUDIO_DATA_CALLBACK_BIT_MASK_PLAYER = 1 << 3
};
// Enable the SDK to acquire raw audio data, specifying that the SDK needs to trigger both captured audio data callback and playback audio data callback
unsigned int bitmask = ZEGO_AUDIO_DATA_CALLBACK_BIT_MASK_CAPTURED | ZEGO_AUDIO_DATA_CALLBACK_BIT_MASK_PLAYBACK;
engine->startAudioDataObserver(bitmask, param);3 Set Callback to Receive and Process Raw Audio Data
Developers can implement callbacks onCapturedAudioData, onPlaybackAudioData, onMixedAudioData, onPlayerAudioData as needed, corresponding to the four audio data types in the above ZegoAudioDataCallbackBitMask respectively.
void OnCapturedAudioData(IntPtr data, uint dataLength, ZegoAudioFrameParam param)
{
// Local captured audio data, callback can be received after publishing stream
}
void OnPlaybackAudioData(IntPtr data, uint dataLength, ZegoAudioFrameParam param)
{
// Audio data played by the SDK. When the engine is started and not in the playing stream state and the media player is not playing media files, the callback audio data is muted audio data
}
void OnMixedAudioData(IntPtr data, uint dataLength, ZegoAudioFrameParam param)
{
// Callback of audio data mixed with local captured audio and SDK played audio
}
void OnPlayerAudioData(IntPtr data, uint dataLength, ZegoAudioFrameParam param, string streamID)
{
// Remote playing stream audio data, callback of each playing stream data after starting to play stream
}4 Stop Audio Data Callback Monitoring
If you want to stop audio data callback monitoring, you can call the stopAudioDataObserver interface.
engine.StopAudioDataObserver();