Sound Level and Audio Spectrum

2026-03-05

Feature Overview

Concept	Description	Application Scenarios	Scenario Diagram
Sound Level	Refers to the volume level of a specific stream, hereinafter referred to as "sound level".	During the publish and play stream process, determine which users on the microphone are speaking and display it in the UI.
Audio Spectrum	Refers to the energy value of digital audio signals at various frequency points.	In Karaoke scenarios, after publishing or playing streams, allow the broadcaster or audience to see animations of pitch and volume changes.

Example Source Code

Please refer to Download Example Source Code to get the source code.

For related source code, please check the files in the "/ZegoExpressExample/AdvancedAudioProcessing/src/main/java/im/zego/advancedaudioprocessing/soundlevelandspectrum" directory.

Prerequisites

Before implementing sound level and audio spectrum functionality, please ensure:

You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
You have integrated ZEGO Express SDK in your project and implemented basic audio and video publish and play stream functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Usage Steps for Non-Mixing Scenario

1 Listen to sound level and audio spectrum callback interfaces

Interface prototypes

Local captured sound level callback interface onCapturedSoundLevelUpdate:

/**
 * Local captured audio sound level callback
 *
 * Callback notification period is 100 ms.
 * @param soundLevel Local captured sound level value, ranging from 0.0 to 100.0
 */
public void onCapturedSoundLevelUpdate(double soundLevel){

}

/**
 * Local captured audio sound level callback
 *
 * Callback notification period is 100 ms.
 * @param soundLevel Local captured sound level value, ranging from 0.0 to 100.0
 */
public void onCapturedSoundLevelUpdate(double soundLevel){

}

Remote audio sound level callback interface onRemoteSoundLevelUpdate:

/**
 * Remote play stream audio sound level callback
 *
 * Callback notification period is 100 ms.
 * @param soundLevels Key-value pairs of remote sound levels, key is stream ID, value is the sound level value of the corresponding stream, value ranges from 0.0 to 100.0
 */
public void onRemoteSoundLevelUpdate(HashMap<String, Double> soundLevels){

}

/**
 * Remote play stream audio sound level callback
 *
 * Callback notification period is 100 ms.
 * @param soundLevels Key-value pairs of remote sound levels, key is stream ID, value is the sound level value of the corresponding stream, value ranges from 0.0 to 100.0
 */
public void onRemoteSoundLevelUpdate(HashMap<String, Double> soundLevels){

}

Local captured audio spectrum callback interface onCapturedAudioSpectrumUpdate:

/**
 * Local captured audio spectrum callback
 *
 * Callback notification period is 100 ms.
 * @param audioSpectrum Array of local captured audio spectrum values, spectrum value range is [0-2^30]
 */
public void onCapturedAudioSpectrumUpdate(float[] audioSpectrum){

}

/**
 * Local captured audio spectrum callback
 *
 * Callback notification period is 100 ms.
 * @param audioSpectrum Array of local captured audio spectrum values, spectrum value range is [0-2^30]
 */
public void onCapturedAudioSpectrumUpdate(float[] audioSpectrum){

}

Remote play stream audio spectrum callback interface onRemoteAudioSpectrumUpdate:

/**
 * Remote play stream audio spectrum callback
 *
 * Callback notification period is 100 ms.
 * @param audioSpectrums Key-value pairs of remote audio spectrums, key is stream ID, value is the array of audio spectrum values of the corresponding stream, spectrum value range is [0-2^30]
 */
public void onRemoteAudioSpectrumUpdate(HashMap<String, float[]> audioSpectrums){

}

/**
 * Remote play stream audio spectrum callback
 *
 * Callback notification period is 100 ms.
 * @param audioSpectrums Key-value pairs of remote audio spectrums, key is stream ID, value is the array of audio spectrum values of the corresponding stream, spectrum value range is [0-2^30]
 */
public void onRemoteAudioSpectrumUpdate(HashMap<String, float[]> audioSpectrums){

}

Usage example

The remote play stream sound level and remote audio spectrum callbacks return a HashMap, where key is the stream ID of other users currently publishing in the room, and value is the sound level/audio spectrum data corresponding to that stream.

You can first get the list of streams existing in the current room through the onRoomStreamUpdate callback method and save it, then use the saved stream list to index the HashMap to get the sound level/audio spectrum data corresponding to each stream.

The following example will demonstrate how to obtain sound level/audio spectrum data from callback methods and pass it to the UI. For specific logic on rendering to the UI, please refer to the Sound Level and Spectrum module related code.

class MyEventHandler extends IZegoEventHandler
{
    @override
    public void onCapturedSoundLevelUpdate(double soundLevel) {
        // Developers can render the local sound level data to specific UI controls here after obtaining it
    }
    @override
    public void onRemoteSoundLevelUpdate(HashMap<String, Double> soundLevels) {
        // Developers can render the remote sound level data to specific UI controls here after obtaining it
    }
    @override
    public void onCapturedAudioSpectrumUpdate(double[] audioSpectrum) {
        // Developers can render the local audio spectrum data to specific UI controls here after obtaining it
    }
    @override
    public void onRemoteAudioSpectrumUpdate(HashMap<String, double[]> audioSpectrums) {
        // Developers can render the remote audio spectrum data to specific UI controls here after obtaining it
    }
}

class MyEventHandler extends IZegoEventHandler
{
    @override
    public void onCapturedSoundLevelUpdate(double soundLevel) {
        // Developers can render the local sound level data to specific UI controls here after obtaining it
    }
    @override
    public void onRemoteSoundLevelUpdate(HashMap<String, Double> soundLevels) {
        // Developers can render the remote sound level data to specific UI controls here after obtaining it
    }
    @override
    public void onCapturedAudioSpectrumUpdate(double[] audioSpectrum) {
        // Developers can render the local audio spectrum data to specific UI controls here after obtaining it
    }
    @override
    public void onRemoteAudioSpectrumUpdate(HashMap<String, double[]> audioSpectrums) {
        // Developers can render the remote audio spectrum data to specific UI controls here after obtaining it
    }
}

2 Start listening to sound level and audio spectrum callback switches

You can start listening to corresponding callbacks for sound level or audio spectrum respectively.

Call the startSoundLevelMonitor interface to start sound level monitoring:

// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.startSoundLevelMonitor();

// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.startSoundLevelMonitor();

In version 1.15.0 and above, when calling the startSoundLevelMonitor interface, you can set the millisecond parameter to adjust the sound level callback interval.

// millisecond: Sound level monitoring time period, unit is milliseconds, value range is [100, 3000]. Default is 100 ms.
mSDKEnging.startSoundLevelMonitor(100);

// millisecond: Sound level monitoring time period, unit is milliseconds, value range is [100, 3000]. Default is 100 ms.
mSDKEnging.startSoundLevelMonitor(100);

After calling the above interface, onCapturedSoundLevelUpdate will be triggered immediately, the callback value is 0 when not publishing and not previewing; onRemoteSoundLevelUpdate will only callback after starting to play stream startPlayingStream .

Call the startAudioSpectrumMonitor interface to start audio spectrum monitoring:
```
// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.startAudioSpectrumMonitor();
```
```
// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.startAudioSpectrumMonitor();
```
After calling the above interface, onCapturedAudioSpectrumUpdate will be triggered immediately, the callback value is 0 when not publishing and not previewing; onRemoteAudioSpectrumUpdate will only callback after starting to play stream startPlayingStream .

3 Stop listening to sound level and audio spectrum callback switches

You can stop listening to corresponding callbacks for sound level or audio spectrum respectively.

Call the stopSoundLevelMonitor interface to stop sound level monitoring:
```
// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.stopSoundLevelMonitor();
```
```
// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.stopSoundLevelMonitor();
```
After calling the above interface, onCapturedSoundLevelUpdate and onRemoteSoundLevelUpdate will no longer callback.
Call the stopAudioSpectrumMonitor interface to stop audio spectrum monitoring:
```
// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.stopAudioSpectrumMonitor();
```
```
// mSDKEnging is an instance of ZegoExpressEngine
mSDKEnging.stopAudioSpectrumMonitor();
```
After calling the above interface, onCapturedAudioSpectrumUpdate and onRemoteAudioSpectrumUpdate will no longer callback.

Usage Steps for Mixing Scenario

Stream mixing is a feature that mixes multiple streams into one stream. When customers need to display sound level information of each stream before mixing, they can use the mixing sound level feature. Since the output of mixing is a single stream, using the sound level information of the mixed output stream cannot meet the need to display the sound level of each input stream. In this case, it is necessary to carry the sound level information of the input streams in the stream information during mixing, and then parse the sound level information of each input stream from the stream information when playing the mixed output stream.
When parsing the sound level information of each input stream from the stream information, we obtain the sound level values corresponding to each input stream, which is a dictionary. The key in the dictionary is the stream identifier, and value is the sound level value. However, due to the size limitation of stream information, key cannot use the stream ID and can only use a numeric ID (soundLevelID) to identify the stream.
In manual mixing configuration, developers need to maintain the association relationship between the numeric ID (soundLevelID) and stream ID. In the callback, developers will get the numeric ID (soundLevelID) and corresponding sound level information.
In room auto mixing, the mixing server and SDK automatically handle the association between numeric IDs and stream IDs. In the callback, developers get stream ID corresponding sound level information.

1 Listen to mixing sound level callback interface

Interface prototypes

Sound level update callback interface for each single stream in manual mixing onMixerSoundLevelUpdate:

/**
 * Sound level update callback for each single stream in mixing
 *
 * Callback notification period is 100 ms.
 * @param soundLevel Key-value pairs of sound levels of each single stream in the mix, key is the soundLevelID of each single stream, value is the sound level value of the corresponding single stream. Value range: value ranges from 0.0 to 100.0.
 */
public void onMixerSoundLevelUpdate(HashMap<Integer, Float> soundLevels){

}

/**
 * Sound level update callback for each single stream in mixing
 *
 * Callback notification period is 100 ms.
 * @param soundLevel Key-value pairs of sound levels of each single stream in the mix, key is the soundLevelID of each single stream, value is the sound level value of the corresponding single stream. Value range: value ranges from 0.0 to 100.0.
 */
public void onMixerSoundLevelUpdate(HashMap<Integer, Float> soundLevels){

}

Sound level update callback interface for each single stream in auto mixing onAutoMixerSoundLevelUpdate:

Warning

Only when logging into the room where auto mixing is located and playing the mix of this room can you get streamID in the callback interface.

/**
 * Sound level update callback for each single stream in auto mixing
 *
 * Callback notification period is 100 ms.
 * @param soundLevels Key-value pairs of sound levels of each single stream in the mix, key is the streamID of each single stream, value is the sound level value of the corresponding single stream, value ranges from 0.0 to 100.0
 */
public void onAutoMixerSoundLevelUpdate(HashMap<String, Float> soundLevels){

}

/**
 * Sound level update callback for each single stream in auto mixing
 *
 * Callback notification period is 100 ms.
 * @param soundLevels Key-value pairs of sound levels of each single stream in the mix, key is the streamID of each single stream, value is the sound level value of the corresponding single stream, value ranges from 0.0 to 100.0
 */
public void onAutoMixerSoundLevelUpdate(HashMap<String, Float> soundLevels){

}

2 Start listening to sound level callback switch

When starting/updating a mixing task, you can start listening to the sound level callback switch.

Manual mixing scenario

When calling the startMixerTask interface to initiate a manual mixing task, set the soundLevel parameter to True to start sound level monitoring, and specify a unique soundLevelID for each input stream:

ZegoMixerTask task = new ZegoMixerTask();
task.taskID = "task123";
// Enable mixing sound level
task.soundLevel = true;

ZegoMixerInput input = new ZegoMixerInput();
// Assign a soundLevelID to the input stream
input.soundLevelID = 123;

// Other configurations

mSDKEnging.startMixerTask(task, null);

ZegoMixerTask task = new ZegoMixerTask();
task.taskID = "task123";
// Enable mixing sound level
task.soundLevel = true;

ZegoMixerInput input = new ZegoMixerInput();
// Assign a soundLevelID to the input stream
input.soundLevelID = 123;

// Other configurations

mSDKEnging.startMixerTask(task, null);

Auto mixing scenario

When calling the startAutoMixerTask interface to initiate an auto mixing task, set the enableSoundLevel parameter to True to start sound level monitoring:

ZegoAutoMixerTask task = new ZegoAutoMixerTask();
task.taskID = "autotask123";
// Enable mixing sound level
task.enableSoundLevel = true;
// Other configurations

mSDKEnging.startAutoMixerTask(task, null);

ZegoAutoMixerTask task = new ZegoAutoMixerTask();
task.taskID = "autotask123";
// Enable mixing sound level
task.enableSoundLevel = true;
// Other configurations

mSDKEnging.startAutoMixerTask(task, null);

3 Stop listening to sound level callback switch

When updating a mixing task, you can set the switch to stop listening to sound level callbacks.