Multi-Source Capture

2024-09-20

Overview

Multi-source capture is used to manage audio and video source configurations for each channel. Through this capability, you can flexibly and quickly implement real-time interaction of different audio and video content, such as screen sharing, audio mixing, and other functions.

The main capabilities and limitations of multi-source capture are as follows:

The publishing channel supports setting multiple audio and video sources, but the same audio and video source can only be occupied by one channel.
The main channel's audio and video sources support dynamic switching and also support audio mixing capabilities. The auxiliary channel only supports specifying capture sources before publishing and does not support audio mixing.
When capturing the screen, only iOS and Android platforms support simultaneous capture of video and audio; other platforms only support capturing video. If you need to capture audio, developers need to implement the relevant logic themselves.

Prerequisites

Before using the multi-source capture feature, ensure:

You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
You have integrated ZEGO Express SDK into the project and implemented basic audio and video streaming functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Implementation Flow

Warning

The Voice Call product only supports setting audio capture sources.

1 Set Audio and Video Capture Sources

Call the setVideoSource interface to set the video capture source.

/** Use camera as video capture source */
ZegoExpressEngine.instance().setVideoSource(ZegoVideoSourceType.Camera);

/** Use camera as video capture source */
ZegoExpressEngine.instance().setVideoSource(ZegoVideoSourceType.Camera);

Note

If you need to use a media player as the video capture source, you need to call the createMediaPlayer interface to create a media player first, then call the setVideoSource interface to set the video capture source type to ZegoVideoSourceType.Player and set the media player instance index used by the video capture source, and then use the newly created media player to load resources and play. For details, please refer to Media Player.
If you need to use screen sharing as the video capture source, you need to set the video capture source type to ZegoVideoSourceType.ScreenCapture and enable screen sharing to push screen sharing video data to the SDK. For details, please refer to Screen Sharing.

Call the setAudioSource interface to set the audio capture source.

/** Use microphone as audio capture source */
ZegoExpressEngine.instance().setAudioSource(ZegoAudioSourceType.Microphone);

/** Use microphone as audio capture source */
ZegoExpressEngine.instance().setAudioSource(ZegoAudioSourceType.Microphone);

Note

If you need to use a media player as the audio capture source, you need to set the audio capture source type to ZegoAudioSourceType.MediaPlayer, call the createMediaPlayer interface to create a media player, and then use the newly created media player to load resources and play. For details, please refer to Media Player.
If you need to use system playback sound as the audio capture source, you need to set the video capture source type to ZegoVideoSourceType.ScreenCapture and enable screen sharing to push screen sharing audio data to the SDK. For details, please refer to Screen Sharing.

For the room login and stream publishing process, please refer to Login Room and Publish Stream for implementing video calls.

3 (Optional) Switch Audio and Video Capture Sources

Call the setVideoSource interface to switch the video capture source.

/** Use screen sharing as video capture source */
ZegoExpressEngine.instance().setVideoSource(ZegoVideoSourceType.ScreenCapture);

/** Use screen sharing as video capture source */
ZegoExpressEngine.instance().setVideoSource(ZegoVideoSourceType.ScreenCapture);

Call the setAudioSource interface to switch the audio capture source.

/** Use media player as audio capture source */
ZegoExpressEngine.instance().setAudioSource(ZegoAudioSourceType.MediaPlayer);

/** Use media player as audio capture source */
ZegoExpressEngine.instance().setAudioSource(ZegoAudioSourceType.MediaPlayer);

4 Stop Publishing Stream

For the stream publishing stop process, please refer to Stop Publishing/Playing Stream for implementing video calls.

// Stop publishing stream
ZegoExpressEngine.instance().stopPublishingStream();

// Stop publishing stream
ZegoExpressEngine.instance().stopPublishingStream();

FAQ

How to capture screen video and system sound?

If you need to capture screen video and system sound, please refer to Screen Sharing.

Does it support setting multiple media players?

Yes, but note that the same player instance can only be occupied by one channel.

What are the limitations when combining video sources and audio sources?

The limitations for combining video sources and audio sources are as follows:

The main channel cannot use the media player's video or audio source.
When the auxiliary channel uses a media player as the audio and video source, the main channel needs to use and enable the microphone device (enableMicrophone). If you don't need microphone sound on the main channel, you can disable audio publishing through mutePublishStreamAudio to keep the microphone enabled.
Only when the main channel uses the camera video source or microphone audio source, the auxiliary channel copying the main channel's video source or audio source can take effect.
The main channel's audio source cannot be set to None, otherwise the audio cannot be rendered.