Multi-Source Capture

Feature overview

Multi-source capture is used to manage audio and video source configurations for each channel. Through this capability, you can flexibly and quickly implement real-time interaction of different audio and video content, such as screen sharing, audio mixing, and other functions.

The main capabilities and limitations of multi-source capture are as follows:

• Publishing stream channels support setting multiple audio and video sources, but the same audio and video source can only be occupied by one channel.
• The main channel's audio and video sources support dynamic switching and also support audio mixing capabilities. The auxiliary channel only supports specifying capture sources before publishing stream and does not support mixing.
• When capturing screen, only iOS and Android platforms support capturing video and audio at the same time; other platforms only support capturing video. If you need to capture audio, please implement relevant logic yourself.

Example source code

For relevant source code, please refer to the official website example source code. For details, please refer to Run Example Code to obtain the source code.

Prerequisites

Before using the multi-source capture feature, please ensure:

• You have created a project in the ZEGOCLOUD Console and applied for valid AppID and AppSign. For details, please refer to Console - Project Information.
• You have integrated ZEGO Express SDK in the project and implemented basic audio and video publishing and playing functions. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Implementation process

1 Create ZegoExpressEngine engine

For the process of creating ZegoExpressEngine engine, please refer to Create Engine for implementing video calls.

// Import ZegoExpressEngine
const zgEngine = window.require('zego-express-engine-electron/ZegoExpressEngine');
const zgDefines = window.require('zego-express-engine-electron/ZegoExpressDefines');

// Specify using live streaming scenario (please fill in the scenario suitable for your business according to the actual situation)
const profile = {
appID : xxx,
appSign : "xxx",
scenario : zgDefines.ZegoScenario.Default
};

zgEngine.createEngine(profile)
.then(() => {
    console.log("init succeed")
}).catch((e) => {
    console.log("init failed", e)
});

// Import ZegoExpressEngine
const zgEngine = window.require('zego-express-engine-electron/ZegoExpressEngine');
const zgDefines = window.require('zego-express-engine-electron/ZegoExpressDefines');

// Specify using live streaming scenario (please fill in the scenario suitable for your business according to the actual situation)
const profile = {
appID : xxx,
appSign : "xxx",
scenario : zgDefines.ZegoScenario.Default
};

zgEngine.createEngine(profile)
.then(() => {
    console.log("init succeed")
}).catch((e) => {
    console.log("init failed", e)
});

2 Set audio and video capture sources

1. Call the setVideoSource interface to set the video capture source.

/** Use camera as video capture source */
zgEngine.setVideoSource(zgDefines.ZegoVideoSourceType.Camera, 0, zgDefines.ZegoPublishChannel.Main)

/** Use camera as video capture source */
zgEngine.setVideoSource(zgDefines.ZegoVideoSourceType.Camera, 0, zgDefines.ZegoPublishChannel.Main)

2. Call the setAudioSource interface to set the audio capture source.

   • Set audio capture source only

     /** Use microphone as audio capture source */
     zgEngine.setAudioSource(zgDefines.ZegoAudioSourceType.Microphone, zgDefines.ZegoPublishChannel.Main)

     /** Use microphone as audio capture source */
     zgEngine.setAudioSource(zgDefines.ZegoAudioSourceType.Microphone, zgDefines.ZegoPublishChannel.Main)

   • Set audio capture source and set mixing configuration at the same time

     let audioSourceMixConfig;
     /** Mix sound played by system sound card*/
     audioSourceMixConfig.enableMixSystemPlayout = true;
     /** Mix sound played by SDK*/
     audioSourceMixConfig.enableMixEnginePlayout = true;
     
     /** Mix sound played by media player*/
     let mediaplayerList = []
     mp = zgEngine.createMediaPlayer();
     mediaplayerList.push(mp.getIndex())
     audioSourceMixConfig.mediaPlayerIndexList = mediaplayerList;
     
     /** Mix sound played by audio effect player*/
     let audioplayerList = []
     audioEffectPlayer = zgEngine.createAudioEffectPlayer();
     audioplayerList.push(audioEffectPlayer.getIndex())
     audioSourceMixConfig.audioEffectPlayerIndexList = audioplayerList ;
     
     /** Use microphone as audio capture source and set mixing configuration */
     zgEngine.setAudioSourceWithConfig(zgDefines.ZegoAudioSourceType.Microphone, audioSourceMixConfig);

     let audioSourceMixConfig;
     /** Mix sound played by system sound card*/
     audioSourceMixConfig.enableMixSystemPlayout = true;
     /** Mix sound played by SDK*/
     audioSourceMixConfig.enableMixEnginePlayout = true;
     
     /** Mix sound played by media player*/
     let mediaplayerList = []
     mp = zgEngine.createMediaPlayer();
     mediaplayerList.push(mp.getIndex())
     audioSourceMixConfig.mediaPlayerIndexList = mediaplayerList;
     
     /** Mix sound played by audio effect player*/
     let audioplayerList = []
     audioEffectPlayer = zgEngine.createAudioEffectPlayer();
     audioplayerList.push(audioEffectPlayer.getIndex())
     audioSourceMixConfig.audioEffectPlayerIndexList = audioplayerList ;
     
     /** Use microphone as audio capture source and set mixing configuration */
     zgEngine.setAudioSourceWithConfig(zgDefines.ZegoAudioSourceType.Microphone, audioSourceMixConfig);

3 Login to room and publish stream

For the process of logging in to the room and publishing stream, please refer to Login to Room and Publish Stream for implementing video calls.

4 (Optional) Switch audio and video capture sources

1. Call the setVideoSource interface to switch the video capture source.

/** Use screen sharing as video capture source */
zgEngine.setVideoSource(zgDefines.ZegoVideoSourceType.ScreenCapture, 0, zgDefines.ZegoPublishChannel.Main)

/** Use screen sharing as video capture source */
zgEngine.setVideoSource(zgDefines.ZegoVideoSourceType.ScreenCapture, 0, zgDefines.ZegoPublishChannel.Main)

2. Call the setAudioSource interface to switch the audio capture source.

/** Use media player as audio capture source */
zgEngine.setAudioSource(zgDefines.ZegoAudioSourceType.MediaPlayer, zgDefines.ZegoPublishChannel.Main)

/** Use media player as audio capture source */
zgEngine.setAudioSource(zgDefines.ZegoAudioSourceType.MediaPlayer, zgDefines.ZegoPublishChannel.Main)

5 End publishing stream

For the process of ending publishing stream, please refer to Stop Publishing and Playing Stream for implementing video calls.

/** Stop publishing stream*/
zgEngine.stopPublishingStream(zgDefines.ZegoPublishChannel.Main);

/** Stop publishing stream*/
zgEngine.stopPublishingStream(zgDefines.ZegoPublishChannel.Main);

FAQ

1. How to capture screen video and system sound?

For capturing screen video and system sound, please refer to Screen Sharing.

2. Does it support setting multiple media players?

Yes, but note that the same player instance can only be occupied by one channel.

3. What are the limitations when combining video sources and audio sources?

   The limitations for combining video sources and audio sources are as follows:

   1. The main channel cannot use media player's video or audio source.
   2. When the auxiliary channel uses a media player as the audio and video source, the main channel needs to use and start the microphone device (enableMicrophone). If you do not need microphone sound on the main channel, you can disable audio publishing through mutePublishStreamAudio to keep the microphone started.
   3. Only when the main channel uses camera video source or microphone audio source, the auxiliary channel copying the main channel video source or audio source can take effect.
   4. The main channel's audio source cannot be set to None, otherwise audio cannot be rendered.