Multi-Source Capture
2024-09-20
Feature Overview
Multi-source capture is used to manage audio/video source configurations for each channel. Through this capability, you can flexibly and quickly implement real-time interaction of different audio/video content, such as screen sharing, audio mixing, etc.
The main capabilities and limitations of multi-source capture are as follows:
- Publishing channels support setting multiple audio/video sources, but the same audio/video source can only be occupied by one channel.
- The main channel's audio/video sources support dynamic switching and also support audio mixing capabilities. Auxiliary channels only support specifying capture sources before publishing and do not support mixing.
- When capturing screen, only iOS and Android platforms support capturing video and audio simultaneously; other platforms only support capturing video. If audio capture is needed, developers need to implement related logic themselves.
Example Source Code
Please refer to Download Example Source Code to get the source code.
For related source code, please check files in the "/ZegoExpressExample/Examples/Others/MultiVideoSource" directory.
Prerequisites
Before using multi-source capture functionality, please ensure:
- You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
- You have integrated ZEGO Express SDK in the project and implemented basic audio/video publishing and playing functions. For details, please refer to Quick Start - Integration and Quick Start - Implementation Flow.
Implementation Flow
Warning
Real-time audio products only support setting audio capture sources.
1 Create ZegoExpressEngine Engine
For the process of creating ZegoExpressEngine engine, please refer to Create Engine in implementing video call.
ZegoEngineProfile profile;
// AppID and AppSign are assigned to each App by ZEGO; for security reasons, it is recommended to store AppSign in the App's business backend and obtain it from the backend when needed
profile.appID = appID;
profile.appSign = appSign;
// Specify using live streaming scenario (please fill in the scenario suitable for your business according to the actual situation)
profile.scenario = ZegoScenario::ZEGO_SCENARIO_BROADCAST;
// Create engine instance
auto engine = ZegoExpressSDK::createEngine(profile, nullptr);2 Set Audio/Video Capture Source
- Call the setVideoSource interface to set the video capture source.
/** Use camera as video capture source */
engine->setVideoSource(ZEGO_VIDEO_SOURCE_TYPE_CAMERA);Note
- If you need to use custom video capture as the video capture source, you need to set the video capture source type to ZegoVideoSourceTypeCustom, call the enableCustomVideoCapture interface to set the video frame data type to be sent, and call the sendCustomVideoCaptureRawData or sendCustomVideoCaptureEncodedData interface to send video frame data to the SDK. For details, please refer to Advanced Features - Custom Video Capture.
- If you need to use media player as the video capture source, you need to call the createMediaPlayer interface to create a media player first, then call the setVideoSource interface to set the video capture source type to ZegoVideoSourceTypePlayer and set the media player instance index used by the video capture source, and then use the newly created media player to load resources and play. For details, please refer to Media Player.
- Call the setAudioSource interface to set the audio capture source.
/** Use microphone as audio capture source */
engine->setAudioSource(ZEGO_AUDIO_SOURCE_TYPE_MICROPHONE);Note
- If you need to use custom audio capture as the audio capture source, you need to set the audio capture source type to ZegoAudioSourceTypeCustom, and call interfaces such as sendCustomAudioCaptureAACData or sendCustomAudioCapturePCMData to send audio data to the SDK. For details, please refer to Advanced Features - Custom Audio Capture and Rendering.
- If you need to use media player as the audio capture source, you need to set the audio capture source type to ZegoAudioSourceTypeMediaPlayer, call the createMediaPlayer interface to create a media player, and then use the newly created media player to load resources and play. For details, please refer to Media Player.
3 Login Room and Publish Stream
For the process of logging in to room and publishing stream, please refer to Login Room and Publish Stream in implementing video call.
4 (Optional) Switch Audio/Video Capture Source
- Call the setVideoSource interface to switch video capture source.
/** Use screen sharing as video capture source */
engine->setVideoSource(ZEGO_VIDEO_SOURCE_TYPE_SCREEN_CAPTURE);- Call the setAudioSource interface to switch audio capture source.
/** Use media player as audio capture source */
engine->setAudioSource(ZEGO_AUDIO_SOURCE_TYPE_MEDIA_PLAYER);5 Stop Publishing Stream
For the process of stopping publishing stream, please refer to Stop Publishing/Playing Stream in implementing video call.
// Stop publishing stream
engine->stopPublishingStream();FAQ
Does it support setting multiple media players?
Yes, but note that the same player instance can only be occupied by one channel.
What are the limitations when combining video sources and audio sources?
The limitations for combining video sources and audio sources are as follows:
- The main channel cannot use media player's video or audio source.
- When an auxiliary channel uses media player as audio/video source, the main channel needs to use and start the microphone device (
enableMicrophone). If you don't need microphone sound on the main channel, you can disable audio publishing throughmutePublishStreamAudioto keep the microphone started. - Only when the main channel uses camera video source or microphone audio source, can the auxiliary channel copying main channel video source or audio source take effect.
- The main channel's audio source cannot be set to
None, otherwise audio cannot be rendered.