Push Whiteboard to Third-Party Platforms

Feature Overview

This article mainly describes how to use ZEGO Express SDK's stream mixing feature to merge audio and video streams and ZegoSuperBoard content into a single stream and output it to third-party platforms, such as WeChat, Video Account, etc., so as to achieve better communication and marketing effects.

Whiteboard is a frequently used feature in education, training and entertainment scenarios. ZegoSuperBoard can be used in combination with Express SDK to provide complete multi-user real-time whiteboard interactive collaboration services, including: whiteboard graffiti, real-time trajectory synchronization, whiteboard and real-time audio and video synchronization, and other capabilities.

Many activities, in order to expand communication effects, hope that the speaker's image, voice and whiteboard content can be played on third-party platforms, such as WeChat, Video Account, Douyin, etc. However, this requires that both the speaker end and the player end integrate ZEGO Express and ZegoSuperBoard to implement real-time whiteboard interactive collaboration services, which is complex to implement.

Currently, ZEGO uses the stream mixing feature to synthesize the speaker's audio and video stream with whiteboard-related operations on the ZEGO server side into a single stream, and then outputs it to third-party platforms. This enables third-party platforms to receive the speaker's image, voice and whiteboard content simultaneously without integrating ZEGO Express and ZegoSuperBoard.

Example Source Code

Please refer to Download Example Source Code to get the source code.

For related source code, please check the files in the "/ZegoExpressExample/Others/src/main/java/com/example/others/streammixing" directory.

Prerequisites

Before implementing whiteboard stream mixing functionality, please ensure:

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.

• You have integrated ZEGO Express SDK in your project and implemented basic audio and video streaming functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

• You have integrated ZegoSuperBoard SDK in your project and implemented whiteboard-related features. For details, please refer to Create Super Board.

Implementation Process

1 Initialize and Login to Room

For specific process, please refer to the Initialization and Login to Room sections of the "Implementing Video Call" document to complete initialization and login to Room.

2 Set Stream Mixing Configuration

In Express SDK, define a stream mixing task configuration object through ZegoMixerTask, which contains information such as input stream layout, output stream and ZEGO Super Board configuration.

Create Stream Mixing Task Object

For specific process, please refer to Create Stream Mixing Task Object in the stream mixing document to create.

Set Stream Mixing Input Streams

For specific process, please refer to Set Stream Mixing Input Streams in the stream mixing document to set.

Set Stream Mixing Output

For specific process, please refer to Set Stream Mixing Output in the stream mixing document to set.

Set Stream Mixing Whiteboard Configuration

Call the setWhiteboard interface to merge ZEGO Super Board content into the stream mix: you need to fill in parameters such as whiteboardID whiteboard ID, which is the ID and other information returned when integrating ZegoSuperBoard SDK and creating a whiteboard in the Prerequisites of this article.

Taking whiteboard layout placed in the upper left corner of the screen as an example:

ZegoMixerWhiteboard whiteboard = new ZegoMixerWhiteboard();

// ZEGO Super Board ID, which is the ID returned when integrating ZegoSuperBoard SDK and creating a whiteboard in the prerequisites of this article
whiteboard.whiteboardID = 0;

// ZEGO Super Board view layout
whiteboard.layout.left = 0;
whiteboard.layout.top = 0;
whiteboard.layout.right = 100;
whiteboard.layout.bottom = 100;

// Whether ZEGO Super Board supports loading dynamic PPT
whiteboard.isPPTAnimation = true;

if (whiteboard != null) {
    mMixerTask.setWhiteboard(whiteboard);
}

ZegoMixerWhiteboard whiteboard = new ZegoMixerWhiteboard();

// ZEGO Super Board ID, which is the ID returned when integrating ZegoSuperBoard SDK and creating a whiteboard in the prerequisites of this article
whiteboard.whiteboardID = 0;

// ZEGO Super Board view layout
whiteboard.layout.left = 0;
whiteboard.layout.top = 0;
whiteboard.layout.right = 100;
whiteboard.layout.bottom = 100;

// Whether ZEGO Super Board supports loading dynamic PPT
whiteboard.isPPTAnimation = true;

if (whiteboard != null) {
    mMixerTask.setWhiteboard(whiteboard);
}

3 Start Stream Mixing Task

For specific process, please refer to the Start Stream Mixing Task section of the stream mixing document.

4 Update Stream Mixing Task Configuration

Express SDK supports updating the whiteboard configuration in the stream mixing task. For example, when you need to switch the ZEGO Super Board instance to be mixed into or no longer mix ZEGO Super Board, you can update the configuration by calling the startMixerTask interface. For updating other configurations of the stream mixing task, please refer to 5.4 Update Stream Mixing Task Configuration in the stream mixing document.

Taking no longer mixing ZEGO Super Board content into the stream mix as an example:

// Reset stream mixing task whiteboard configuration information
mMixerTask.setWhiteboard(null);

// Call the start stream mixing task interface again to update stream mixing configuration
engine.startMixerTask(mMixerTask, null);

// Reset stream mixing task whiteboard configuration information
mMixerTask.setWhiteboard(null);

// Call the start stream mixing task interface again to update stream mixing configuration
engine.startMixerTask(mMixerTask, null);

5 Stop Stream Mixing

For specific process, please refer to Stop Stream Mixing in the stream mixing document.

FAQ