Push Whiteboard to Third-party Platforms

Feature Introduction

This article mainly describes how to use the stream mixing feature of ZEGO Express SDK to merge audio and video streams with ZegoSuperBoard content into one stream and output it to third-party platforms, such as WeChat, Video Account, etc., thereby achieving better dissemination and marketing effects.

Whiteboard is a high-frequency feature used in education, training, and entertainment scenarios. ZegoSuperBoard can be used in combination with Express SDK to provide complete multi-person real-time whiteboard interactive collaboration services, including: whiteboard doodling, real-time trajectory synchronization, whiteboard and real-time audio and video synchronization, and other capabilities.

Many activities hope to broadcast the speaker's picture, voice, and whiteboard content on third-party platforms, such as WeChat, Video Account, Douyin, etc., to expand the dissemination effect. However, this requires both the speaker end and the playing end to integrate ZEGO Express and ZegoSuperBoard to achieve real-time whiteboard interactive collaboration services, which is complex to implement.

Currently, ZEGO uses the stream mixing feature to synthesize the speaker's audio and video streams with whiteboard-related operations into one stream at the ZEGO server side, and then outputs it to third-party platforms. This enables third-party platforms to receive the speaker's picture, voice, and whiteboard content simultaneously without integrating ZEGO Express and ZegoSuperBoard.

Sample Source Code

Please refer to Download Sample Source Code to get the source code.

For related source code, please check the files in the "/ZegoExpressExample/Examples/Others/Mixer" directory.

Prerequisites

Before implementing the whiteboard stream mixing feature, please ensure:

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.

• You have integrated ZEGO Express SDK in the project and implemented basic audio and video publishing and playing functions. For details, please refer to Quick Start - Integration and Quick Start - Implementation Flow.

• You have integrated ZegoSuperBoard SDK in the project and implemented whiteboard-related functions. For details, please refer to Create Super Board.

Implementation Flow

1 Initialize and login room

For the specific process, please refer to the Initialization and Login Room sections of the "Implementing Video Call" documentation to complete initialization and login room.

2 Set stream mixing configuration

In Express SDK, the stream mixing task configuration object is defined through ZegoMixerTask, which includes information such as input stream layout, output stream, and ZEGO Super Board configuration.

/// Stream mixing task object
@interface ZegoMixerTask : NSObject

- (instancetype)init NS_UNAVAILABLE;

/// Construct a stream mixing task object through TaskID
- (instancetype)initWithTaskID:(NSString *)taskID;

/// Stream mixing task ID
@property (nonatomic, copy, readonly) NSString *taskID;

/// Set the input stream list of the stream mixing task object
- (void)setInputList:(NSArray<ZegoMixerInput *> *)inputList;

/// Set the output list of the stream mixing task object
- (void)setOutputList:(NSArray<ZegoMixerOutput *> *)outputList;

/// Set the whiteboard configuration of the stream mixing task object
- (void)setWhiteboard:(ZegoMixerWhiteboard *)whiteboard;

@end

/// Stream mixing task object
@interface ZegoMixerTask : NSObject

- (instancetype)init NS_UNAVAILABLE;

/// Construct a stream mixing task object through TaskID
- (instancetype)initWithTaskID:(NSString *)taskID;

/// Stream mixing task ID
@property (nonatomic, copy, readonly) NSString *taskID;

/// Set the input stream list of the stream mixing task object
- (void)setInputList:(NSArray<ZegoMixerInput *> *)inputList;

/// Set the output list of the stream mixing task object
- (void)setOutputList:(NSArray<ZegoMixerOutput *> *)outputList;

/// Set the whiteboard configuration of the stream mixing task object
- (void)setWhiteboard:(ZegoMixerWhiteboard *)whiteboard;

@end

Create stream mixing task object

For the specific process, please refer to Create Stream Mixing Task Object in the stream mixing documentation.

Set stream mixing input stream

For the specific process, please refer to Set Stream Mixing Input Stream in the stream mixing documentation.

Set stream mixing output

For the specific process, please refer to Set Stream Mixing Output in the stream mixing documentation.

Set stream mixing whiteboard configuration

Call the setWhiteboard interface to merge ZEGO Super Board content into the stream mixing: You need to fill in parameters such as whiteboardID whiteboard ID, which is the ID and other information returned when integrating ZegoSuperBoard SDK and creating a whiteboard in the Prerequisites of this article.

Taking the whiteboard layout placed in the upper left corner of the picture as an example:

// ZEGO Super Board ID, which is the ID returned when integrating ZegoSuperBoard SDK and creating a whiteboard in the prerequisites of this article
long whiteboardID = 0;
ZegoMixerWhiteboard *whiteboard= [[ZegoMixerWhiteboard alloc] initWithWhiteboardID:whiteboardID layout:CGRectMake(0, 0, 100, 100)];

// Whether ZEGO Super Board supports loading dynamic PPT
[whiteboard setIsPPTAnimation:YES];

[task setWhiteboard:whiteboard];

// ZEGO Super Board ID, which is the ID returned when integrating ZegoSuperBoard SDK and creating a whiteboard in the prerequisites of this article
long whiteboardID = 0;
ZegoMixerWhiteboard *whiteboard= [[ZegoMixerWhiteboard alloc] initWithWhiteboardID:whiteboardID layout:CGRectMake(0, 0, 100, 100)];

// Whether ZEGO Super Board supports loading dynamic PPT
[whiteboard setIsPPTAnimation:YES];

[task setWhiteboard:whiteboard];

3 Start stream mixing task

For the specific process, please refer to the Start Stream Mixing Task section in the stream mixing documentation.

4 Update stream mixing task configuration

Express SDK supports updating the whiteboard configuration in the stream mixing task. For example, when you need to switch the ZEGO Super Board instance to be mixed into the stream or no longer mix ZEGO Super Board into the stream, you can update the configuration by calling the startMixerTask interface. To update other configurations of the stream mixing task, please refer to Update Stream Mixing Task Configuration in the stream mixing documentation.

Taking no longer mixing ZEGO Super Board content into the stream mixing as an example:

// Reset stream mixing task whiteboard configuration information
[self.mixerTask setWhiteboard:nil];

// Call the start stream mixing task interface again to update the stream mixing configuration
[self.engine startMixerTask:self.mixerTask callback:^(ZegoMixerStartResult * _Nonnull result) {
    if (result.errorCode == 0) {
        NSLog(@"Start mixer task success");
    } else {
        NSLog(@"Start mixer task fail");
    }
}];

// Reset stream mixing task whiteboard configuration information
[self.mixerTask setWhiteboard:nil];

// Call the start stream mixing task interface again to update the stream mixing configuration
[self.engine startMixerTask:self.mixerTask callback:^(ZegoMixerStartResult * _Nonnull result) {
    if (result.errorCode == 0) {
        NSLog(@"Start mixer task success");
    } else {
        NSLog(@"Start mixer task fail");
    }
}];

5 Stop stream mixing

For the specific process, please refer to Stop Stream Mixing in the stream mixing documentation.

FAQ