Play Transparent Gift Effects

Overview

In scenarios such as live audio rooms and live streaming, gifts are an important part of the interaction between hosts and audience. Exquisite gift effects can enhance interactivity and improve the viewing experience. With the vigorous development of the industry, gift effects have gradually evolved from static to dynamic, and from opaque to transparent.

Playing MP4 gift videos to achieve dynamic gift effects is a relatively common method. Since MP4 does not support the Alpha channel, when playing gift effects in full screen, it will block the content of the live streaming room, thereby affecting the user experience. Therefore, ZEGO Express SDK media player provides transparent rendering capabilities, which can realize the function of playing transparent dynamic gifts.

ZEGO Express SDK media player provides the function of playing MP4 materials (MP4 materials after splicing RGB and Alpha) with separated RGB channel and Alpha channel, realizing the dynamic effect of playing transparent gifts, that is, when playing gift effects, it will not block the content of the live streaming room, greatly improving the user experience.

Functional Advantages

• Low Development Cost

  Developers do not need to learn complex animation implementation schemes. They only need to use materials processed by designers to realize playing transparent gift effects, improving development efficiency and effects.

• High Animation Fidelity

  Help designers achieve WYSIWYG output materials. That is, when designers design animation effects, they don't need to worry about whether the technology stack used by developers can restore complex 3D effects, stroke effects, particle effects, etc.

• Good Compatibility

  Avoid the problems of complex implementation, low performance, and poor compatibility with some models of semi-transparent animation implementations such as Lottie and Cocos2d-X, as well as high difficulty in later maintenance.

Implementation Principle

As shown in the figure below, the upper part of the figure shows some pixel points of a material that has undergone Alpha channel separation. The left side uses the RGB channel to store the RGB channel information of the gift material, and the right side uses the RGB channel to store the Alpha channel information of the gift material.

When playing on the client, the R channel of the video on the right will be extracted and used as the Alpha channel value of the corresponding pixel on the left.

When rendering, divide the Alpha channel value by 255 for normalization to get an Alpha value between 0 and 1, and finally render each pixel to the screen based on the RGBA value to achieve the semi-transparent effect.

Prerequisites

Before using the media player to play MP4 dynamic effects, that is, to realize playing transparent gift effects, ensure:

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
• You have integrated ZEGO Express SDK into the project and implemented basic audio and video streaming functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Implementation Flow

1 Output Materials

Taking outputting materials in AE software as an example.

1. In AE, normally make a video with transparent effects, and output RGB and Alpha channel videos respectively.

   • Output RGB channel video.
     
   • Output Alpha channel video.
     

2. In AE, splice the output RGB and Alpha channel videos into one video.

   ZEGO media player supports the following four splicing methods. The closer the spliced video is to a square, the better the performance. Developers can choose the appropriate splicing method according to the ratio of source materials:

   • Alpha channel is at the bottom.
     
   • Alpha channel is on the left.
     
   • Alpha channel is on the right.
     
   • Alpha channel is at the upper right. Alpha channel only supports 0.5x scaling rate.
     

2 Create Media Player

Call createMediaPlayer to create a media player.

ZegoExpressEngine.instance().createMediaPlayer().then((player) => {
  this.mediaPlayer = player;
});

ZegoExpressEngine.instance().createMediaPlayer().then((player) => {
  this.mediaPlayer = player;
});

3 Load Materials

After calling createMediaPlayer to create a media player, through the loadResourceWithConfig interface, through file path, play the spliced effect materials.

var RNFS = require('react-native-fs');

let resource = new ZegoMediaPlayerResource()
let path = "/static/source1_complex_rl.mp4"
resource.loadType = ZegoMultimediaLoadType.FilePath // Select file path loading for loading method
resource.filePath = RNFS.MainBundlePath + path // File path
resource.alphaLayout = ZegoAlphaLayoutType.Left // Select Alpha channel layout according to actual resources
this.mediaPlayer.loadResourceWithConfig(resource).then((ret) => {
    console.log("load resource error: " + ret.errorCode)
});

var RNFS = require('react-native-fs');

let resource = new ZegoMediaPlayerResource()
let path = "/static/source1_complex_rl.mp4"
resource.loadType = ZegoMultimediaLoadType.FilePath // Select file path loading for loading method
resource.filePath = RNFS.MainBundlePath + path // File path
resource.alphaLayout = ZegoAlphaLayoutType.Left // Select Alpha channel layout according to actual resources
this.mediaPlayer.loadResourceWithConfig(resource).then((ret) => {
    console.log("load resource error: " + ret.errorCode)
});

4 Play Materials

After successfully calling loadResourceWithConfig to load the file, you can call start, pause, resume, stop to start and stop playing. Once the player's internal state changes, the mediaPlayerStateUpdate callback of ZegoMediaPlayer will be triggered.

Users can also call getCurrentState at any time to get the current state of the player.

If enableRepeat is set to YES, the player will automatically replay after finishing playing the file.

// Start playing. Before playing, you need to call the interface to load the media file first
this.mediaPlayer.start();
// Pause
this.mediaPlayer.pause();
// Resume
this.mediaPlayer.resume();
// Stop
this.mediaPlayer.stop();

// Start playing. Before playing, you need to call the interface to load the media file first
this.mediaPlayer.start();
// Pause
this.mediaPlayer.pause();
// Resume
this.mediaPlayer.resume();
// Stop
this.mediaPlayer.stop();

5 Player Transparent View Control

When playing video resources, set the alphaBlend parameter in ZegoView to true to enable the player transparent effect, and then call the setPlayerView interface to set the video display view.

let mediaPlayer = await ZegoExpressEngine.instance().createMediaPlayer();
mediaPlayer.setPlayerView({"reactTag": findNodeHandle(this.refs.zego_media_view), "viewMode": 0, "backgroundColor": 0, alphaBlend: true});

let mediaPlayer = await ZegoExpressEngine.instance().createMediaPlayer();
mediaPlayer.setPlayerView({"reactTag": findNodeHandle(this.refs.zego_media_view), "viewMode": 0, "backgroundColor": 0, alphaBlend: true});

6 Destroy Media Player

After using the media player, you need to call the destroyMediaPlayer interface in time to destroy the player instance variable and release the occupied resources.

// Destroy media player
ZegoExpressEngine.instance().destroyMediaPlayer(this.mediaPlayer)

// Destroy media player
ZegoExpressEngine.instance().destroyMediaPlayer(this.mediaPlayer)