Set Video Encoding Method

Feature Overview

When developers publish/play video streams, they can set encoding and decoding in detail, including enabling layered video encoding, enabling video large and small stream encoding, using hardware encoding and decoding, and setting encoding methods.

Layered Video Encoding

Layered video encoding divides the bitrate into a base layer and an enhancement layer. This encoding method can provide better experience for users with different network conditions. The base layer guarantees the most basic video quality, while the enhancement layer supplements the base layer. For users with better networks, they can only play the enhancement layer to get a better experience. For users with poor network conditions, playing only the base layer can guarantee basic video quality.

When the following situations occur in developers' real-time connection or stream mixing business, it is recommended to use the layered video encoding feature:

• Need different terminals to display video streams of different qualities.
• Need to maintain the smoothness of real-time connection in poor network environments.
• Need to adaptively play the quality of video streams based on network status.

Video Large and Small Stream Encoding

Video large and small stream encoding and layered video encoding both work to divide the bitrate into large resolution type and small resolution type.

The most significant difference is that layered video encoding uses one encoder to encode base layer and enhancement layer bitstreams, while video large and small stream encoding uses two encoders to encode base layer and enhancement layer bitstreams.

For specific differences, advantages and disadvantages, please refer to Video Large and Small Stream and Layered Encoding. Developers can choose layered video encoding or video large and small stream encoding by combining their differences and specific business requirements.

Hardware Encoding and Decoding

Developers can choose to enable hardware encoding and hardware decoding. After enabling hardware encoding and decoding, GPU will be used for encoding and decoding, reducing CPU usage. If some devices have severe heating when publishing/playing large-resolution audio and video streams, you can enable hardware encoding and decoding.

Video Encoding Method

Developers can perform video encoding configuration to align encoding between different ends, thereby achieving multi-end interoperability.

Usage scenarios:

• In general cases, use default encoding.
• When you need to reduce bitrate under the same resolution and frame rate, you can use H.265.
• When you need to interoperate with mini programs, you need to use H.264.

Example Source Code Download

Please refer to Download Example Source Code to get the source code.

For related source code, please check the files in the "/ZegoExpressExample/AdvancedVideoProcessing/src/main/java/im/zego/advancedvideoprocessing/encodinganddecoding" directory.

Prerequisites

Before implementing video encoding and decoding functionality, please ensure:

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
• You have integrated ZEGO Express SDK in your project and implemented basic audio and video streaming functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Implementation Steps

Layered Video Encoding

Using layered video encoding requires the following two steps:

• Enable layered video encoding by specifying a specific encoder before publishing stream.
• Specify the layered video to be played when playing stream.

Enable layered video encoding (H.264 SVC)

Before publishing stream (startPublishingStream), call the setVideoConfig interface to set the parameter "codecID" in the ZegoVideoConfig class to enable/disable layered video encoding functionality.

• Setting "codecID" to "ZegoVideoCodecID.SVC" can enable this feature.
• Setting "codecID" to "ZegoVideoCodecID.DEFAULT", "ZegoVideoCodecID.VP8", or "ZegoVideoCodecID.H265" can disable this feature.

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
videoConfig.codecID = ZegoVideoCodecID.SVC;
engine.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
engine.startPublishingStream(streamID);

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
videoConfig.codecID = ZegoVideoCodecID.SVC;
engine.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
engine.startPublishingStream(streamID);

Specify the layered video to be played

After the publish stream end enables layered video encoding, the play stream end can call the setPlayStreamVideoType interface before or after playing stream. At this time, the play stream end will by default pull appropriate video layers according to network conditions, for example, only pull the base layer in weak networks. Developers can also pass in specific play parameters to pull specific video layers. Currently supported video layers are as follows:

Taking pulling the enhancement layer as an example:

engine.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.BIG);
engine.startPlayingStream(playStreamID);

engine.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.BIG);
engine.startPlayingStream(playStreamID);

Video large and small stream encoding (H.264 DualStream)

The implementation of video large and small stream encoding (H.264 DualStream) is similar to layered video encoding (H.264 SVC), requiring the following two steps:

• Before publishing stream, enable video large and small stream encoding by specifying a specific encoder.
• When playing stream, specify the video bitrate to be played.

Enable video large and small stream encoding

Before publishing stream (startPublishingStream), call the setVideoConfig interface to set the parameter codecID in the ZegoVideoConfig class to ZegoVideoCodecID.H264DualStream to enable video large and small stream encoding functionality.

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
videoConfig.codecID = ZegoVideoCodecID.H264DualStream;
engine.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
engine.startPublishingStream(streamID);

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
videoConfig.codecID = ZegoVideoCodecID.H264DualStream;
engine.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
engine.startPublishingStream(streamID);

Specify the video bitrate to be played

After the publish stream end enables video large and small stream encoding, the play stream end can call the setPlayStreamVideoType interface before or after playing stream. At this time, the play stream end will by default pull appropriate video stream layers according to network conditions, for example, only pull the base layer in weak networks. Developers can also pass in specific play parameters to pull specific video layers. Currently supported video layers are as follows:

Taking pulling the enhancement layer as an example:

engine.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.BIG);
engine.startPlayingStream(playStreamID);

engine.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.BIG);
engine.startPlayingStream(playStreamID);

Hardware Encoding and Decoding

Since a small number of device models do not support hardware encoding/decoding well, the SDK uses software encoding and software decoding by default. If developers need to use hardware encoding, they can set it themselves by referring to this section.

Enable hardware encoding

If developers need to enable hardware encoding, they can call the enableHardwareEncoder interface.

// Enable hardware encoding
engine.enableHardwareEncoder(true);

// Enable hardware encoding
engine.enableHardwareEncoder(true);

Enable hardware decoding

If developers need to enable hardware decoding, they can call the enableHardwareDecoder interface.

// Enable hardware decoding
engine.enableHardwareDecoder(true);

// Enable hardware decoding
engine.enableHardwareDecoder(true);

Set video encoding method

Before publishing stream (startPublishingStream), call the setVideoConfig interface to set the parameter "codecID" under the "ZegoVideoConfig" class to set the video encoding method. Currently supported video encoding methods are as follows:

Taking setting the encoding method to H.265 as an example:

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
videoConfig.codecID = ZegoVideoCodecID.H265;
engine.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
engine.startPublishingStream(streamID);

ZegoVideoConfig videoConfig = new ZegoVideoConfig();
videoConfig.codecID = ZegoVideoCodecID.H265;
engine.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
engine.startPublishingStream(streamID);

FAQ

1. When relaying or directly publishing to CDN, audiences play streams from CDN. Are layered video encoding and large and small streams effective? What are the bitrate and resolution of streams pulled from CDN?

   • Video layered encoding and video large and small stream encoding use ZEGO's proprietary protocol. The play stream end can only pull video streams of different layers when pulling RTC streams or L3 streams from ZEGO servers.

   • In the direct publish to CDN scenario, since it does not go through ZEGO servers, the stream layering is invalid, and the SDK will fall back to H.264 encoding. The resolution and bitrate of streams pulled from CDN are consistent with the resolution and bitrate set by the publishing user.

   • In the relay to CDN scenario, since CDN playing stream does not use ZEGO's proprietary protocol, the stream relayed by ZEGO servers to CDN servers does not support layered video encoding and video large and small stream encoding. You can only choose one from relay base layer or relay enhancement layer. The resolution, bitrate and frame rate when CDN plays stream depend on whether the relay is base layer or enhancement layer.

     Warning

     When relaying to CDN, the default is to relay the enhancement layer. If the business needs to relay the base layer to CDN, please contact ZEGO Technical Support for configuration.

2. What is the difference between layered video encoding and large and small stream video encoding?

   For details, please refer to Video Large and Small Stream and Layered Encoding.

Related Documentation

How to solve video abnormalities (such as black screen, green screen, pixelated screen, etc.) when interoperating between Web platform and Native platform?