Set Video Encoding Method

Function Overview

When developers publish and play video streams, they can set detailed encoding and decoding, including enabling layered video encoding, enabling video large and small stream encoding, using hardware encoding and decoding, and setting encoding methods.

Layered Video Encoding

Layered video encoding divides the bitstream into a base layer and an enhancement layer. This encoding method can provide better experience for users with different network conditions. The base layer ensures the most basic video quality, while the enhancement layer supplements the base layer. For users with better networks, only pulling the enhancement layer can get a better experience. For users with poor network conditions, only pulling the base layer can ensure basic video quality.

When developers encounter the following situations in co-hosting or stream mixing business, it is recommended to use the layered video encoding function:

• Need to display video streams of different qualities on different terminals.
• Need to maintain smooth co-hosting in poor network environments.
• Need to adaptively pull video stream quality based on network status.

Video Large and Small Stream Encoding

Video large and small stream encoding and layered video encoding both work together to divide the bitstream into large resolution type and small resolution type.

The most significant difference is that layered video encoding uses one encoder to encode base layer and enhancement layer streams, while video large and small stream encoding uses two encoders to encode base layer and enhancement layer streams.

For specific differences, advantages, and disadvantages, please refer to Video Large and Small Stream and Layered Encoding. Developers can choose layered video encoding or video large and small stream encoding based on the differences and specific business requirements.

Hardware Encoding and Decoding

Developers can choose to enable hardware encoding and hardware decoding. After enabling hardware encoding and decoding, GPU will be used for encoding and decoding, reducing CPU usage. If some devices have serious heating problems when publishing or playing large-resolution audio/video streams, you can enable hardware encoding and decoding.

Video Encoding Method

Developers can perform video encoding configuration to align encoding between different ends, thereby achieving multi-end interoperability.

Usage scenarios:

• Generally, the default encoding can be used.
• When you need to reduce bitrate at the same resolution and frame rate, you can use H.265.
• When you need to interoperate with mini-programs, you need to use H.264.

Example Source Code Download

Please refer to Download Example Source Code to get the source code.

For related source code, please check the files in the "lib\topics\VideoAdvanced\encoding_and_decoding" directory.

Prerequisites

Before implementing video encoding and decoding functions, please ensure:

• A project has been created in the ZEGOCLOUD Console and valid AppID and AppSign have been applied. For details, please refer to Console - Project Information.
• ZEGO Express SDK has been integrated in the project and basic audio/video publishing and playing functions have been implemented. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Implementation Steps

Layered Video Encoding

Using layered video encoding requires the following two steps:

• Enable layered video encoding by specifying a specific encoder before publishing.
• Specify the layered video to pull when playing.

Enable Layered Video Encoding (H.264 SVC)

Before publishing (startPublishingStream), call the setVideoConfig interface to set the parameter "codecID" in the ZegoVideoConfig class to enable/disable layered video encoding.

• Setting "codecID" to "ZegoVideoCodecID.Svc" enables this function.
• Setting "codecID" to "ZegoVideoCodecID.Default", "ZegoVideoCodecID.Vp8", or "ZegoVideoCodecID.H265" disables this function.

var videoConfig = ZegoVideoConfig.preset(ZegoVideoConfigPreset.Preset720P);
videoConfig.codecID = ZegoVideoCodecID.Svc;
ZegoExpressEngine.instance.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
ZegoExpressEngine.instance.startPublishingStream(streamID);

var videoConfig = ZegoVideoConfig.preset(ZegoVideoConfigPreset.Preset720P);
videoConfig.codecID = ZegoVideoCodecID.Svc;
ZegoExpressEngine.instance.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
ZegoExpressEngine.instance.startPublishingStream(streamID);

Specify Layered Video to Pull

After the publishing end enables layered video encoding, the playing end can call the setPlayStreamVideoType interface before or after playing. At this time, the playing end will pull appropriate video layers according to network conditions by default, such as pulling only the base layer in weak networks. Developers can also pass in specific playing parameters to pull specific video layers. Currently supported video layers are as follows:

Taking pulling the enhancement layer as an example:

ZegoExpressEngine.instance.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.Big);
ZegoExpressEngine.instance.startPlayingStream(playStreamID);

ZegoExpressEngine.instance.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.Big);
ZegoExpressEngine.instance.startPlayingStream(playStreamID);

Video Large and Small Stream Encoding (H.264 DualStream)

The implementation of video large and small stream encoding (H.264 DualStream) is similar to layered video encoding (H.264 SVC), requiring the following two steps:

• Before publishing, enable video large and small stream encoding by specifying a specific encoder.
• When playing, specify the video bitstream to pull.

Enable Video Large and Small Stream Encoding

Before publishing (startPublishingStream), call the setVideoConfig interface to set the parameter codecID in the ZegoVideoConfig class to ZegoVideoCodecID.H264DualStream to enable the video large and small stream encoding function.

var videoConfig = ZegoVideoConfig.preset(ZegoVideoConfigPreset.Preset720P);
videoConfig.codecID = ZegoVideoCodecID.H264DualStream;
ZegoExpressEngine.instance.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
ZegoExpressEngine.instance.startPublishingStream(streamID);

var videoConfig = ZegoVideoConfig.preset(ZegoVideoConfigPreset.Preset720P);
videoConfig.codecID = ZegoVideoCodecID.H264DualStream;
ZegoExpressEngine.instance.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
ZegoExpressEngine.instance.startPublishingStream(streamID);

Specify Video Bitstream to Pull

After the publishing end enables video large and small stream encoding, the playing end can call the setPlayStreamVideoType interface before or after playing. At this time, the playing end will pull appropriate video stream layers according to network conditions by default, such as pulling only the base layer in weak networks. Developers can also pass in specific playing parameters to pull specific video layers. Currently supported video layers are as follows:

Taking pulling the enhancement layer as an example:

ZegoExpressEngine.instance.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.Big);
ZegoExpressEngine.instance.startPlayingStream(playStreamID);

ZegoExpressEngine.instance.setPlayStreamVideoType(playStreamID,ZegoVideoStreamType.Big);
ZegoExpressEngine.instance.startPlayingStream(playStreamID);

Hardware Encoding and Decoding

Since a small number of device models do not support hardware encoding/decoding well, the SDK uses software encoding and software decoding by default. If developers need to use hardware encoding, they can refer to this section to set it themselves.

Enable Hardware Encoding

If developers need to enable hardware encoding, they can call the enableHardwareEncoder interface.

// Enable hardware encoding
ZegoExpressEngine.instance.enableHardwareEncoder(true);

// Enable hardware encoding
ZegoExpressEngine.instance.enableHardwareEncoder(true);

Enable Hardware Decoding

If developers need to enable hardware decoding, they can call the enableHardwareDecoder interface.

// Enable hardware decoding
ZegoExpressEngine.instance.enableHardwareDecoder(true);

// Enable hardware decoding
ZegoExpressEngine.instance.enableHardwareDecoder(true);

Set Video Encoding Method

Before publishing (startPublishingStream), call the setVideoConfig interface to set the parameter "codecID" under the "ZegoVideoConfig" class to set the video encoding method. Currently supported video encoding methods are as follows:

Taking setting the encoding method to H.265 as an example:

var videoConfig = ZegoVideoConfig.preset(ZegoVideoConfigPreset.Preset720P);
videoConfig.codecID = ZegoVideoCodecID.H265;
ZegoExpressEngine.instance.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
ZegoExpressEngine.instance.startPublishingStream(streamID);

var videoConfig = ZegoVideoConfig.preset(ZegoVideoConfigPreset.Preset720P);
videoConfig.codecID = ZegoVideoCodecID.H265;
ZegoExpressEngine.instance.setVideoConfig(videoConfig);

String streamID = "MultiLayer-1";
ZegoExpressEngine.instance.startPublishingStream(streamID);

API Reference List

FAQ

1. When forwarding to or directly publishing to CDN, the audience pulls streams from CDN. Are layered video encoding and large and small streams effective? What are the bitrate, resolution, and frame rate of streams pulled from CDN?

   • Video Layered Encoding and Video Large and Small Stream Encoding use ZEGO's proprietary protocol. The playing end can only pull video streams of different layers when pulling RTC streams or L3 streams from the ZEGO server.

   • In the direct publish to CDN scenario, since it does not go through the ZEGO server, the layering of the bitstream is invalid, and the SDK will fall back to H.264 encoding. The resolution and bitrate of streams pulled from CDN are consistent with the resolution and bitrate set by the publishing user.

   • In the forward to CDN scenario, since CDN pulling does not use ZEGO's proprietary protocol, the stream forwarded by the ZEGO server to the CDN server does not support layered video encoding and video large and small stream encoding. You can only choose between forwarding the base layer or forwarding the enhancement layer. The resolution, bitrate, and frame rate when pulling from CDN depend on whether the base layer or the enhancement layer is forwarded.

2. What is the difference between layered video encoding and large and small stream video encoding?

   For details, please refer to Video Large and Small Stream and Layered Encoding.