Set Video Encoding Mode

Feature Introduction

When developers publish and play video streams, they can configure the encoding and decoding in detail, including enabling layered video encoding, enabling video large and small stream encoding, using hardware encoding and decoding, and setting the encoding mode.

Layered Video Encoding

Layered video encoding divides the stream into a base layer and an enhancement layer. This encoding method can provide a better experience for users with different network conditions. The base layer ensures the most basic video quality, while the enhancement layer supplements the base layer. For users with better networks, they can play only the enhancement layer to get a better experience. For users with poor network conditions, playing only the base layer can ensure basic video quality.

When developers encounter the following situations in co-hosting or stream mixing business, it is recommended to use the layered video encoding feature:

• Need to display video streams of different qualities on different terminals.
• Need to maintain smooth co-hosting in poor network environments.
• Need to adaptively play stream video quality based on network status.

Video Large and Small Stream Encoding

Video large and small stream encoding and layered video encoding both work to divide the stream into large resolution type and small resolution type.

The most significant difference is that layered video encoding uses one encoder to encode the base layer and enhancement layer streams, while video large and small stream encoding uses two encoders to encode the base layer and enhancement layer streams.

For specific differences, advantages and disadvantages, please refer to Video Large and Small Stream and Layered Encoding. Developers can choose between layered video encoding or video large and small stream encoding based on the differences and specific business requirements.

Hardware Encoding and Decoding

Developers can choose to enable hardware encoding and hardware decoding. After enabling hardware encoding and decoding, GPU will be used for encoding and decoding, reducing CPU usage. If certain models have severe device heating when publishing or playing large-resolution audio and video streams, you can enable hardware encoding and decoding.

Video Encoding Mode

Developers can configure video encoding to align encoding between different ends, thereby achieving multi-terminal interoperability.

Usage scenarios:

• Under normal circumstances, use the default encoding.
• To reduce bitrate at the same resolution and frame rate, H.265 can be used.
• When interoperability with mini-programs is required, H.264 must be used.

Sample Source Code Download

Please refer to Download Sample Source Code to get the source code.

For related source code, please check the files in the "/ZegoExpressExample/Examples/AdvancedVideoProcessing/Encoding&Decoding" directory.

Prerequisites

Before implementing video encoding and decoding features, please ensure:

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
• You have integrated the ZEGO Express SDK in the project and implemented basic audio and video publishing and playing functions. For details, please refer to Quick Start - Integration and Quick Start - Implementation Flow.

Implementation Steps

Layered Video Encoding

Using layered video encoding requires the following two steps:

• Enable layered video encoding by specifying a specific encoder before publishing stream.
• Specify the layered video to play when playing stream.

Enable layered video encoding (H.264 SVC)

Before calling startPublishingStream, call the setVideoConfig interface to set the parameter "codecID" in the ZegoVideoConfig class to enable/disable layered video encoding.

• Setting "codecID" to "ZegoVideoCodecIDSVC" can enable this feature.
• Setting "codecID" to "ZegoVideoCodecIDDefault", "ZegoVideoCodecIDVP8", or "ZegoVideoCodecIDH265" can disable this feature.

ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.codecID = ZegoVideoCodecIDSVC;
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];

self.streamID = @"0012";
[[ZegoExpressEngine sharedEngine] startPublishingStream:self.streamID];

ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.codecID = ZegoVideoCodecIDSVC;
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];

self.streamID = @"0012";
[[ZegoExpressEngine sharedEngine] startPublishingStream:self.streamID];

Specify the layered video to play

After the publishing end enables layered video encoding, the playing end can call the setPlayStreamVideoType interface before or after playing stream. At this time, the playing end will automatically play the appropriate video layer based on network conditions, for example, only playing the base layer in weak networks. Developers can also pass in specific playing parameters to play specific video layers. The currently supported video layers are as follows:

Taking playing the enhancement layer as an example:

[[ZegoExpressEngine sharedEngine] setPlayStreamVideoType:ZegoVideoStreamTypeBig streamID:self.streamID];
[[ZegoExpressEngine sharedEngine] startPlayingStream:self.streamID canvas:playCanvas];

[[ZegoExpressEngine sharedEngine] setPlayStreamVideoType:ZegoVideoStreamTypeBig streamID:self.streamID];
[[ZegoExpressEngine sharedEngine] startPlayingStream:self.streamID canvas:playCanvas];

Video Large and Small Stream Encoding (H.264 DualStream)

The implementation of video large and small stream encoding (H.264 DualStream) is similar to layered video encoding (H.264 SVC), requiring the following two steps:

1. Before publishing stream, enable video large and small stream encoding by specifying a specific encoder.
2. When playing stream, specify the large or small stream video to play.

Enable video large and small stream encoding

Before publishing stream (startPublishingStream), call the setVideoConfig interface to set the parameter codecID in the ZegoVideoConfig class to ZegoVideoCodecIDH264DualStream to enable the video large and small stream encoding feature.

ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.codecID = ZegoVideoCodecIDH264DualStream;
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];

self.streamID = @"0012";
[[ZegoExpressEngine sharedEngine] startPublishingStream:self.streamID];

ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.codecID = ZegoVideoCodecIDH264DualStream;
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];

self.streamID = @"0012";
[[ZegoExpressEngine sharedEngine] startPublishingStream:self.streamID];

Specify the video stream to play

After the publishing end enables video large and small stream encoding, the playing end can call the setPlayStreamVideoType interface before or after playing stream. At this time, the playing end will automatically play the appropriate video stream layer based on network conditions, for example, only playing the base layer in weak networks. Developers can also pass in specific playing parameters to play specific video layers. The currently supported video layers are as follows:

Taking playing the enhancement layer as an example:

[[ZegoExpressEngine sharedEngine] setPlayStreamVideoType:ZegoVideoStreamTypeBig streamID:self.streamID];
[[ZegoExpressEngine sharedEngine] startPlayingStream:self.streamID canvas:playCanvas];

[[ZegoExpressEngine sharedEngine] setPlayStreamVideoType:ZegoVideoStreamTypeBig streamID:self.streamID];
[[ZegoExpressEngine sharedEngine] startPlayingStream:self.streamID canvas:playCanvas];

Hardware Encoding and Decoding

Since a small number of models have poor support for hardware encoding/decoding, the SDK uses software encoding and software decoding by default. If developers need to use hardware encoding, they can configure it themselves by referring to this section.

Enable hardware encoding

If developers need to enable hardware encoding, they can call the enableHardwareEncoder interface.

// Enable hardware encoding
[[ZegoExpressEngine sharedEngine] enableHardwareEncoder:YES];

// Enable hardware encoding
[[ZegoExpressEngine sharedEngine] enableHardwareEncoder:YES];

Enable hardware decoding

If developers need to enable hardware decoding, they can call the enableHardwareDecoder interface.

// Enable hardware decoding
[[ZegoExpressEngine sharedEngine] enableHardwareDecoder:YES];

// Enable hardware decoding
[[ZegoExpressEngine sharedEngine] enableHardwareDecoder:YES];

Set video encoding mode

Before publishing stream (startPublishingStream), call the setVideoConfig interface to set the parameter "codecID" under the "ZegoVideoConfig" class to set the video encoding mode. The currently supported video encoding modes are as follows:

Taking setting the encoding mode to H.265 as an example:

ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.codecID = ZegoVideoCodecIDH265;
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];

ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.codecID = ZegoVideoCodecIDH265;
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];

FAQ

1. When relaying or directly publishing to CDN, and viewers play streams from CDN, are layered video encoding and large and small streams valid? What are the bitrate and resolution of streams played from CDN?

   • Video layered encoding and video large and small stream encoding use ZEGO's private protocol. The playing end can only play streams of different layers when playing RTC streams or L3 streams from the ZEGO server.

   • In the direct CDN publishing scenario, since it does not go through the ZEGO server, the stream layering is invalid, and the SDK will fall back to H.264 encoding. The resolution and bitrate of streams played from CDN are consistent with the resolution and bitrate set by the publishing user.

   • In the CDN relaying scenario, since CDN playing does not use ZEGO's private protocol, the stream relayed by the ZEGO server to the CDN server does not support layered video encoding and video large and small stream encoding. Only one can be selected from relaying base layer or relaying enhancement layer. The resolution, bitrate, and frame rate when playing from CDN depend on whether the base layer or enhancement layer is relayed.

   Warning

   When relaying to CDN, the enhancement layer is relayed by default. If the business needs to relay the base layer to CDN, please contact ZEGO technical support to configure.

2. What is the difference between layered video encoding and large and small stream video encoding?

   For details, please refer to Video Large and Small Stream and Layered Encoding.

Related Documentation

How to solve screen abnormality issues (such as black screen, green screen, flower screen, etc.) when interoperating between Web platform and Native platform?