Super Resolution
Feature Overview
Super resolution (abbreviated as super resolution) function can double the pixels of the width and height of the pulled video stream image at the play stream end. For example: if the original image resolution pulled by the play stream end is 640p x 360p, after super resolution processing, the resolution will be improved to 1280p x 720p.
Effect Display
- Portrait
| Before Super Resolution | After Super Resolution |
|---|---|
 |  |
- Text
| Before Super Resolution | After Super Resolution |
|---|---|
 |  |
Application Scenarios
Since the same device can only enable super resolution function for 1 stream at the same time, the super resolution function is only suitable for scenarios with only a single stream or 1 focus stream.
- 1V1 video call scenario
- Live streaming scenario: In live streaming scenarios, in most cases only 1 stream is pulled. You can enable super resolution for the pulled single-stream live image. When pulling multiple streams, you can enable super resolution function for the focused anchor.
- Online education: Online education scenarios may have multiple streams, but there will be 1 focus stream (such as a teacher). You can choose to enable super resolution function for the teacher's blackboard image or the image of the student who is speaking to achieve enhancement effects.
Feature Advantages
- Low power consumption: After enabling 360p super resolution, taking OPPO R11 as an example, the current increase is less than 60mA, and the additional power consumption is extremely small.
- Low heat generation: After enabling 360p super resolution for half an hour, taking OPPO R11 (Snapdragon 660) as an example, the temperature rise is less than 1.5°C.
- High performance: For devices that can enable super resolution function, over 95% have CPU increment less than 2% and memory increment less than 100MB.
Example Source Code
Please refer to Download Example Source Code to get the source code.
For related source code, please check the files in the "/ZegoExpressExample/Others/SuperResolution" directory.
Prerequisites
Enabling the super resolution function will consume additional system resources. To ensure user experience, currently only supports enabling super resolution for one play stream image, and the original resolution of this stream is not recommended to exceed 640p × 360p.
Before using the super resolution function, please ensure:
- You have contacted ZEGO Technical Support for special packaging.
- You have integrated ZEGO Express SDK in your project and implemented basic real-time audio and video functionality. For details, please refer to Quick Start.
- You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Management.
Implementation Flow
For Android 12 and above versions, you need to add the following configuration in <application> in AndroidManifest.xml. For details, please refer to Android Developers related instructions.
<application>
<uses-native-library android:name="libOpenCL.so" android:required="false" />
<uses-native-library android:name="libOpenCL-pixel.so" android:required="false" />
</application>Initialize and login to Room
For the specific process of initialization and logging in to Room, please refer to "Create Engine" and "Login to Room" in the implementing video call documentation.
Listen to super resolution state callback
After enabling the super resolution function, you can confirm whether the super resolution function is successfully enabled through onPlayerVideoSuperResolutionUpdate.
The super resolution function state definitions are as follows:
| Enum value | Description |
|---|---|
| ZegoSuperResolutionStateOff | Super resolution function is disabled. |
| ZegoSuperResolutionStateOn | Super resolution function is enabled. |
Through onPlayerVideoSizeChanged, you can obtain the play stream video resolution after the super resolution function is enabled.
engine.setEventHandler(new IZegoEventHandler() {
// Notification of play stream video resolution change
@Override
public void onPlayerVideoSizeChanged(String streamID, int width, int height) {
super.onPlayerVideoSizeChanged(streamID, width, height);
}
// Notification of super resolution state change
@Override
public void onPlayerVideoSuperResolutionUpdate(String streamID, ZegoSuperResolutionState state, int errorCode) {
super.onPlayerVideoSuperResolutionUpdate(streamID, state, errorCode);
if(state == ZegoSuperResolutionState.ON) {
// Super resolution enabled
}else if(state == ZegoSuperResolutionState.OFF) {
// Super resolution disabled
if(errorCode == 0){
// Normal shutdown
}
else if(errorCode == 1004004){
// The device does not support super resolution
}
else if(errorCode == 1004005){
// The number of super resolution streams exceeds the limit, only supports super resolution for one stream
}
else if(errorCode == 1004006){
// Super resolution original resolution exceeds the limit
}
else if(errorCode == 1004007){
// Super resolution device performance is insufficient
}
else if(errorCode == 1004008){
// Super resolution not initialized, please initialize super resolution first
}
}
}
});Initialize video super resolution
Before using the super resolution function, you need to call the initVideoSuperResolution interface to initialize super resolution.
Initializing video super resolution is a time-consuming operation. It only needs to be executed once in the SDK lifecycle. It is not recommended to frequently initialize and uninitialize super resolution.
engine.initVideoSuperResolution();Enable video super resolution
Developers can choose to enable the video super resolution function before or during playing stream. Taking enabling super resolution function for stream "STREAM_ID" before playing stream as an example:
engine.enableVideoSuperResolution("STREAM_ID", true);After enabling the super resolution function, you need to listen to the callback onPlayerVideoSuperResolutionUpdate to confirm whether this super resolution function was successfully enabled.
To facilitate managing the super resolution state, it is recommended to wait for the super resolution state callback before enabling the next super resolution function.
Start playing stream
Call startPlayingStream and pass in "STREAM_ID" to pull the remote user's audio and video stream.
engine.startPlayingStream("STREAM_ID", new ZegoCanvas(playView));Disable video super resolution
When the super resolution function is not needed or an error occurs during the super resolution function (insufficient performance or original video resolution exceeds the limit), you can disable the super resolution function to release system resources.
engine.enableVideoSuperResolution("STREAM_ID", false);Stop playing stream
After stopping the play stream, the SDK will automatically disable the super resolution function and release the occupied system resources.
engine.stopPlayingStream("STREAM_ID");Uninitialize video super resolution
When the video super resolution function is not needed, you can call the uninitVideoSuperResolution interface to uninitialize video super resolution to save memory.
engine.uninitVideoSuperResolution();