Publishing Stream Video Enhancement
Feature Overview
ZEGO Express SDK provides various video pre-processing enhancement capabilities. Developers can adjust the video effects at the publishing end according to business needs.
| Feature | Feature Description | Feature Advantages | Effect Display | |
|---|---|---|---|---|
| Basic Face Beautification | Adjust the levels of whitening, smoothing, sharpening, and rosy effects according to user and business needs. Easily implement basic face beautification functionality, presenting users with good skin condition and creating unique, natural beautification effects. | Covers high-frequency beautification capabilities. |  | |
| Low Light Enhancement | When ambient light is dark and the brightness captured by the camera doesn't meet business needs such as seeing faces clearly or performing face recognition, enhance the image brightness. |
|  |  |
| Video Noise Reduction | Poor camera capture quality or low ambient light brightness may cause obvious noise in the image. Video noise reduction functionality can reduce image noise. |
|  |  |
| Color Enhancement | Due to camera characteristics, captured video may have insufficient saturation. Using color enhancement, enhance under-saturated colors while protecting skin tones, making image colors more realistic and more in line with human visual perception. |
|  |  |
Prerequisites
Before using publishing stream video enhancement functionality, please ensure:
- You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
- You have integrated ZEGO Express SDK in your project and implemented basic audio/video publishing and playing functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.
Usage Steps
1 Initialization and Log In to Room
For specific initialization and room login flow, please refer to "Create Engine" and "Log In to Room" in the Implementing Video Call documentation.
2 Set Publishing Stream Video Enhancement Effects
Please configure the corresponding features according to your needs.
This feature is the basic face beautification functionality of ZEGO Effects SDK built into ZEGO Express SDK. It only includes four beautification effects: whitening, rosy, smoothing, and sharpening. For more advanced beautification features, please refer to Using Real-time Audio/Video with AI Face Beautification.
Initialize Basic Face Beautification Environment
If you need to use basic face beautification functionality, you must call the startEffectsEnv interface to initialize the beautification environment before startPreview starts preview and startPublishingStream starts publishing.
On the Web platform, you don't need to additionally enable the beautification environment. You can skip calling startEffectsEnv.
// Create beautification environment
ZegoExpressEngine.instance.startEffectsEnv();Toggle Basic Face Beautification Effects
- There is no specific order for calling the enableEffectsBeauty interface and the setEffectsBeautyParam interface.
- On the Web platform, you need to execute enableEffectsBeauty and setEffectsBeautyParam before publishing for beautification effects to take effect.
After initializing the beautification environment, you can call the enableEffectsBeauty interface before or after publishing (on the Web platform, before publishing) to toggle beautification effects in real-time.
// Toggle beautification effects
ZegoExpressEngine.instance.enableEffectsBeauty(true);Set Basic Face Beautification Effect Parameters
After initializing the beautification environment, you can call the setEffectsBeautyParam interface before or after publishing to set beautification effect parameters in real-time.
- smoothIntensity: Smoothing, performs smoothing while preserving facial details, such as keeping moles on the face.
- whitenIntensity: Whitening, brightens the overall image to whiten the face.
- rosyIntensity: Rosy, applies warm color processing to the overall image.
- sharpenIntensity: Sharpening, applies sharpening to the overall image. When the image is slightly blurry, you can increase sharpening slightly to make contours clearer.
The value range for all four parameters above is 0 to 100. The larger the value, the higher the beautification degree. The default value is 50.
// Create beautification parameter object
var beautyParam = ZegoEffectsBeautyParam.defaultParam();
// Whitening, rosy, smoothing, sharpening
beautyParam.whitenIntensity = 50;
beautyParam.rosyIntensity = 50;
beautyParam.smoothIntensity = 50;
beautyParam.sharpenIntensity = 50;
// Set beautification parameters
ZegoExpressEngine.instance.setEffectsBeautyParam(beautyParam);（Optional）Destroy Basic Face Beautification Environment
When calling the enableEffectsBeauty interface and setting it to "false", beautification effects will be disabled, but the beautification module will still occupy resources and consume performance. If you want to completely release resources and save performance consumption, you need to call the stopEffectsEnv interface to destroy the beautification environment before preview and publishing.
- When calling destroyEngine to destroy the engine, the SDK will automatically destroy the beautification environment.
- On the Web platform, since you don't need to additionally enable the beautification environment, you don't need to call stopEffectsEnv. If you need to stop beautification effects, you can execute stopEffectsEnv or set the enableEffectsBeauty interface to "false". If you don't execute stopEffectsEnv or don't set the enableEffectsBeauty interface to "false", when stopping publishing or preview streams, the stream will be destroyed and beautification effects will disappear.
// Destroy beautification environment
ZegoExpressEngine.instance.stopEffectsEnv();ZEGO provides two methods for low light enhancement:
| Method | Description |
|---|---|
| Enable adaptive frame rate for enhancement | Dynamically reduce capture frame rate within the set frame rate range to extend each frame's exposure time, achieving enhanced captured image brightness.
For example: A developer enables and sets the adaptive frame rate range to [10, 25]. When ambient light is sufficient, the capture frame rate remains 25 fps. When ambient light is low, the capture frame rate dynamically adjusts between [10, 25] to extend each frame's exposure time and enhance captured image brightness. |
| Algorithm enhances captured image brightness |
|
Both processing methods can be used independently or simultaneously, and both are compatible with beautification functionality.
Set Low Light Enhancement Mode
RTC SDK supports different algorithm low light enhancement modes. Call the setLowlightEnhancementParams interface to set the low light enhancement mode ZegoLowlightEnhancementMode:
| Enumeration Value | Description |
|---|---|
| ZegoLowlightEnhancementMode.Off | Turn off low light enhancement functionality. |
| ZegoLowlightEnhancementMode.On | Turn on low light enhancement functionality, controlled by the developer. |
| ZegoLowlightEnhancementMode.Auto | Smart control of low light enhancement functionality, automatically enables or disables based on image lighting conditions. |
Low light enhancement type ZegoExpLowlightEnhancementType includes the following types:
| Enumeration Value | Description |
|---|---|
| ZegoExpLowlightEnhancementType.Normal | Regular low light enhancement. |
| ZegoExpLowlightEnhancementType.AI | AI low light enhancement. To use this feature, please contact ZEGO technical support. |
Example of enabling automatic low light enhancement:
ZegoExpLowlightEnhancementParams params = ZegoExpLowlightEnhancementParams(ZegoLowlightEnhancementMode.Auto, ZegoExpLowlightEnhancementType.Normal);
ZegoExpressEngine.instance.setLowlightEnhancementParams(params, channel: ZegoPublishChannel.Main);（Optional）Enable Camera Adaptive Frame Rate
When ambient light is low and the camera capture frame rate is too high, each frame's exposure time is insufficient, which may cause the captured video to appear dark. In this case, you can enable camera adaptive frame rate through enableCameraAdaptiveFPS. The SDK will match the camera's supported capture frame rate range based on the set frame rate range. Within this range, it dynamically adjusts the camera capture frame rate based on ambient brightness to improve image brightness when the set frame rate is too high.
//Set minimum frame rate
int minCamFPS = 15;
//Set maximum frame rate
int maxCamFPS = 35;
ZegoExpressEngine.instance.enableCameraAdaptiveFPS(true, minCamFPS, maxCamFPS, ZegoPublishChannel.Main);Call the setVideoDenoiseParams interface to set video noise reduction parameters ZegoVideoDenoiseParams including: video noise reduction mode ZegoVideoDenoiseMode and video noise reduction intensity ZegoVideoDenoiseStrength.
Video noise reduction functionality is not enabled by default, i.e., ZegoVideoDenoiseMode defaults to ZegoVideoDenoiseMode.off.
Example of setting automatic noise reduction:
ZegoVideoDenoiseParams p = ZegoVideoDenoiseParams(ZegoVideoDenoiseMode.Auto, ZegoVideoDenoiseStrength.Light);
ZegoExpressEngine.instance.setVideoDenoiseParams(p, channel: ZegoPublishChannel.Main);Call the enableColorEnhancement interface, set color enhancement parameters ZegoColorEnhancementParams (parameter properties are in the following table), and other parameters to enable color enhancement functionality. Color enhancement functionality is not enabled by default.
| Color Enhancement Parameter | Description |
|---|---|
| intensity | Intensity of color enhancement, value range [0.0, 1.0], default is 0.0. The larger the value, the greater the color enhancement effect. |
| skinToneProtectionLevel | Intensity of skin tone protection, value range [0.0, 1.0], default is 1.0. The larger the value, the better the protection effect. |
| lipColorProtectionLevel | Intensity of lip color protection, value range [0.0, 1.0], default is 0.0. The larger the value, the better the protection effect. |
ZegoColorEnhancementParams p = ZegoColorEnhancementParams.defaultParam();
p.intensity = 1; //Value range: [0,1], the larger the value, the greater the color enhancement intensity. Default value: 0.
p.skinToneProtectionLevel = 1; //Value range: [0,1], the larger the value, the greater the skin tone protection. Default value: 1.
p.lipColorProtectionLevel = 1; //Value range: [0,1], the larger the value, the greater the lip color protection. Default value: 0.
ZegoExpressEngine.instance.enableColorEnhancement(true, p, channel: ZegoPublishChannel.Main);//Enable color enhancement3 Start Preview
After calling the startPreview interface to start preview, you can set different video enhancement features and experience the effects in real-time.
Get the Widget for preview, then use viewID to create a ZegoCanvas object and start preview.
// Add this Widget to the page's rendering tree to display the preview
_previewViewWidget = await ZegoExpressEngine.instance.createCanvasView((viewID) {
_previewViewID = viewID;
// Set the preview canvas
ZegoCanvas previewCanvas = ZegoCanvas.view(viewID);
// Start preview
ZegoExpressEngine.instance.startPreview(canvas: previewCanvas);
});4 Start Publishing
When the preview effect meets expectations, you can call the startPublishingStream interface to start publishing. The published stream effect will be consistent with the preview effect.
ZegoExpressEngine.instance.startPublishingStream("streamID");The above completes the pre-publishing video enhancement functionality.