Publishing Stream Video Enhancement
Feature Overview
ZEGO Express SDK provides various video pre-processing enhancement capabilities. Developers can adjust the visual effects on the publishing stream side according to business requirements.
| Feature | Description | Advantages | Effect Display | |
|---|---|---|---|---|
| Basic Beauty | Adjust the levels of whitening, smoothing, sharpening, and rosy according to user and business needs. Easily implement basic beauty features to present good skin condition for users and create unique, natural beauty effects. | Covers high-frequency beauty capabilities. |  | |
| Low-light Enhancement | When the ambient light is dim and the camera capture brightness does not meet business needs such as facial recognition, enhance the image brightness. |
|  |  |
| Color Enhancement | Due to camera characteristics, captured video may have insufficient saturation. Using color enhancement enhances undersaturated colors while protecting skin tones, making colors more vivid and more aligned with human visual perception. |
|  |  |
Prerequisites
Before using the publishing stream video enhancement feature, ensure that:
- You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, refer to Console - Project Information.
- You have integrated ZEGO Express SDK in your project and implemented basic audio and video stream publishing and playing. For details, refer to Quick Start - Integration and Quick Start - Implementation.
Usage Steps
Initialize and Login to Room
For the specific process of initialization and logging in to a room, refer to "Create Engine" and "Login Room" in the Implementing Video Call documentation.
Set Publishing Stream Video Enhancement Effects
Please configure the corresponding features according to your needs.
This feature is the basic beauty functionality of ZEGO Effects SDK built into ZEGO Express SDK, which only includes four beauty effects: whitening, rosy, smoothing, and sharpening. For more advanced beauty features, refer to Using Video Call with AI Effects.
Initialize Basic Beauty Environment
If you need to use the basic beauty feature, you must call the startEffectsEnv interface to initialize the beauty environment before starting preview with startPreview or starting stream publishing with startPublishingStream.
// Create beauty environment
engine->startEffectsEnv();After initializing the beauty environment, the SDK internally uses a fixed video frame data type for transmission. If you need to use custom video pre-processing while using the basic beauty feature, you can only configure the corresponding video frame data type:
Windows platform: Only supports video frame raw data, that is, the bufferType of ZegoCustomVideoCaptureConfig takes the value "ZEGO_VIDEO_BUFFER_TYPE_RAW_DATA".
Enable/Disable Basic Beauty Effects
There is no specific order requirement for calling enableEffectsBeauty and setEffectsBeautyParam.
After initializing the beauty environment, you can call the enableEffectsBeauty interface before or after publishing stream to enable or disable beauty effects in real time.
// Enable/disable beauty effects
engine->enableEffectsBeauty(true);Set Basic Beauty Effect Parameters
After initializing the beauty environment, you can call the setEffectsBeautyParam interface before or after publishing stream to set beauty effect parameters in real time.
- smoothIntensity: Smoothing, performs smoothing while retaining facial details, such as keeping moles on the face.
- whitenIntensity: Whitening, increases overall brightness of the image to whiten the face.
- rosyIntensity: Rosy, applies warm color processing to the overall image.
- sharpenIntensity: Sharpening, performs sharpening on the overall image. When the image is slightly blurry, you can increase sharpening slightly to make contours clearer.
The value range for all four parameters above is 0 to 100. A larger value indicates a higher degree of beauty enhancement. The default value is 50.
// Create beauty parameter object
ZegoEffectsBeautyParam beautyParam;
// Whitening, rosy, smoothing, sharpening
beautyParam.whitenIntensity = 50;
beautyParam.rosyIntensity = 50;
beautyParam.smoothIntensity = 50;
beautyParam.sharpenIntensity = 50;
// Set beauty parameters
engine->setEffectsBeautyParam(beautyParam);(Optional) Destroy Basic Beauty Environment
When the enableEffectsBeauty interface is called and set to "false", the beauty effects will be disabled, but the beauty module will still occupy resources and consume performance. If you want to completely release resources and save performance consumption, you need to call the stopEffectsEnv interface to destroy the beauty environment before preview and stream publishing.
When destroyEngine is called to destroy the engine, the SDK will automatically destroy the beauty environment.
// Destroy beauty environment
engine->stopEffectsEnv();ZEGO provides two methods for low-light enhancement:
| Method | Description |
|---|---|
| Enable adaptive frame rate for enhancement | Dynamically reduce the capture frame rate within the set frame rate range to extend the exposure time of each frame, achieving the effect of enhancing the brightness of captured images.
For example: The developer enables and sets the adaptive frame rate range to [10, 25]. When ambient light is sufficient, the capture frame rate remains 25 fps. When ambient light is low, the capture frame rate will be dynamically adjusted between [10, 25] to extend the exposure time of each frame and enhance the brightness of captured images. |
| Algorithm enhances captured image brightness |
|
Both processing methods can be used independently or simultaneously, and both are compatible with beauty features.
Set Low-light Enhancement Mode
RTC SDK supports different algorithm low-light enhancement modes. Call the setLowlightEnhancement interface to set the low-light enhancement mode ZegoLowlightEnhancementMode:
| Enumeration | Description |
|---|---|
| ZEGO_LOWLIGHT_ENHANCEMENT_MODE_OFF | Disable low-light enhancement feature. |
| ZEGO_LOWLIGHT_ENHANCEMENT_MODE_ON | Enable low-light enhancement feature, controlled by the developer. |
| ZEGO_LOWLIGHT_ENHANCEMENT_MODE_AUTO | Smart control of low-light enhancement feature, automatically enables or disables based on image lighting conditions. |
Low-light enhancement type ZegoExpLowlightEnhancementType includes the following types:
| Enumeration | Description |
|---|---|
| ZEGO_EXP_LOWLIGHT_ENHANCEMENT_TYPE_NORMAL | Regular low-light enhancement. |
| ZEGO_EXP_LOWLIGHT_ENHANCEMENT_TYPE_AI | AI low-light enhancement. To use this feature, you need to contact ZEGOCLOUD Technical Support. |
Taking enabling automatic low-light enhancement as an example:
EXPRESS::ZegoExpLowlightEnhancementParams params;
params.mode = ZEGO_LOWLIGHT_ENHANCEMENT_MODE_AUTO;
params.type = ZEGO_EXP_LOWLIGHT_ENHANCEMENT_TYPE_AI
engine->setLowlightEnhancementParams(params);(Optional) Enable Camera Adaptive Frame Rate
When ambient light is low and the camera capture frame rate is too high, the exposure time of each frame may be insufficient, resulting in dark captured video images. In this case, you can enable camera adaptive frame rate through enableCameraAdaptiveFPS. The SDK will match the camera-supported capture frame rate range based on the set frame rate range. Within this range, it dynamically adjusts the camera capture frame rate according to ambient brightness, improving image brightness when the set frame rate is too high.
int minCamFPS = 15;//Set minimum frame rate
int maxCamFPS = 35;//Set maximum frame rate
engine->enableCameraAdaptiveFPS(true, minCamFPS, maxCamFPS);Call the enableColorEnhancement interface, set color enhancement parameters ZegoColorEnhancementParams (parameter properties are shown in the table below), and other parameters to enable the color enhancement feature. Color enhancement is disabled by default.
| Color Enhancement Parameters | Description |
|---|---|
| intensity | Intensity of color enhancement, value range [0.0, 1.0], default is 0.0. A larger value indicates a stronger color enhancement effect. |
| skinToneProtectionLevel | Intensity of skin tone protection, value range [0.0, 1.0], default is 1.0. A larger value indicates better protection effect. |
| lipColorProtectionLevel | Intensity of lip color protection, value range [0.0, 1.0], default is 0.0. A larger value indicates better protection effect. |
ZegoColorEnhancementParams p;
p.intensity = 1;//Value range: [0,1], larger value indicates stronger color enhancement intensity. Default value: 0.
p.skinToneProtectionLevel = 1;//Value range: [0,1], larger value indicates stronger skin tone protection. Default value: 1.
p.lipColorProtectionLevel = 1;//Value range: [0,1], larger value indicates stronger lip color protection. Default value: 0.
engine->enableColorEnhancement(true, p, ZEGO_PUBLISH_CHANNEL_MAIN);//Enable color enhancementStart Preview
After calling the startPreview interface to start preview, you can set different video enhancement features and experience the effects in real time.
ZegoCanvas previewCanvas(preview);
engine->startPreview(&previewCanvas);Start Publishing Stream
When the preview effect meets expectations, you can call the startPublishingStream interface to start publishing stream. The published stream image effect will be consistent with the preview effect.
engine->startPublishingStream("STREAM_ID");The above implements the video enhancement feature before publishing stream.