Background segmentation

The ZegoEffects SDK provides various background segmentation features that can be used in a wide range of use cases, such as online education, video conferencing, short videos, and more.

Prerequisites

Before you begin to use the SDK's background segmentation feature in your project, make sure you complete the following steps:

• You have integrated the ZegoEffects SDK and implemented basic image processing functionality. For details, see Implement basic image processing.
• You have imported the basic "SegmentationModel" resource files. For details, see Import resources and models.

Implementation steps

The SDK supports two background segmentation methods: portrait segmentation and green screen segmentation. For each segmentation method, you can also enable the background blurring or background mosaicing feature as needed.

Portrait Segmentation

1. Call the setPortraitSegmentationBackgroundPath interface to set the background path for portrait segmentation.

   Untitled

   // Set the background path for portrait segmentation
   mEffects.setPortraitSegmentationBackgroundPath("MY_BACKGROUND_PATH", ZegoEffectsScaleMode.ASPECT_FILL);
   

   1

   Copied!

2. Call the enablePortraitSegmentation interface to enable the portrait segmentation feature.

   Untitled

   // Enable portrait segmentation feature
   mEffects.enablePortraitSegmentation(true);
   

   1

   Copied!

After enabling the portrait segmentation feature, developers can further set background blur, background mosaic, and other functions as needed.

Portrait Segmentation Background Blur

Call the enablePortraitSegmentationBackgroundBlur interface to enable the background blur feature for portrait segmentation. After enabling the blur feature, you can call the setPortraitSegmentationBackgroundBlurParam interface to set the "intensity" parameter of the ZegoEffectsBlurParam object to adjust the degree of background blur.

// Enable portrait segmentation background blur
mEffects.enablePortraitSegmentationBackgroundBlur(true);

// Set portrait segmentation background blur parameters
mEffects.setPortraitSegmentationBackgroundBlurParam({ intensity: 100 });

Portrait Segmentation Background Mosaic

Call the enablePortraitSegmentationBackgroundMosaic interface to enable the background mosaic feature for portrait segmentation. After enabling the mosaic feature, you can call the setPortraitSegmentationBackgroundMosaicParam interface to set the "intensity" parameter of the ZegoEffectsMosaicParam object to adjust the degree of mosaic.

// Enable portrait segmentation background mosaic
mEffects.enablePortraitSegmentationBackgroundMosaic(true);

// Set portrait segmentation background mosaic parameters
mEffects.setPortraitSegmentationBackgroundMosaicParam({ intensity: 100 });

Chroma Key Segmentation

1. Call the setChromaKeyBackgroundPath interface to set the custom background path for chroma key segmentation.

   Untitled

   // Set the background path for chroma key segmentation
   mEffects.setChromaKeyBackgroundPath("MY_BACKGROUND_PATH", ZegoEffectsScaleMode.ASPECT_FILL);
   

   1

   Copied!

2. Call the enableChromaKey interface to enable the chroma key segmentation feature, then call the setChromaKeyParam interface to set parameters and adjust color tolerance, edge smoothness, etc.

   Untitled

   // Enable chroma key segmentation feature
   mEffects.enableChromaKey(true);
   
   // Set chroma key segmentation parameters
   let param = new ZegoEffectsChromaKeyParam();
   param.smoothness = 80;
   param.opacity = 100;
   param.keyColor = 65280;
   param.similarity = 67;
   mEffects.setChromaKeyParam(param);
   

   1

   Copied!

After enabling the chroma key segmentation feature, developers can further set background blur, background mosaic, and other functions as needed.

Chroma Key Segmentation Background Blur

Call the enableChromaKeyBackgroundBlur interface to enable the background blur feature for chroma key segmentation. After enabling the blur feature, you can call the setChromaKeyBackgroundBlurParam interface to set the "intensity" parameter of the ZegoEffectsBlurParam object to adjust the degree of blur.

// Enable chroma key segmentation background blur
mEffects.enableChromaKeyBackgroundBlur(true);

// Set chroma key segmentation background blur parameters
mEffects.setChromaKeyBackgroundBlurParam({ intensity: 100 });

Chroma Key Segmentation Background Mosaic

Call the enableChromaKeyBackgroundMosaic interface to enable the background mosaic feature for chroma key segmentation. After enabling the mosaic feature, you can call the setChromaKeyBackgroundMosaicParam interface to set the "intensity" parameter of the ZegoEffectsMosaicParam object to adjust the degree of mosaic.

// Enable chroma key segmentation background mosaic
mEffects.enableChromaKeyBackgroundMosaic(true);

// Set chroma key segmentation background mosaic parameters
mEffects.setChromaKeyBackgroundMosaicParam({ intensity: 100 });