Background Virtualization and Virtual Background

Feature Overview

The background virtualization and virtual background features can separate the subject from the original video image and process the areas outside the subject as follows:

• Blur the areas outside the subject
• Replace the areas outside the subject with a custom image

Background virtualization and virtual background features are widely suitable for the following scenarios:

• Online meetings
• Remote education
• Video calls

Using background virtualization and virtual background features helps to:

• Protect personal privacy
• Avoid cluttered backgrounds from interfering with the audience
• Enhance the fun of interaction

Effect Demonstration

Feature	Effect
Background Blur-Weak
Background Blur-Medium
Background Blur-Strong
Virtual Background

To experience the actual effect, please visit the Background Virtualization and Virtual Background Demo.

Compatibility

The browsers that support background virtualization and virtual background features are as follows:

Notes

• Background virtualization and virtual background features have high requirements for device performance. It is recommended to enable the virtual background feature on devices that meet the following conditions:
  • Equipped with Core i5 or above CPU.
  • Equipped with 8GB or more RAM.
• To better use background virtualization and virtual background features, it is recommended to use the latest version of the desktop Google Chrome browser.
• Background virtualization and virtual background features are paid features. If you need to apply for a trial or inquire about official pricing, please contact ZEGOCLOUD sales.

Example Source Code

Please refer to Download Example Source Code to obtain the source code.

For related source code, please view files in the "src/Examples/Others/VirtualBackground" directory.

Prerequisites

• You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and Server address. For details, please refer to Console - Project Information.
• You have integrated ZEGO Express SDK in your project and implemented basic audio and video stream publishing and playing functions. For details, please refer to Quick Start - Integration and Quick Start - Implementation.
• To use this feature, please contact ZEGOCLOUD technical support.

Implementation Flow

1. Integrate the background processing module.

   • Method 1: Integrate using npm

   import { ZegoExpressEngine } from'zego-express-engine-webrtc';
   import { BackgroundProcess } from'zego-express-engine-webrtc/background-process';
   // Reference background processing module
   ZegoExpressEngine.use(BackgroundProcess);

   import { ZegoExpressEngine } from'zego-express-engine-webrtc';
   import { BackgroundProcess } from'zego-express-engine-webrtc/background-process';
   // Reference background processing module
   ZegoExpressEngine.use(BackgroundProcess);

   • Method 2: Manual integration after downloading the SDK package from the official website

   <script src="./background-process-XXX.js"></script>
   <script src="./ZegoExpressWebRTC-XXX.js"></script>
   <script>
           ZegoExpressEngine.use(BackgroundProcess);
           ...
   </script>

   <script src="./background-process-XXX.js"></script>
   <script src="./ZegoExpressWebRTC-XXX.js"></script>
   <script>
           ZegoExpressEngine.use(BackgroundProcess);
           ...
   </script>

2. Dynamically load resource files such as wasm that background virtualization and virtual background depend on.

   Publish all resource files in the zego-express-engine-webrtc/background-process/assets directory from the example source code to a CDN or static resource server, and they need to be published to the same resource directory, and generate a URL.

3. Initialize the background processing module and pass in the URL of the resource file from step 2 above. The SDK will dynamically load the dependent files.

   // Initialize instance, appID and server please obtain from the console
   const zg = new ZegoExpressEngine(appID, server);
   try {
      // The second parameter passes in the URL path where the wasm and other resource files from step 2 are located
      await zg.initBackgroundModule(0, "../assets");
   } catch (err) {
      // Console prints initialization background processing module error
      console.error(err);
   }

   // Initialize instance, appID and server please obtain from the console
   const zg = new ZegoExpressEngine(appID, server);
   try {
      // The second parameter passes in the URL path where the wasm and other resource files from step 2 are located
      await zg.initBackgroundModule(0, "../assets");
   } catch (err) {
      // Console prints initialization background processing module error
      console.error(err);
   }

4. After creating the local data stream createZegoStream, set the use of background blur or virtual background, and set related parameters.

   • Use background blur effect

     Call the setBackgroundBlurOptions interface to set the background blur effect.

     const stream = await zg.createZegoStream();
     zg.setBackgroundBlurOptions(stream, {
         blurDegree: 1  // Blur level 1, 2, 3, the higher the level, the higher the blur degree
     });

     const stream = await zg.createZegoStream();
     zg.setBackgroundBlurOptions(stream, {
         blurDegree: 1  // Blur level 1, 2, 3, the higher the level, the higher the blur degree
     });

   • Use virtual background effect

     Call the setVirtualBackgroundOptions interface to set the virtual background effect.

const stream = await zg.createZegoStream();
// source is the HTMLImageElement of the background image
// objectFit is the virtual background fill method: 'contain', 'cover', 'fill'
zg.setVirtualBackgroundOptions(stream, {
    source: img,
    objectFit: 'fill'
});

const stream = await zg.createZegoStream();
// source is the HTMLImageElement of the background image
// objectFit is the virtual background fill method: 'contain', 'cover', 'fill'
zg.setVirtualBackgroundOptions(stream, {
    source: img,
    objectFit: 'fill'
});

5. Call the enableBackgroundProcess interface and set enable to true to enable the background blur and virtual background features.

   // enable passes true to enable virtual background, false to disable
   zg.enableBackgroundProcess(stream, true, 0);

   // enable passes true to enable virtual background, false to disable
   zg.enableBackgroundProcess(stream, true, 0);

6. Call enableBackgroundProcess and set enable to false to disable the background blur and virtual background features.

   // enable passes true to enable virtual background, false to disable
   zg.enableBackgroundProcess(stream, false, 0);

   // enable passes true to enable virtual background, false to disable
   zg.enableBackgroundProcess(stream, false, 0);