Video Call (Unity3D C#) Screen Sharing

2025-01-20

Function Overview

Screen sharing refers to sharing screen content as video with other audiences during video calls or interactive live streaming to enhance interactive experience and improve communication efficiency.

Screen sharing is widely used in the following scenarios:

In video conference scenarios, screen sharing can share local files, data, web pages, PPT, and other visuals of the speaker with other participants;
In online classroom scenarios, screen sharing can display courseware, notes, lecture content, and other visuals of the teacher to students.

Download Example Source Code

Please refer to Download Example Source Code to get the source code.

For related source code, please view the "/Assets/ZegoExpressExample/Examples/Others/ScreenCapture.cs" file.

Prerequisites

Before implementing screen sharing functionality, please ensure:

Android: Supports Android 5.0 or above, and supports Android devices or emulators with audio and video (real devices are recommended).
iOS:
- Supports iOS 12.0 or above, and supports iOS devices or emulators with audio and video (real devices are recommended).
- This feature has high device performance requirements and is recommended for use on iPhone X and later models.
Windows:
- Visual Studio 2013 or above.
- Windows7, Windows8, Windows10 or above.
You have created a project in ZEGOCLOUD Console and applied for valid AppID and AppSign. For details, please refer to Console - Project Information.
You have integrated ZEGO Express SDK in the project and implemented basic audio and video publishing and playing functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation Flow.

Implementation Flow

Warning

During screen capture, only iOS and Android platforms support simultaneous capture of video and audio; other platforms only support video capture. If audio capture is required, developers need to implement related logic themselves.

The following describes how to enable screen sharing functionality for each development platform (Android, iOS, macOS, Windows).

Android

The following figure shows the data flow for screen sharing on Android platform:

If you need to use the screen sharing feature, please make sure to obtain user screen recording authorization, otherwise you cannot use this feature.
Set the capture source to screen sharing source.
- The default "video source" for SDK publishing is camera. If you need to publish screen sharing source, you need to switch to screen sharing through SetVideoSource.
```
engine.SetVideoSource(ZegoVideoSourceType.ScreenCapture, ZegoPublishChannel.Main);
```
```
engine.SetVideoSource(ZegoVideoSourceType.ScreenCapture, ZegoPublishChannel.Main);
```
- The default "audio source" for SDK publishing is microphone. If you need to publish screen sharing source, you need to switch to screen sharing through SetAudioSource.
```
engine.SetAudioSource(ZegoAudioSourceType.ScreenCapture, ZegoPublishChannel.Main);
```
```
engine.SetAudioSource(ZegoAudioSourceType.ScreenCapture, ZegoPublishChannel.Main);
```

Call the StartScreenCapture interface to share the entire system's screen and capture third-party application audio to start screen sharing.

engine.StartScreenCapture();

engine.StartScreenCapture();

Developers can also use ZegoScreenCaptureConfig parameters to set whether to capture video, whether to capture audio, set the sample rate and sample channel when capturing audio, etc.

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = true;
config.audioParam.sampleRate = ZegoAudioSampleRate.ZegoAudioSampleRate16K;
config.audioParam.channel = ZegoAudioChannel.Stereo;

engine.StartScreenCapture(config);

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = true;
config.audioParam.sampleRate = ZegoAudioSampleRate.ZegoAudioSampleRate16K;
config.audioParam.channel = ZegoAudioChannel.Stereo;

engine.StartScreenCapture(config);

Call the UpdateScreenCaptureConfig interface to update the screen sharing configuration.

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = false;
config.audioParam.sampleRate = ZegoAudioSampleRate.ZegoAudioSampleRate32K;
config.audioParam.channel = ZegoAudioChannel.Stereo;

engine.UpdateScreenCaptureConfig(config);

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = false;
config.audioParam.sampleRate = ZegoAudioSampleRate.ZegoAudioSampleRate32K;
config.audioParam.channel = ZegoAudioChannel.Stereo;

engine.UpdateScreenCaptureConfig(config);

Developers need to listen to the OnMobileScreenCaptureExceptionOccurred callback in the IZegoEventHandler class to receive exception notifications during screen sharing.

void OnMobileScreenCaptureExceptionOccurred(ZegoScreenCaptureExceptionType exceptionType)
{

}

engine.onMobileScreenCaptureExceptionOccurred = OnMobileScreenCaptureExceptionOccurred;

void OnMobileScreenCaptureExceptionOccurred(ZegoScreenCaptureExceptionType exceptionType)
{

}

engine.onMobileScreenCaptureExceptionOccurred = OnMobileScreenCaptureExceptionOccurred;

iOS

iOS platform implements screen recording based on Apple's Replaykit framework, which can share the entire system's screen content. However, the current App (main App process) needs to additionally provide an Extension extension component (Extension process) for recording the screen, and then combine with ZEGO Express SDK related APIs to implement screen sharing functionality. For details, please refer to the "Implementation Flow" description in Screen Sharing (iOS).

Set the capture source to screen sharing source in the main App.
- The default "video source" for SDK publishing is camera. If you need to publish screen sharing source, you need to switch to screen sharing through SetVideoSource.
```
engine.SetVideoSource(ZegoVideoSourceType.ScreenCapture, ZegoPublishChannel.Main);
```
```
engine.SetVideoSource(ZegoVideoSourceType.ScreenCapture, ZegoPublishChannel.Main);
```
- The default "audio source" for SDK publishing is microphone. If you need to publish screen sharing source, you need to switch to screen sharing through SetAudioSource.
```
engine.SetAudioSource(ZegoAudioSourceType.ScreenCapture, ZegoPublishChannel.Main);
```
```
engine.SetAudioSource(ZegoAudioSourceType.ScreenCapture, ZegoPublishChannel.Main);
```

In-app screen sharing. If users only share visuals and sound within the app, they can call the StartScreenCaptureInApp interface to enable screen sharing. You can also call the broadcastFinished interface callback to notify the end of screen sharing. If screen capture fails, the reason for the failure can be received.

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = true;
engine.StartScreenCaptureInApp(config);

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = true;
engine.StartScreenCaptureInApp(config);

Cross-app screen sharing. If users need to share the entire system's visuals and sound, they can call the StartScreenCapture interface to enable screen sharing.

Note

Cross-app screen sharing is recorded by the iOS system through the Extension extension process, so you need to additionally create the extension process and start it.

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = true;

engine.StartScreenCapture(config);

ZegoScreenCaptureConfig config = new ZegoScreenCaptureConfig();
config.captureVideo = true;
config.captureAudio = true;

engine.StartScreenCapture(config);

Create a new Broadcast Upload Extension. For details, please refer to "Cross-app screen sharing" step 1 in Screen Sharing (iOS).
Set up Extension. For details, please refer to "Cross-app screen sharing" step 2 in Screen Sharing (iOS).
Start screen sharing. For details, please refer to "Cross-app screen sharing" step 3 in Screen Sharing (iOS).
Start the Extension extension process. For details, please refer to "Cross-app screen sharing" step 4 in Screen Sharing (iOS).
Get screen data. For details, please refer to "Cross-app screen sharing" step 5 in Screen Sharing (iOS).

macOS/Windows

To develop the "screen sharing" feature on macOS platform, you need to set the following permissions. No permissions need to be set on Windows platform.

Calling the screen sharing interface on macOS platform will obtain relevant permissions. You need to enable screen recording permissions and accessibility permissions in "Security & Privacy". If it does not take effect, you need to delete the previous permissions and add them again.
- Screen recording permission
- Accessibility permission

Get window (including screen) list information. You can get information about all currently shareable windows through the GetScreenCaptureSources interface.

List<ZegoScreenCaptureSourceInfo> captureSources = engine.GetScreenCaptureSources(400, 400, 100, 100);

List<ZegoScreenCaptureSourceInfo> captureSources = engine.GetScreenCaptureSources(400, 400, 100, 100);

Through the window ID and window type in the above window information, call the CreateScreenCaptureSource interface to create a screen sharing source object.

if (captureSources.Count > 0) {
    var selectSource = captureSources.ElementAt(0);
    ZegoScreenCaptureSource captureSource = engine.CreateScreenCaptureSource(selectSource.sourceID, selectSource.sourceType);
}

if (captureSources.Count > 0) {
    var selectSource = captureSources.ElementAt(0);
    ZegoScreenCaptureSource captureSource = engine.CreateScreenCaptureSource(selectSource.sourceID, selectSource.sourceType);
}

Set the capture source to screen sharing source. The default video source for SDK publishing is camera source. If you need to publish screen sharing source, you need to switch to screen sharing through SetVideoSource.

engine.SetVideoSource(ZegoVideoSourceType.ScreenCapture, (uint)captureSource.GetIndex(), ZegoPublishChannel.Main);

engine.SetVideoSource(ZegoVideoSourceType.ScreenCapture, (uint)captureSource.GetIndex(), ZegoPublishChannel.Main);

Start screen sharing. Call the StartCapture interface to share window visuals.
```
captureSource.StartCapture();
```
```
captureSource.StartCapture();
```

Call the UpdateCaptureSource interface to update the shared window visuals.

var selectSource = captureSources.ElementAt(0);
captureSource.UpdateCaptureSource(selectSource.sourceID, selectSource.sourceType);

var selectSource = captureSources.ElementAt(0);
captureSource.UpdateCaptureSource(selectSource.sourceID, selectSource.sourceType);

Update sharing source region. Users can call the UpdateCaptureRegion interface to dynamically update the shared window region. Setting it to (0, 0, 0, 0) can restore the entire window sharing.

ZegoRect rect = new ZegoRect
{
    x = 10,
    y = 10,
    width = 400,
    height = 400
};
captureSource.UpdateCaptureRegion(rect);

ZegoRect rect = new ZegoRect
{
    x = 10,
    y = 10,
    width = 400,
    height = 400
};
captureSource.UpdateCaptureRegion(rect);

Filter specified windows. Call the SetExcludeWindowList interface to filter out specified window visuals in the shared screen. This is only applicable when sharing the entire screen.

var selectSource = captureSources.ElementAt(0);
List<long> excludeList = new List<long>();
excludeList.Add(selectSource.sourceID);

captureSource.SetExcludeWindowList(excludeList);

var selectSource = captureSources.ElementAt(0);
List<long> excludeList = new List<long>();
excludeList.Add(selectSource.sourceID);

captureSource.SetExcludeWindowList(excludeList);

Users can call the EnableWindowActivate interface to activate the currently shared window.
```
captureSource.EnableWindowActivate(true);
```
```
captureSource.EnableWindowActivate(true);
```

Call the EnableCursorVisible interface to show or hide the mouse.

captureSource.EnableCursorVisible(true);

captureSource.EnableCursorVisible(true);

Listen to event callbacks.

Through the methods OnAvailableFrame, OnExceptionOccurred, OnWindowStateChanged, OnRectChanged in IZegoScreenCaptureSourceEventHandler to listen to event callbacks.

// Capture data callback, can be used for preview video rendering, local recording.
void OnAvailableFrame(ZegoScreenCaptureSource source, System.IntPtr data,
                                        uint dataLength, ZegoVideoFrameParam param)
{
#if (UNITY_STANDALONE_WIN || UNITY_STANDALONE_OSX)
    localVideoSurface.LoadNativeVideoRawData(data, param.width, param.height, param.strides, param.format);
#endif
}

// Capture exception callback, when there is an exception callback, capture will be interrupted.
void OnExceptionOccurred(ZegoScreenCaptureSource source,
                                                ZegoScreenCaptureSourceExceptionType exceptionType)
{

}

// Window capture state callback, when the window region position changes, it will be notified through this callback. When the window is not in the current screen region, capture will stop.
public void OnWindowStateChanged(ZegoScreenCaptureSource source,
                                                ZegoScreenCaptureWindowState windowState,
                                                ZegoRect windowRect)
{

}

// Capture region changes, used to update encoding resolution when resolution changes
public void OnRectChanged(ZegoScreenCaptureSource source, ZegoRect captureRect)
{

}

captureSource.onAvailableFrame = OnAvailableFrame;
captureSource.onExceptionOccurred = OnExceptionOccurred;
captureSource.onWindowStateChanged = OnWindowStateChanged;
captureSource.onRectChanged = OnRectChanged;

// Capture data callback, can be used for preview video rendering, local recording.
void OnAvailableFrame(ZegoScreenCaptureSource source, System.IntPtr data,
                                        uint dataLength, ZegoVideoFrameParam param)
{
#if (UNITY_STANDALONE_WIN || UNITY_STANDALONE_OSX)
    localVideoSurface.LoadNativeVideoRawData(data, param.width, param.height, param.strides, param.format);
#endif
}

// Capture exception callback, when there is an exception callback, capture will be interrupted.
void OnExceptionOccurred(ZegoScreenCaptureSource source,
                                                ZegoScreenCaptureSourceExceptionType exceptionType)
{

}

// Window capture state callback, when the window region position changes, it will be notified through this callback. When the window is not in the current screen region, capture will stop.
public void OnWindowStateChanged(ZegoScreenCaptureSource source,
                                                ZegoScreenCaptureWindowState windowState,
                                                ZegoRect windowRect)
{

}

// Capture region changes, used to update encoding resolution when resolution changes
public void OnRectChanged(ZegoScreenCaptureSource source, ZegoRect captureRect)
{

}

captureSource.onAvailableFrame = OnAvailableFrame;
captureSource.onExceptionOccurred = OnExceptionOccurred;
captureSource.onWindowStateChanged = OnWindowStateChanged;
captureSource.onRectChanged = OnRectChanged;

Call the LoginRoom interface, pass in the room ID parameter "roomID" and user parameter "user", to login to the room.

Call the StartPublishingStream interface, pass in the stream ID parameter "streamID", to send the local audio and video stream to remote users.

// Create user
ZegoUser user = new ZegoUser("user1");

// Start logging in to the room
engine.LoginRoom("room1", user);
// Start publishing stream
engine.StartPublishingStream("stream1");

// Create user
ZegoUser user = new ZegoUser("user1");

// Start logging in to the room
engine.LoginRoom("room1", user);
// Start publishing stream
engine.StartPublishingStream("stream1");

At this point, we have completed the operation of capturing screen data and sharing it to remote ends through ZEGO Express SDK.

After completing the above steps, other users can use the StartPlayingStream interface to play the screen sharing stream.

// Play stream, need to pass in the streamID used when the user initiating screen sharing published the stream
engine.StartPlayingStream(streamID);

// Play stream, need to pass in the streamID used when the user initiating screen sharing published the stream
engine.StartPlayingStream(streamID);

Call the StopScreenCapture interface to stop sharing.

engine.StopScreenCapture();

engine.StopScreenCapture();

FAQ

When an Android device is fixed in portrait mode (auto-rotation disabled), opening screen capture causes popup orientation abnormalities and interface landscape/portrait switching. How to solve this?

Due to the conflict between the device's fixed portrait mode and the App's landscape interface, rapid landscape/portrait switching during screen capture causes system data abnormalities and black screen. Solutions:

Solution 1: Override SDK configuration through tools:replace

The default orientation of the SDK screen sharing auxiliary Activity is "fullUser". You need to modify it in the project's AndroidManifest.xml:
1. Add the "tools" tool namespace to the root node (manifest tag);
2. Find the above Activity configuration, change its screen orientation (screenOrientation) to "unspecified", and add tools:replace to force override the SDK default configuration. After recompiling, the popup will be consistent with the App's landscape, avoiding switching black screens.
Solution 2: Enable device auto-rotation

Go to device "Settings > Display > Auto-rotate screen". After enabling, the popup orientation will follow the App's landscape without switching black screen issues.

Screen Sharing

Function Overview

Download Example Source Code

Prerequisites

Implementation Flow

1 Enable screen sharing

2 Login room and start publishing stream

3 Watch remote screen sharing

4 Stop screen sharing

FAQ