Live Streaming
  • iOS : Swift
  • Android
  • Web
  • Flutter
  • React Native
  • Electron
  • Unity3D
  • Windows
  • macOS
  • Linux
  • Overview
  • Live Streaming vs. Interactive Live Streaming
  • Develop your app
    • Live Streaming
      • Quick start
      • Enhance basic livestream
    • Interactive Live Streaming
  • Best practices
    • Implement co-hosting
    • Implement a live audio room
    • Implement PK battles
  • Upgrade the livestream
    • Advanced features
      • Enhance the livestream
        • Share the screen
  • Resources & Reference
    • SDK
    • Sample code
    • API reference
      • Client APIs
      • Server APIs
    • FAQs
    • Key concepts
  • Documentation
  • Live Streaming
  • Upgrade the livestream
  • Advanced features
  • Enhance the livestream
  • Share the screen

Share the screen

Last updated:2023-08-24 14:12

During the live streaming, the host can share his screen content with the viewers in the form of a video to enhance interaction and improve the live streaming experience.

This doc will introduce how to implement the screen sharing feature in the live streaming scenario.

Prerequisites

Before you begin, make sure you complete the following:

  • Complete SDK integration and basic live stream functions by referring to Quick Start doc.
  • Download the demo that comes with this doc.

Preview the effect

You can achieve the following effect with the demo provided in this doc:

Screen sharing not enabled Host is sharing the screen What viewers see when host switches apps

Understand the tech

1. You should already have some knowledge of streams when following the integration instructions in **Quick Start doc.

If you need a refresher, you can review it here.
  • ZEGO Express SDK: the real-time Audio and Video Call SDK provided by ZEGOCLOUD. It can help you provide audio and video services that feature convenient access, high-definition and fluency, cross-platform communication, low latency, and high concurrency.
  • Stream publishing: the process of publishing the audio and video data streams that are captured and packaged to ZEGOCLOUD real-time audio and video cloud.
  • Stream playing: the process of receiving and playing audio and video data streams from ZEGOCLOUD real-time audio and video cloud.
  • Room: the service provided by ZEGOCLOUD for organizing user groups and allowing users in the same room to receive and send real-time audio, video, and messages to each other.
    1. Users can publish or play streams only after logging in to a room.
    2. Users can receive notifications about changes (such as users joining or leaving a room, and audio and video stream changes) in the room where they are in.

2. What is a screen-sharing stream?

In the Express SDK, screen sharing is abstracted as a stream, just like the stream for the camera and microphone. This stream transmits the visual data of the shared screen.

3. How to publish multiple streams?

In the SDK's stream publishing interface startPublishingStream, there is an additional parameter ZegoPublishChannel, which defaults to ZegoPublishChannel.Main. This channel uses the device's camera and microphone as the audio and video source.

If you need to publish multiple streams, specify a ZegoPublishChannel when calling this interface and configure the audio and video source for that channel.

Upcoming chapters will cover how to create a screen sharing source and configure it to a channel.

  1. We recommend managing ZegoPublishChannel as follows:

    • Use ZegoPublishChannel.Main to share the host's camera and microphone (SDK configures the Main channel's audio and video sources to use the camera and microphone by default).
    • Use ZegoPublishChannel.Aux for screen sharing and manually configure its audio and video sources.
    • Use ZegoPublishChannel.Third and ZegoPublishChannel.Fourth for other audio and video sources.
  2. Regarding whether to publish multiple streams simultaneously:

    • For hosts, you can choose to "publish both screen-sharing and audio/video streams simultaneously" or "publish only one stream" based on your business needs.
    • For viewers, even if the host publishes multiple streams, you can choose to "play and render two streams simultaneously" or "play and render only one stream" based on your preferences.

Implementation

The following example shows how to implement screen sharing while publishing both screen-sharing and audio/video streams simultaneously.

1. Create a share extension

For the iOS platform, you need to create a Broadcast Upload Extension.

The memory limit of the Broadcast Upload Extension is 50 MB, make sure the memory usage of the Extension for screen sharing does not exceed 50 MB.

Create share extension
  1. Open your project in Xcode, and select File > New > Target.
    \
  1. In the following window, select the Broadcast Upload Extension, and click Next.

    \

  2. Fill in items or choose options for your new target, such as fill in the Product Name as ScreenShare, and choose options for the Team, Language, and other required information, and then click Finish.

    Don't need to check the Include UI Extension option.

    \

  3. After you created the Broadcast Upload Extension, you will see a folder for this Extension in your project with a structure similar to the following. This folder is used to store the implementation codes for the screen sharing feature:

    \

  4. Add the ZegoExpressEngine.xcframework dependency.

    • Click the + button as shown in the following figure.

      \

    • Select and add the ZegoExpressEngine.xcframework.

      \

    • Set Embed to Do Not Embed.

      \

  5. Set the Extension version

You need to set the Minimum Deployment Version of the Extension to be consistent with the application, but the minimum version of the Extension should be iOS 12.

\

  1. Replace the code in the SampleHandler file with the following code:
import ReplayKit
import ZegoExpressEngine

class SampleHandler: RPBroadcastSampleHandler {

    override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
        // User has requested to start the broadcast. Setup info from the UI extension can be supplied but optional.
        ZegoReplayKitExt.sharedInstance().setup(withDelegate: self)
    }

    override func broadcastPaused() {
        // User has requested to pause the broadcast. Samples will stop being delivered.
    }

    override func broadcastResumed() {
        // User has requested to resume the broadcast. Samples delivery will resume.
    }

    override func broadcastFinished() {
        // User has requested to finish the broadcast.
        ZegoReplayKitExt.sharedInstance().finished()
    }

    override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
        ZegoReplayKitExt.sharedInstance().send(sampleBuffer, with: sampleBufferType)
    }
}

extension SampleHandler: ZegoReplayKitExtHandler {
    func broadcastFinished(_ broadcast: ZegoReplayKitExt, reason: ZegoReplayKitExtReason) {
        switch reason {
        case .hostStop:
            let userInfo = [NSLocalizedDescriptionKey: "Host app stop srceen capture"]
            let error = NSError(domain: NSCocoaErrorDomain, code: 0, userInfo: userInfo)
            finishBroadcastWithError(error)
        case .connectFail:
            let userInfo = [NSLocalizedDescriptionKey: "Connect host app fail need startScreenCapture in host app"]
            let error = NSError(domain: NSCocoaErrorDomain, code: 0, userInfo: userInfo)
            finishBroadcastWithError(error)
        case .disconnect:
            let userInfo = [NSLocalizedDescriptionKey: "disconnect with host app"]
            let error = NSError(domain: NSCocoaErrorDomain, code: 0, userInfo: userInfo)
            finishBroadcastWithError(error)
        @unknown default:
            break
        }
    }
}

2. Set video source and video config with PublishChannel.

For example, to use ZegoPublishChannel.Aux:

ZegoExpressEngine.shared().setVideoSource(.screenCapture, channel: .aux)

let videoConfig = ZegoVideoConfig(preset: .preset720P)
videoConfig.fps = 10
ZegoExpressEngine.shared().setVideoConfig(videoConfig, channel: .aux)

3. Start the screen sharing

Start screen sharing using the startScreenCapture, and when calling startPublishingStream, specify ZegoPublishChannel.Aux as the ZegoPublishChannel used for screen sharing:

let config = ZegoScreenCaptureConfig()
config.captureAudio = true
config.captureVideo = true
ZegoExpressEngine.shared().startScreenCapture(config)

let streamID = "\(roomID)_\(userID)_screen"
ZegoExpressEngine.shared().startPublishingStream(streamID, channel: .aux)
Usually, previewing your own screen-sharing stream is not necessary. If you need to preview it in your scenario, create a `CanvasView` and use it for previewing.
let canvas = ZegoCanvas(view: streamView)
ZegoExpressEngine.shared().startPreview(canvas, channel: .aux)

4. Stop the screen sharing

Stop screen sharing using the stopScreenCapture, and when calling stopPublishingStream, specify ZegoPublishChannel.Aux as the ZegoPublishChannel used for screen sharing:

ZegoExpressEngine.shared().stopScreenCapture()
ZegoExpressEngine.shared().stopPublishingStream(.aux)

5. How do viewers render the shared screen?

For viewers, the rendering method for screen-sharing streams is the same as that for normal camera streams.

In the demo, stream types are distinguished by stream name (see the FAQs at the end of the document). Therefore, you can determine the stream type in this way when playing streams and rendering different streams to different positions on the screen.

func startPlayingScreenShareStream(_ streamID: String, view: UIView) {
    let canvas = ZegoCanvas(view: view)
    ZegoExpressEngine.shared().startPlayingStream(streamID, canvas: canvas)
}

FAQs

1. How to distinguish between the screen-sharing stream and the audio/video stream?

In general, the type of stream can be agreed upon through the construction rules of stream ID, which is suitable for most scenarios:

  1. You can configure the stream ID of the camera and microphone on ZegoPublishChannel.Main as "\(roomID)_\(userID)_live", and configure the stream ID of screen sharing on ZegoPublishChannel.Aux as "\(roomID)_\(userID)_screen".

  2. When the callback on the stream added is received by the stream players, the stream type can be judged according to the stream ID rule.

  3. The server callback can also judge the stream type according to the same rule.

If using stream ID to identify stream type is not suitable, you can use stream attachment information to mark stream type. See setStreamExtraInfo method for details.

2. How to use only the microphone and share the screen (without using the camera) when the business requires it?

In this case, publishing multiple streams is not required. You just need to replace the video source of the main channel with the screen sharing source:

  1. Set video source and configure it to ZegoPublishChannel.Main, like this:
ZegoExpressEngine.shared().setVideoSource(.screenCapture, channel: .main)

let videoConfig = ZegoVideoConfig(preset: .preset720P)
videoConfig.fps = 10
ZegoExpressEngine.shared().setVideoConfig(videoConfig, channel: .main)

This way, the ZegoPublishChannel.Main channel will use screen sharing as the video source and use the microphone as the audio source.

  1. Use ZegoPublishChannel.Main as the channel parameter in the following steps. Like this:
ZegoExpressEngine.shared().startPublishingStream(streamID, channel: .main);
ZegoExpressEngine.shared().stopPublishingStream(.main);

Conclusion

Congratulations! Hereby you have completed the development of the screen sharing feature.

If you have any suggestions or comments, feel free to share them with us via Discord. We value your feedback.

Page Directory