Implementing Live Streaming

This article will introduce how to quickly implement a simple video live streaming through the live streaming function.

Introduction

Explanation of related concepts:

• ZEGO Express SDK: Real-time audio and video and live streaming SDK provided by ZEGO, which can provide developers with convenient access, high definition and smoothness, multi-platform interoperability, low latency, and high concurrency audio and video services.
• Stream: A group of audio and video data that is continuously sent and encapsulated in a specified encoding format. A user can push multiple streams simultaneously (for example, one pushes camera data and one pushes screen sharing data) and can also pull multiple streams simultaneously. Each stream is identified by a stream ID (streamID).
• Publish stream: The process of pushing the audio and video data stream packaged in the capture phase to the ZEGOCLOUD.
• Play stream: The process of pulling and playing existing audio and video streams from the ZEGO MSDN network.
• Room: An audio and video space service provided by ZEGO, used to organize user groups. Users in the same room can send and receive real-time audio and video and messages to each other.
  • Users need to log in to a room first before they can perform publish and play stream operations.
  • Users can only receive relevant messages in the room they are in (user entry/exit, audio and video stream changes, etc.).
  • Each room is identified by a unique roomID within an AppID. All users who log in to the room using the same roomID belong to the same room.

Prerequisites

Before implementing basic live streaming functions, please make sure:

• ZEGO Express SDK has been integrated in the project. For details, please refer to Quick Start - Integration.
• A project has been created in the ZEGOCLOUD Console, and a valid AppID has been applied for.

Implementation Process

The basic process for users to perform video live streaming through ZEGO Express SDK is:

User A and B join the room. User B previews and pushes the audio and video stream to the ZEGO cloud service (publish stream). After user A receives the notification of user B's pushed audio and video stream, user A plays user B's audio and video stream in the notification (play stream).

Enable Service

The live streaming function is not enabled by default. Please enable it by yourself in the ZEGOCLOUD Console before use (for enabling steps, please refer to Project Management - Service Configuration in "Live Streaming"), or contact ZEGOCLOUD Technical Support to enable it.

Initialization

1. Create UI (Optional)

Add UI Elements

Before starting, it is recommended that developers add the following UI elements to facilitate the implementation of basic real-time audio and video functions.

• Local preview window
• Remote video window
• End button

2. Create Engine

Call the CreateEngine interface, pass the applied AppID and AppSign into the parameters "appId" and "appSign", and create an engine singleton object.

Developers can choose to use anonymous functions to implement callback functions as needed, and assign them to the corresponding callback delegates of the engine instance to register callbacks.

// Define SDK engine object
ZegoExpressEngine engine;

ZegoEngineProfile profile = new ZegoEngineProfile();
profile.appID = appID; // Please obtain through official website registration, the format is 123456789
profile.appSign = appSign; // Please obtain through official website registration, the format is "0123456789012345678901234567890123456789012345678901234567890123", 64 characters
profile.scenario = ZegoScenario.HighQualityVideoCall; // High-quality audio and video call scenario access (please select the appropriate scenario according to the actual situation)
// Initialize SDK
engine = ZegoExpressEngine.CreateEngine(profile);


// Set SDK callback delegate
engine.OnRoomStateUpdate = (roomID, state, errorCode, extendedData) => {

};

// Define SDK engine object
ZegoExpressEngine engine;

ZegoEngineProfile profile = new ZegoEngineProfile();
profile.appID = appID; // Please obtain through official website registration, the format is 123456789
profile.appSign = appSign; // Please obtain through official website registration, the format is "0123456789012345678901234567890123456789012345678901234567890123", 64 characters
profile.scenario = ZegoScenario.HighQualityVideoCall; // High-quality audio and video call scenario access (please select the appropriate scenario according to the actual situation)
// Initialize SDK
engine = ZegoExpressEngine.CreateEngine(profile);


// Set SDK callback delegate
engine.OnRoomStateUpdate = (roomID, state, errorCode, extendedData) => {

};

void OnRoomStateChanged(string roomID, ZegoRoomStateChangedReason reason, int errorCode, string extendedData) {
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onRoomStateChanged = OnRoomStateChanged;

void OnRoomStateChanged(string roomID, ZegoRoomStateChangedReason reason, int errorCode, string extendedData) {
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onRoomStateChanged = OnRoomStateChanged;

void OnRoomUserUpdate(string roomID, ZegoUpdateType updateType, List<ZegoUser> userList, uint userCount)
{
    // You can handle corresponding business logic based on user entry/exit situations in the callback
    if (updateType == ZegoUpdateType.Add)
    {
        userList.ForEach((user)=>{
            //string.Format("user {0} enter room {1}", user.userID, roomID);
        });
    }
    else
    {
        userList.ForEach((user)=>{
            //string.Format("user {0} exit room {1}", user.userID, roomID);
        });
    }
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onRoomUserUpdate = OnRoomUserUpdate;

void OnRoomUserUpdate(string roomID, ZegoUpdateType updateType, List<ZegoUser> userList, uint userCount)
{
    // You can handle corresponding business logic based on user entry/exit situations in the callback
    if (updateType == ZegoUpdateType.Add)
    {
        userList.ForEach((user)=>{
            //string.Format("user {0} enter room {1}", user.userID, roomID);
        });
    }
    else
    {
        userList.ForEach((user)=>{
            //string.Format("user {0} exit room {1}", user.userID, roomID);
        });
    }
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onRoomUserUpdate = OnRoomUserUpdate;

void OnPublisherStateUpdate(string streamID, ZegoPublisherState state, int errorCode, string extendedData)
{
    // Implement event callback as needed
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onPublisherStateUpdate = OnPublisherStateUpdate;

void OnPublisherStateUpdate(string streamID, ZegoPublisherState state, int errorCode, string extendedData)
{
    // Implement event callback as needed
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onPublisherStateUpdate = OnPublisherStateUpdate;

void OnPlayerStateUpdate(string streamID, ZegoPlayerState state, int errorCode, string extendedData)
{
    // Implement event callback as needed
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onPlayerStateUpdate = OnPlayerStateUpdate;

void OnPlayerStateUpdate(string streamID, ZegoPlayerState state, int errorCode, string extendedData)
{
    // Implement event callback as needed
}

// Assign to the corresponding event delegate of ZegoExpressEngine
engine.onPlayerStateUpdate = OnPlayerStateUpdate;

Login Room

Create a ZegoUser user object, set user information "userID" and "userName", then call LoginRoom, pass in the room ID parameter "roomId" and user parameter "user", and log in to the room. If the room does not exist, calling this interface will create and log in to this room.

• Within the same AppID, ensure that "roomId" is globally unique.
• Within the same AppID, ensure that "userId" is globally unique. It is recommended that developers set it to a meaningful value and associate "userId" with their business account system.
• "userId" and "userName" cannot be empty, otherwise logging in to the room will fail.
• When running in the WebGL environment, only Token authentication is supported. Token cannot be empty. Please refer to Use Token for Authentication.

// Create user
ZegoUser user = new ZegoUser();
user.userId="xxx";
user.userName="xxxx";
// Only by passing in a ZegoRoomConfig with "isUserStatusNotify" parameter set to "true" can you receive the onRoomUserUpdate callback.
ZegoRoomConfig roomConfig = new ZegoRoomConfig();
// If you use AppSign for authentication, the Token parameter does not need to be filled in (except for the WebGL platform); if you need to use a more secure authentication method: Token authentication, please refer to [How to upgrade from AppSign authentication to Token authentication](https://www.zegocloud.com/docs/faq/express-token-upgrade)
// On the WebGL platform, "token" cannot be empty
roomConfig.token = "xxxx";
roomConfig.isUserStatusNotify = true;
// Log in to room
engine.LoginRoom("123666", user, roomConfig);

// Create user
ZegoUser user = new ZegoUser();
user.userId="xxx";
user.userName="xxxx";
// Only by passing in a ZegoRoomConfig with "isUserStatusNotify" parameter set to "true" can you receive the onRoomUserUpdate callback.
ZegoRoomConfig roomConfig = new ZegoRoomConfig();
// If you use AppSign for authentication, the Token parameter does not need to be filled in (except for the WebGL platform); if you need to use a more secure authentication method: Token authentication, please refer to [How to upgrade from AppSign authentication to Token authentication](https://www.zegocloud.com/docs/faq/express-token-upgrade)
// On the WebGL platform, "token" cannot be empty
roomConfig.token = "xxxx";
roomConfig.isUserStatusNotify = true;
// Log in to room
engine.LoginRoom("123666", user, roomConfig);

Host preview their own screen and push to ZEGO audio and video cloud

1. Host preview their own screen

2. Host push their audio and video stream to ZEGO audio and video cloud

After the user calls the LoginRoom interface, they can directly call the StartPublishingStream interface and pass in "streamID" to push their audio and video stream to the ZEGO audio and video cloud. You can know whether the push streaming is successful by listening to the OnPublisherStateUpdate callback.

"streamID" is generated locally by you, but needs to ensure:

Under the same AppID, "streamID" is globally unique. If under the same AppID, different users each push a stream with the same "streamID", the user who pushes the stream later will fail to push the stream.

The example here pushes the stream immediately after calling the LoginRoom interface. When implementing specific business, you can choose other timings to push the stream, as long as you call LoginRoom first.

// After the user calls loginRoom, call this interface to push the stream
// Under the same AppID, developers need to ensure that "streamID" is globally unique. If different users each push a stream with the same "streamID", the user who pushes the stream later will fail to push the stream.
engine.StartPublishingStream("stream1");

// After the user calls loginRoom, call this interface to push the stream
// Under the same AppID, developers need to ensure that "streamID" is globally unique. If different users each push a stream with the same "streamID", the user who pushes the stream later will fail to push the stream.
engine.StartPublishingStream("stream1");

Play host's audio and video

When live streaming, we need to play the host's audio and video. The latency of live streaming playing is within 1s, which can achieve the ultimate live streaming experience of ultra-low latency, strong synchronization, extreme weak network resistance, ultra-low stuttering, ultra-clear picture quality, and instant first frame.

When other users in the same room push audio and video streams to the ZEGO audio and video cloud, we will receive notifications of new audio and video streams in the OnRoomStreamUpdate callback, and can obtain the "streamID" of a certain stream through ZegoStream.

In this callback, we can call the StartPlayingStream interface and pass in "streamID" to play the user's audio and video. You can know whether the audio and video are successfully played by listening to the OnPlayerStateUpdate callback. You can use the following playing methods.

Call the StartPlayingStream interface and set the resourceMode parameter to "ZegoScenePlayerConfig.ZegoStreamResourceModeOnlyL3", indicating live streaming playing.

Unity3D supports playing rendering through two renderers: RawImage and Renderer.

• Method 1: RawImage

  1. Create RawImage.
  
  2. Preview displays on RawImage.

  private const float Offset = 100;
  
  GameObject go = new GameObject();
  
  if (go == null)
  {
      return;
  }
  
  go.name = "play-rawimage";
  // Dynamically create RawImage
  go.AddComponent<RawImage>();
  GameObject canvas = GameObject.Find("Canvas");
  if (canvas != null)
  {
      go.transform.parent = canvas.transform;
  }
  float xPos = UnityEngine.Random.Range(Offset - Screen.width / 2f, Screen.width / 2f - Offset);
  float yPos = UnityEngine.Random.Range(Offset, Screen.height / 2f - Offset);
  go.transform.localPosition = new Vector3(xPos, yPos, 0f);
  go.transform.localScale = new Vector3(3f, 4f, 1f);
  
  // Add a RawImageVideoSurface component, renderer is RawImage
  RawImageVideoSurface videoSurface = go.AddComponent<RawImageVideoSurface>();
  // Set the video stream ID to display
  videoSurface.SetPlayVideoInfo("123");
  // Set video source to engine
  videoSurface.SetVideoSource(engine);
  
  // Call playing stream interface
  ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
  playerConfig.resourceMode = ZegoStreamResourceMode.OnlyL3;
  engine.StartPlayingStream(streamID, playerConfig);

  private const float Offset = 100;
  
  GameObject go = new GameObject();
  
  if (go == null)
  {
      return;
  }
  
  go.name = "play-rawimage";
  // Dynamically create RawImage
  go.AddComponent<RawImage>();
  GameObject canvas = GameObject.Find("Canvas");
  if (canvas != null)
  {
      go.transform.parent = canvas.transform;
  }
  float xPos = UnityEngine.Random.Range(Offset - Screen.width / 2f, Screen.width / 2f - Offset);
  float yPos = UnityEngine.Random.Range(Offset, Screen.height / 2f - Offset);
  go.transform.localPosition = new Vector3(xPos, yPos, 0f);
  go.transform.localScale = new Vector3(3f, 4f, 1f);
  
  // Add a RawImageVideoSurface component, renderer is RawImage
  RawImageVideoSurface videoSurface = go.AddComponent<RawImageVideoSurface>();
  // Set the video stream ID to display
  videoSurface.SetPlayVideoInfo("123");
  // Set video source to engine
  videoSurface.SetVideoSource(engine);
  
  // Call playing stream interface
  ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
  playerConfig.resourceMode = ZegoStreamResourceMode.OnlyL3;
  engine.StartPlayingStream(streamID, playerConfig);

• Method 2: Renderer

  1. Create a Plane with Renderer.
  
  2. Preview to the Plane's Renderer.

  remoteVideoPlane = GameObject.Find("PlayRender");
  if (remoteVideoPlane != null)
  {
      if (remoteVideoSurface == null)
      {
          // Add a RendererVideoSurface component, renderer is Renderer
          remoteVideoSurface = remoteVideoPlane.AddComponent<RendererVideoSurface>();
          // Ensure the screen can be displayed
          remoteVideoSurface.transform.Rotate(90.0f, -180.0f, 0.0f);
          remoteVideoSurface.transform.position = new Vector3(0.0f, 0.0f, 0.0f);
          remoteVideoSurface.transform.localScale = new Vector3(0.36f, 1f, 0.64f);
      }
      if (remoteVideoSurface != null)
      {
          // Set the video stream ID to display
          remoteVideoSurface.SetPlayVideoInfo("123");
          // Set video source to engine
          remoteVideoSurface.SetVideoSource(engine);
      }
  }
  
  // Call playing stream interface
  ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
  playerConfig.resourceMode = ZegoStreamResourceMode.OnlyL3;
  engine.StartPlayingStream(streamID, playerConfig);

  remoteVideoPlane = GameObject.Find("PlayRender");
  if (remoteVideoPlane != null)
  {
      if (remoteVideoSurface == null)
      {
          // Add a RendererVideoSurface component, renderer is Renderer
          remoteVideoSurface = remoteVideoPlane.AddComponent<RendererVideoSurface>();
          // Ensure the screen can be displayed
          remoteVideoSurface.transform.Rotate(90.0f, -180.0f, 0.0f);
          remoteVideoSurface.transform.position = new Vector3(0.0f, 0.0f, 0.0f);
          remoteVideoSurface.transform.localScale = new Vector3(0.36f, 1f, 0.64f);
      }
      if (remoteVideoSurface != null)
      {
          // Set the video stream ID to display
          remoteVideoSurface.SetPlayVideoInfo("123");
          // Set video source to engine
          remoteVideoSurface.SetVideoSource(engine);
      }
  }
  
  // Call playing stream interface
  ZegoPlayerConfig playerConfig = new ZegoPlayerConfig();
  playerConfig.resourceMode = ZegoStreamResourceMode.OnlyL3;
  engine.StartPlayingStream(streamID, playerConfig);

Debug live streaming function

Run the project on a real device. After successful operation, you can see the local video screen.

For convenience, ZEGO provides a Web platform for debugging. On the debugging page, you can enter the AppID and room ID of the real device user, and a different user ID to log in to the same room for communicating with the real device user. After a video call starts successfully, you can hear the remote audio and view the remote video.

Stop publishing/playing audio and video streams

1. Stop publishing stream, stop preview

Call the StopPublishingStream interface to stop sending local audio and video streams to remote users.

// Stop publishing stream
engine.StopPublishingStream();

// Stop publishing stream
engine.StopPublishingStream();

If local preview is enabled, call the StopPreview interface to stop preview.

// Stop local preview
engine.StopPreview();

// Stop local preview
engine.StopPreview();

2. Stop playing stream

Call the StopPlayingStream interface to stop playing remote pushed audio and video streams.

// Stop playing stream
engine.StopPlayingStream("stream1");

// Stop playing stream
engine.StopPlayingStream("stream1");

Logout Room

Call the LogoutRoom interface to log out of the room.

// Logout room
engine.LogoutRoom();

// Logout room
engine.LogoutRoom();

Destroy Engine

If users completely stop using audio and video functions, they can call the DestroyEngine interface to destroy the engine and release resources such as microphones, cameras, memory, and CPU.

• If you need to listen for callbacks to ensure that device hardware resources are released, you can pass in "callback" when destroying the engine. This callback is only used to send notifications, and developers cannot release engine-related resources in the callback.
• If you don't need to listen for callbacks, you can pass in "null".

ZegoExpressEngine.DestroyEngine();
engine = null;

ZegoExpressEngine.DestroyEngine();
engine = null;

Live Streaming API Call Sequence

FAQ

1. Can I kill the process directly when calling LogoutRoom to log out of the room?

After calling LogoutRoom, killing the process directly has a certain probability that the LogoutRoom command will not be sent out. Then the ZEGO server can only consider that this user has left the room after waiting for the heartbeat timeout. To ensure that the LogoutRoom command is sent out, it is recommended to call DestroyEngine again and receive the callback before killing the process.

Related Documentation

• Common Error Codes

• How to handle room and user related issues?

• How to set and get SDK logs and stack information?

• Does the SDK support automatic reconnection?

• In live streaming scenarios, how to listen for remote viewer role users logging in/logout of rooms?

• How to adjust the camera focal length (zoom function)?

• Why can't I open the camera?

• How to ensure smooth audio and video in poor network environments (flow control function)?