Call Quality Monitoring
Function Overview
During calls using ZEGO Express SDK, users may sometimes experience poor network conditions. At this time, you can understand the current call's network quality and audio/video information changes through relevant callbacks.
For example, during multi-person audio/video calls or multi-person singing, we need to display users' network quality in real time. You can refer to this document to implement corresponding functions.
This function is not supported in WebGL environments.
Example Source Code
Please refer to Download Example Source Code to get the source code.
For related source code, please view files in the "Assets/ZegoExpressExample/Examples/AdvancedStreaming/StreamMonitoring" directory.
Prerequisites
Before monitoring call quality, please ensure:
- You have created a project in ZEGOCLOUD Console and applied for valid AppID and AppSign. For details, please refer to Console - Project Information.
- You have integrated ZEGO Express SDK in the project and implemented basic audio and video publishing and playing functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation Flow.
Basic Network Quality Report
You can receive the uplink and downlink network quality of users in the room (including yourself) by listening to the OnNetworkQuality callback. This callback is received every two seconds. For network quality levels, please refer to ZegoStreamQualityLevel.
Different versions of OnNetworkQuality callback logic differ:
| Version Number | Callback Logic |
|---|---|
| 2.22.0 and above | Based on the onNetworkQuality interface callback logic of version 2.14.0 ~ 2.21.1, it can also estimate the network situation of remote publishing users. If the remote publishing user's heartbeat is lost once, the callback network quality is unknown; if the remote publishing user's heartbeat is lost 3 times, the callback network quality is die. |
| 2.14.0 ~ 2.21.1 |
|
| 2.10.0 ~ 2.13.1 |
|
OnNetworkQuality is not suitable for scenarios using CDN for live streaming. You can refer to Advanced Quality Report - Publishing Stream Quality Report to monitor CDN publishing stream quality.
void OnNetworkQuality(string userID, ZegoStreamQualityLevel upstreamQuality, ZegoStreamQualityLevel downstreamQuality)
{
// Developers can monitor users' uplink and downlink network quality in this callback to report to business server for monitoring, or give user-friendly hints
if (userID == "") {
// Represents local user's (local) network quality
} else {
//Represents other users' network quality in the room
}
/*
ZegoStreamQualityLevel.Excellent Network quality excellent
ZegoStreamQualityLevel.Good Network quality good
ZegoStreamQualityLevel.Medium Network quality normal
ZegoStreamQualityLevel.Bad Network quality poor
ZegoStreamQualityLevel.Die Network abnormal
ZegoStreamQualityLevel.Unknown Network quality unknown
*/
}
engine.onNetworkQuality = OnNetworkQuality;Advanced Quality Report
If the above basic network quality report cannot meet your needs, ZEGO also provides more detailed publishing stream quality reports, playing stream quality reports, and other related information.
Publishing Stream Quality Report
Publishing stream quality report refers to the quality report describing the process of users pushing audio and video to ZEGO server, including the frame rate of audio and video streams in the capture and encoding stages, and the frame rate, bitrate, latency, and packet loss rate of the audio and video streams in transmission (sending).
You can receive publishing stream quality callbacks by registering OnPublisherQualityUpdate. After publishing stream is successful, this callback will be received every three seconds. You can understand the health status of the published audio and video stream in real time based on the quality(ZegoPublishStreamQuality) parameter.
- In most cases, you only need to pay attention to the "level" parameter of "quality", and judge the comprehensive quality of publishing stream by the "level" enumeration value. For details, please refer to ZegoStreamQualityLevel.
- If you want to pay attention to more detailed publishing stream quality parameters, please refer to ZegoPublishStreamQuality.
void OnPublisherQualityUpdate(string streamID, ZegoPublishStreamQuality quality){
// Developers can monitor specific quality in this callback to report to business server for monitoring, or monitor a certain field of the quality object to give user-friendly hints
// If developers don't know which field of monitoring quality to focus on, they can focus on the level field, which is the comprehensive value of the quality object
switch (quality.level) {
case ZegoStreamQualityLevel.Excellent:
// Excellent
break;
case ZegoStreamQualityLevel.Good:
// Good
break;
case ZegoStreamQualityLevel.Medium:
// Medium
break;
case ZegoStreamQualityLevel.Bad:
// Poor
break;
case ZegoStreamQualityLevel.Die:
// Failed
break;
case ZegoStreamQualityLevel.Unknown:
// Unknown
break;
default:
break;
}
}
engine.onPublisherQualityUpdate = OnPublisherQualityUpdate;Playing Stream Quality Report
Playing stream quality report refers to the quality report of the process where users play audio and video streams, including the frame rate, bitrate, latency, and packet loss rate of the received audio and video streams, the frame rate of the audio and video streams in the decoding stage, and the frame rate, freeze rate, and overall audio and video quality in the rendering stage.
You can receive playing stream quality callbacks by registering OnPlayerQualityUpdate. After playing stream is successful, this callback will be received every three seconds. Developers can understand the health status of the played audio and video stream in real time based on the quality(ZegoPlayStreamQuality) parameter.
- In most cases, you only need to pay attention to the "level" parameter of "quality", and judge the comprehensive quality of playing stream by the "level" enumeration value. For details, please refer to ZegoStreamQualityLevel.
- If you want to pay attention to more detailed playing stream quality parameters, please refer to ZegoPlayStreamQuality.
void OnPlayerQualityUpdate(string streamID, ZegoPlayStreamQuality quality)
{
// Developers can monitor specific quality in this callback to report to business server for monitoring, or monitor a certain field of the quality object to give user-friendly hints
// If developers don't know which field of monitoring quality to focus on, they can focus on the level field, which is the comprehensive value of the quality object
}
engine.onPlayerQualityUpdate = OnPlayerQualityUpdate;MOS Audio Quality Score
Starting from ZEGO Express SDK version 2.16.0, the "mos" field is added to the playing stream quality callback OnPlayerQualityUpdate, representing the score of playing stream audio quality. When developers are more concerned about audio quality, they can understand the current audio quality situation through this field.
The value range of the mos field is [-1, 5], where -1 means unknown (for example, cannot score when playing stream abnormally), and [0, 5] means the normal scoring range. The corresponding subjective audio quality perception of real-time audio MOS scoring is as follows:
| MOS Value | Evaluation Criteria |
|---|---|
| 4.0 ~ 5.0 | Audio quality excellent, clear and smooth, can hear clearly. |
| 3.5 ~ 4.0 | Audio quality good, occasional audio quality damage, but still clear and smooth, can hear clearly. |
| 3.0 ~ 3.5 | Audio quality medium, occasional freezes, need some attention to hear clearly. |
| 2.5 ~ 3.0 | Audio quality poor, frequent freezes, need concentrated attention to hear clearly. |
| 2.0 ~ 2.5 | Audio quality very poor, partial semantic loss, difficult to communicate. |
| 0 ~ 2.0 | Audio quality extremely poor, massive semantic loss, unable to communicate. |
| -1 | Unknown. |
Other Information Monitoring
Publishing/Playing Stream Status Change Notification
Publishing Stream Status Callback
After publishing stream is successful, you can get notifications of publishing stream status changes through OnPublisherStateUpdate.
void OnPublisherStateUpdate(string streamID, ZegoPublisherState state, int errorCode, string extendedData)
{
// When state is ZegoPublisherState.NoPublish, and errcode is non-zero, it indicates publishing stream failed, and will not retry publishing stream anymore. At this time, you can make a publishing stream failure hint on the interface;
// When state is ZegoPublisherState.PublishRequesting, and errcode is non-zero, it indicates retrying publishing stream. At this time, if publishing stream is not successful within the retry time, a publishing stream failure notification will be thrown.
}
engine.onPublisherStateUpdate = OnPublisherStateUpdate;You can roughly judge the user's publishing stream network situation based on whether the "state" parameter in the callback is in "requesting publishing stream status". The values of the "state" parameter correspond to the user's publishing stream status as follows:
| Enumeration Value | Description |
|---|---|
| ZegoPublisherState.NoPublish | No publishing stream status, in this state before publishing stream. If a steady-state exception occurs during the publishing stream process, such as incorrect AppID, AppSign, or Token, or if other users are already publishing streams, publishing streams with the same stream ID will fail, all will enter the no publishing stream status. |
| ZegoPublisherState.PublishRequesting | Requesting publishing stream status. After the publishing stream operation is successfully executed, it will enter the requesting publishing stream status, usually this state is used for UI interface display. If an interruption occurs due to poor network quality, the SDK will internally retry and will also return to the requesting publishing stream status. |
| ZegoPublisherState.Publishing | Publishing stream status, entering this state indicates that publishing stream has been successful and users can communicate normally. |
The parameter "extendedData" is extended information attached to the status update. If using ZEGO's CDN content distribution network, after publishing stream is successful, the keys of the content of this parameter are "flv_url_list", "rtmp_url_list", "hls_url_list", corresponding to the playing stream URLs of flv, rtmp, and hls protocols respectively.
Playing Stream Status Change Callback
After playing stream is successful, developers can get notifications of playing stream status changes through OnPlayerStateUpdate.
void OnPlayerStateUpdate(string streamID, ZegoPlayerState state, int errorCode, string extendedData)
{
// When state is ZegoPlayerState.NoPlay, and errcode is non-zero, it indicates playing stream failed, and will not retry playing stream anymore. At this time, you can make a playing stream failure hint on the interface;
// When state is ZegoPlayerState.PlayRequesting, and errcode is non-zero, it indicates retrying playing stream. At this time, if playing stream is not successful within the retry time, a playing stream failure notification will be thrown.
}
engine.onPlayerStateUpdate = OnPlayerStateUpdate;Developers can roughly judge the user's playing stream network situation based on whether the "state" parameter is in "requesting playing stream status". The values of the "state" parameter correspond to the user's playing stream status as follows:
| Enumeration Value | Description |
|---|---|
| ZegoPlayerState.NoPlay | No playing stream status, in this state before playing stream. If a steady-state exception occurs during the playing stream process, such as incorrect AppID, AppSign, or Token, all will enter the no playing stream status. |
| ZegoPlayerState.PlayRequesting | Requesting playing stream status. After the playing stream operation is successfully executed, it will enter the requesting playing stream status, usually this state is used for application interface display. If an interruption occurs due to poor network quality, the SDK will internally retry and will also return to the requesting playing stream status. |
| ZegoPlayerState.Playing | Playing stream status, entering this state indicates that playing stream has been successful and users can communicate normally. |
Notification of Receiving Audio/Video First Frame
Publishing Side Audio Capture First Frame Callback
You can receive audio first frame callbacks by registering OnPublisherCapturedAudioFirstFrame. After calling the publishing stream interface successfully, this callback will be received when the SDK captures the first frame of audio data.
When not publishing stream or not previewing, for the first publishing stream or first preview, that is, when the engine of the audio and video module inside the SDK starts, it will capture the local device's audio data and this callback will be received. Developers can judge whether the SDK really captured audio data based on this callback. If this callback is not received, it means the audio capture device is occupied or abnormal.
void OnPublisherCapturedAudioFirstFrame()
{
}
engine.onPublisherCapturedAudioFirstFrame = OnPublisherCapturedAudioFirstFrame;Publishing Side Video Capture First Frame Callback
You can receive video first frame callbacks by registering OnPublisherCapturedVideoFirstFrame. After calling the publishing stream interface successfully, this callback will be received when the SDK captures the first frame of video data.
When not publishing stream or not previewing, for the first publishing stream or first preview, that is, when the engine of the audio and video module inside the SDK starts, it will capture the local device's video data and this callback will be received. You can judge whether the SDK really captured video data based on this callback. If this callback is not received, it means the video capture device is occupied or abnormal.
void OnPublisherCapturedVideoFirstFrame(ZegoPublishChannel channel)
{
}
engine.onPublisherCapturedVideoFirstFrame = OnPublisherCapturedVideoFirstFrame;Playing Side Audio Receive First Frame Callback
Developers can listen to playing side audio receive first frame callbacks by registering OnPlayerRecvAudioFirstFrame. After calling the playing stream interface successfully, this callback will be received when the SDK plays stream and gets the first frame of audio data.
void OnPlayerRecvAudioFirstFrame(string streamID)
{
}
engine.onPlayerRecvAudioFirstFrame = OnPlayerRecvAudioFirstFrame;Playing Side Video Receive First Frame Callback
You can listen to playing side video receive first frame callbacks by registering OnPlayerRecvVideoFirstFrame. After calling the playing stream interface successfully, this callback will be received when the SDK plays stream and gets the first frame of video data.
void OnPlayerRecvVideoFirstFrame(string streamID)
{
}
engine.onPlayerRecvVideoFirstFrame = OnPlayerRecvVideoFirstFrame;Playing Side Rendered Video First Frame Callback
You can listen to playing side rendered video first frame callbacks by registering OnPlayerRenderVideoFirstFrame. After calling the playing stream interface successfully, this callback will be received when the SDK plays stream and renders the first frame of video data.
You can use this callback to calculate first frame time consumption or update the playing stream UI component.
void OnPlayerRenderVideoFirstFrame(string streamID)
{
}
engine.onPlayerRenderVideoFirstFrame = OnPlayerRenderVideoFirstFrame;Video Resolution Change Callback
Capture Video Resolution Change Callback
You can listen to capture video size change callbacks by registering OnPublisherVideoSizeChanged. After publishing stream is successful, if the video capture resolution changes during publishing stream, this callback will be received.
When not publishing stream or not previewing, for the first publishing stream or first preview, that is, when the engine of the audio and video module inside the SDK starts, it will capture the local device's video data, and the capture resolution will change at this time.
You can use this callback to remove cover operations from local preview UI and similar operations. You can also dynamically adjust the preview view ratio based on the resolution of this callback.
void OnPublisherVideoSizeChanged(int width, int height, ZegoPublishChannel channel)
{
}
engine.onPublisherVideoSizeChanged = OnPublisherVideoSizeChanged;Playing Stream Resolution Change Notification
You can get playing stream resolution change notifications by registering OnPlayerVideoSizeChanged. After playing stream is successful, if the video resolution changes during playing stream, this callback will be received, and users can adjust the display based on the final resolution of the stream.
- If the played stream only has audio data, this callback will not be received.
- If the publishing side triggers SDK internal flow control due to network issues, it may dynamically reduce the encoding resolution of the publishing side, and this callback will also be received at this time. When the played audio and video stream is really rendered to the set UI playing interface, this callback will be triggered. Developers can use this callback notification to update or switch the UI component that really plays the stream.
void OnPlayerVideoSizeChanged(string streamID, int width, int height)
{
}
engine.onPlayerVideoSizeChanged = OnPlayerVideoSizeChanged;