Call Quality Monitoring
Introduction
When using ZEGO Express SDK for calls, users may sometimes experience poor network conditions. In such cases, you can understand the current call's network quality and changes in audio/video information through relevant callbacks.
For example, during multi-person audio/video calls or multi-person singing, we need to display the user's network quality in real time, so you can refer to this document to implement corresponding functions.
Example Source Code
Please refer to Download Example Source Code to obtain the source code.
For related source code, please see files in the "Assets/ZegoExpressExample/Examples/AdvancedStreaming/StreamMonitoring" directory.
Prerequisites
Before monitoring call quality, please ensure:
-
A project has been created in the ZEGOCLOUD Console and a valid AppID and AppSign have been obtained.
-
The ZEGO Express SDK has been integrated in your project and basic audio/video publishing and playing functions have been implemented. For details, please refer to Quick Start - Integrate and Quick Start - Implementing Flow.
Basic Network Quality Report
You can monitor the current network quality of users in the room (including yourself) by listening to the OnNetworkQuality callback. This callback is received every 2 seconds.
OnNetworkQuality callback logic is as follows:
- As long as you are publishing or playing streams, you will receive your own network quality callback.
- When you are playing an audio/video stream published by another user in your room, you will receive that user's network quality callback.
- When "userID" is "null", it represents this time's your own network quality. When "userID" is not "null", it represents another user's network quality in the room.
OnNetworkQuality callback does not apply to scenarios using CDN for live streaming. You can refer to Advanced Quality Report - Streaming Quality Report to CDN for CDN streaming quality monitoring.
void OnNetworkQuality(string userID, ZegoStreamQualityLevel upstreamQuality, ZegoStreamQualityLevel downstreamQuality)
{
// Developers can monitor the current network quality reported to the business server in this callback, or give users friendly prompts
if (userID == "") {
// Represents the local user's (my) network quality
} else {
// Represents another user's network quality in the room
}
}
engine.onNetworkQuality = OnNetworkQuality;Advanced Quality Report
If the above basic network quality report cannot meet your needs, ZEGO also provides more detailed publishing quality report, playing quality report, and other related information.
Publishing Quality Report
The publishing quality report refers to the quality report of the process where a user pushes an audio/video stream to the ZEGO service server. It includes the frame rate of the audio/video stream during the capture and encoding stages, the frame rate and bitrate of the audio/video stream during the transmission (sending) stage, delay, and packet loss rate.
You can register the OnPublisherQualityUpdate callback to receive the publishing quality callback. Every 3 seconds after the stream is successfully published, you will receive this callback. You can understand the health status of the published audio/video stream in real time through the quality(ZegoPublishStreamQuality) parameter.
In most cases, you only need to focus on the "level" parameter of "quality" with the enumerated value "level" to determine the comprehensive quality of the publishing stream. For details, please refer to ZegoPublishStreamQuality.
If you want to focus on more detailed publishing quality parameters, you can refer to ZegoPublishStreamQuality.
void OnPublisherQualityUpdate(string streamID, ZegoPublishStreamQuality quality)
{
// Developers can monitor the specific quality in this callback and report to the business server for monitoring, or give users friendly prompts based on quality objects
// If developers don't know which quality field to focus on, they can focus on the level field of the quality object; this field is the comprehensive value of quality
switch (quality.level) {
case ZegoStreamQualityLevel.Excellent:
// Excellent
break;
case ZegoStreamQualityLevel.Good:
// Good
break;
case ZegoStreamQualityLevel.Medium:
// Medium
break;
case ZegoStreamQualityLevel.Bad:
// Bad
break;
case ZegoStreamQualityLevel.Die:
// Die
break;
case ZegoStreamQualityLevel.Unknown:
// Unknown
break;
default:
break;
}
}
engine.onPublisherQualityUpdate = OnPublisherQualityUpdate;Playing Quality Report
The playing quality report refers to the quality report of the process where a user pulls and plays an audio/video stream from the ZEGO service server. It includes the frame rate, bitrate, delay, and packet loss rate of the received audio/video stream during the decoding and rendering stages, the video overall quality during the rendering stage, and the freeze rate during the rendering stage.
You can register the OnPlayerQualityUpdate callback to receive the playing quality callback. Every 3 seconds after the stream is successfully played, you will receive this callback. You can understand the health status of the pulled audio/video stream in real time through the quality(ZegoPlayStreamQuality) parameter.
In most cases, you only need to focus on the "level" parameter of "quality" with the enumerated value "level" to determine the comprehensive quality of the playing stream. For details, please refer to ZegoStreamQualityLevel.
If you want to focus on more detailed playing quality parameters, you can refer to ZegoPlayStreamQuality.
void OnPlayerQualityUpdate(string streamID, ZegoPlayStreamQuality quality)
{
// Developers can monitor the specific quality in this callback and report to the business server for monitoring, or give users friendly prompts based on quality objects
// If developers don't know which quality field to focus on, they can focus on the level field of the quality object; this field is the comprehensive value of quality
switch (quality.level) {
case ZegoStreamQualityLevel.Excellent:
// Excellent
break;
case ZegoStreamQualityLevel.Good:
// Good
break;
case ZegoStreamQualityLevel.Medium:
// Medium
break;
case ZegoStreamQualityLevel.Bad:
// Bad
break;
case ZegoStreamQualityLevel.Die:
// Die
break;
case ZegoStreamQualityLevel.Unknown:
// Unknown
break;
default:
break;
}
}
engine.onPlayerQualityUpdate = OnPlayerQualityUpdate;MOS Audio Quality Score
Starting from ZEGO Express SDK version 2.16.0, a "mos" field is added to the playing stream quality callback OnPlayerQualityUpdate, representing the audio quality score of the played stream. When developers are concerned about audio quality, they can understand the current audio quality through this field.
The value range of the mos field is [-1, 5], where -1 indicates unknown (for example, abnormal stream playing when unable to score), and [0, 5] indicates a score. The corresponding MOS (Mean Opinion Score) for real-time audio quality is as follows:
| MOS Score | Evaluation Standard |
|---|---|
| 4.0～5.0 | Audio quality is excellent, clear and smooth, easy to hear clearly. |
| 3.5～4.0 | Audio quality is fairly good, occasional audio quality damage, but still clear and smooth, easy to hear clearly. |
| 3.0～3.5 | Audio quality is average, occasional stuttering, requires some attention to hear clearly. |
| 2.5～3.0 | Audio quality is fairly poor, frequent stuttering, requires concentrated attention to hear clearly. |
| 2.0～2.5 | Audio quality is poor, frequent stuttering, some semantic loss, difficult to communicate. |
| Less than 2.0 | Audio quality is extremely poor, massive semantic loss, unable to communicate. |
| -1 | Unknown. |
Other Quality Monitoring
Publishing/Playing State Change Notifications
Publishing State Callback
After the stream is successfully published, you can obtain the notification of publishing state changes through the OnPublisherStateUpdate callback.
void OnPublisherStateUpdate(string streamID, ZegoPublisherState state, int errorCode, string extendedData)
{
// When state is NoPublish and errorCode is not 0, it indicates publishing failure, and no more re-publishing attempts will be made. At this point, you can make a publishing failure prompt on the interface;
// When state is PublishRequesting and errorCode is not 0, it indicates re-publishing attempts. If re-publishing time is exceeded without successful publishing, a publishing failure notification will be thrown.
}
engine.onPublisherStateUpdate = OnPublisherStateUpdate;You can roughly determine the user's publishing network situation based on whether the "state" parameter is in the "requesting publishing state". The values of the "state" parameter corresponding to user publishing states are as follows:
| Enumerated Value | Description |
|---|---|
| ZegoPublisherState.NoPublish | Not publishing state. Before publishing, SDK is in this state. If a steady-state exception occurs during the publishing process, such as incorrect AppID, AppSign, or Token, or if other users are already publishing streams with the same stream ID, publishing will fail and enter the not publishing state. |
| ZegoPublisherState.PublishRequesting | Requesting publishing state. After the publishing operation is executed successfully, SDK will enter the requesting publishing state. This state is usually used for UI interface display. If the publishing process is interrupted due to poor network quality, SDK will perform internal retrying and also return to the requesting publishing state. |
| ZegoPublisherState.Publishing | Publishing state. Entering this state indicates that the publishing has been successful and users can communicate normally. |
The "extendedData" parameter is additional information attached with the state update. If you use ZEGO's CDN content distribution network, after the publishing is successful, the content keys of this parameter include "flv_url_list", "rtmp_url_list", "hls_url_list", corresponding to the pull stream URLs of the flv, rtmp, and hls protocols.
Playing State Callback
After the stream is successfully played, you can obtain the notification of playing state changes through the OnPlayerStateUpdate callback.
void OnPlayerStateUpdate(string streamID, ZegoPlayerState state, int errorCode, string extendedData)
{
// When state is NoPlay and errorCode is not 0, it indicates playing failure, and no more re-playing attempts will be made. At this point, you can make a playing failure prompt on the interface;
// When state is PlayRequesting and errorCode is not 0, it indicates re-playing attempts. If re-playing time is exceeded without successful playing, a playing failure notification will be thrown.
}
engine.onPlayerStateUpdate = OnPlayerStateUpdate;You can roughly determine the user's playing network situation based on whether the "state" parameter is in the "requesting playing state". The values of the "state" parameter corresponding to user playing states are as follows:
| Enumerated Value | Description |
|---|---|
| ZegoPlayerState.NoPlay | Not playing state. Before playing, SDK is in this state. If a steady-state exception occurs during the playing process, such as incorrect AppID, AppSign, or Token, playing will fail and enter the not playing state. |
| ZegoPlayerState.PlayRequesting | Requesting playing state. After the playing operation is executed successfully, SDK will enter the requesting playing state. This state is usually used for UI interface display. If the playing process is interrupted due to poor network quality, SDK will perform internal retrying and also return to the requesting playing state. |
| ZegoPlayerState.Playing | Playing state. Entering this state indicates that the playing has been successful and users can communicate normally. |
Publisher Audio Capture First Frame Notification
Publisher Audio Capture First Frame Callback
You can receive the audio capture first frame callback by registering OnPublisherCapturedAudioFirstFrame callback. After the calling publishing interface successfully, the SDK will receive this callback when it captures the first frame of audio data.
When not publishing or not previewing, for the first publishing or first previewing, i.e., when the SDK's internal audio/video module engine starts, it will collect the local device's audio data and this callback will be received. Developers can determine whether the SDK actually captures audio data based on this callback. If this callback is not received, it indicates that the audio capture device is busy or abnormal.
void OnPublisherCapturedAudioFirstFrame()
{
}
engine.onPublisherCapturedAudioFirstFrame = OnPublisherCapturedAudioFirstFrame;Player Audio Receive First Frame Callback
Developers can listen to the playing end audio receive first frame callback by registering OnPlayerRecvAudioFirstFrame callback. After the calling playing interface successfully, the SDK will receive this callback when it plays the first frame of audio data.
void OnPlayerRecvAudioFirstFrame(string streamID)
{
}
engine.onPlayerRecvAudioFirstFrame = OnPlayerRecvAudioFirstFrame;