Call Quality Monitoring
Overview
During calls using ZEGO Express SDK, users sometimes experience poor network conditions. At this time, you can understand the changes in current call network quality and audio/video information through relevant callbacks.
For example, when conducting multi-person audio and video calls or multi-person singing, if we need to display the user's network quality in real time, we can refer to this document to implement the corresponding functions.
Prerequisites
Before monitoring call quality, ensure:
- You have created a project in the ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to Console - Project Information.
- You have integrated ZEGO Express SDK into the project and implemented basic audio and video streaming functionality. For details, please refer to Quick Start - Integration and Quick Start - Implementation.
Quality Reports
ZEGO provides detailed publishing stream quality reports, playing stream quality reports, and other related information.
Publishing Stream Quality Report
The publishing stream quality report refers to the quality report of the process of users pushing audio and video to the ZEGO server, including the frame rate of the audio and video stream during the capture and encoding stages, and the frame rate, bitrate, latency, and packet loss rate of the transmitted (sent) audio and video stream.
You can receive publishing stream quality callbacks by registering publisherQualityUpdate. After successfully publishing, you will receive this callback every three seconds. You can understand the health status of the published audio and video stream in real time based on the quality(ZegoPublishStreamQuality) parameter.
- In most cases, you only need to pay attention to the "level" parameter of "quality" to judge the comprehensive quality of the publishing stream based on the "level" enumeration value. For details, please refer to ZegoStreamQualityLevel.
- If you want to pay attention to more detailed publishing stream quality parameters, you can refer to ZegoPublishStreamQuality.
ZegoExpressEngine.instance().on("publisherQualityUpdate", (streamID, quality) => {
console.info(`publisherQualityUpdate:width:streamID:${streamID}, quality:${JSON.stringify(quality)}`);
// Developers can monitor specific quality in this callback and report to the business server for monitoring, or monitor a certain field of the quality object to give user-friendly prompts
// If developers don't know which field of quality to monitor, they can focus on the level field, which is the comprehensive value of the quality object
}Playing Stream Quality Report
The playing stream quality report refers to the quality report of the process of users playing audio and video streams, including the frame rate, bitrate, latency, and packet loss rate of the received audio and video stream, the frame rate of the audio and video stream during the decoding stage, and the frame rate, freeze rate, and overall audio and video quality during the rendering stage.
You can receive playing stream quality callbacks by registering playerQualityUpdate. After successfully playing, you will receive this callback every three seconds. Developers can understand the health status of the played audio and video stream in real time based on the quality(ZegoPlayStreamQuality) parameter.
- In most cases, you only need to pay attention to the "level" parameter of "quality" to judge the comprehensive quality of the playing stream based on the "level" enumeration value. For details, please refer to ZegoStreamQualityLevel.
- If you want to pay attention to more detailed playing stream quality parameters, you can refer to ZegoPlayStreamQuality.
ZegoExpressEngine.instance().on("playerQualityUpdate", (streamID, quality) => {
console.info(`playerQualityUpdate:streamID:${streamID}, quality:${JSON.stringify(quality)}`););
// Developers can monitor specific quality in this callback and report to the business server for monitoring, or monitor a certain field of the quality object to give user-friendly prompts
// If developers don't know which field of quality to monitor, they can focus on the level field, which is the comprehensive value of the quality object
}MOS Audio Quality Score
Starting from ZEGO Express SDK version 2.16.0, a "mos" field has been added to the playing stream quality callback playerQualityUpdate, representing the score of the playing stream audio quality. When developers pay more attention to audio quality, they can understand the current audio quality through this field.
The value range of the mos field is [-1, 5], where -1 means unknown (for example, cannot score when playing stream is abnormal), and [0, 5] means the normal scoring range. The corresponding subjective audio quality perception of real-time audio MOS scoring is as follows:
| MOS Value | Evaluation Criteria |
|---|---|
| 4.0 ~ 5.0 | Excellent audio quality, clear and smooth, easy to hear. |
| 3.5 ~ 4.0 | Good audio quality, occasional audio quality damage, but still clear and smooth, easy to hear. |
| 3.0 ~ 3.5 | Fair audio quality, occasional freezes, requires some attention to hear clearly. |
| 2.5 ~ 3.0 | Poor audio quality, frequent freezes, requires concentrated attention to hear clearly. |
| 2.0 ~ 2.5 | Very poor audio quality, partial semantic loss, difficult to communicate. |
| 0 ~ 2.0 | Extremely poor audio quality, massive semantic loss, unable to communicate. |
| -1 | Unknown. |
Other Information Monitoring
Publishing/Playing Stream Status Change Notification
Publishing Stream Status Callback
After successfully publishing, you can receive notifications of publishing stream status changes through publisherStateUpdate.
ZegoExpressEngine.instance().on("publisherStateUpdate", (streamID, state, errorCode, extendedData) => {
console.info(`publisherStateUpdate:streamID:${streamID}, state:${state}, errorCode:${errorCode}, extendedData:${JSON.stringify(extendedData)}`);
// When state is ZegoPublisherState.NoPublish and errcode is non-zero, it indicates publishing failure and no more retry publishing will be performed. At this time, a publishing failure prompt can be displayed on the interface;
// When state is ZegoPublisherState.PublishRequesting and errcode is non-zero, it indicates retrying publishing. If publishing is not successful within the retry time, a publishing failure notification will be thrown.
}You can roughly judge the user's publishing network situation based on whether the "state" parameter in the callback is in "requesting publishing status". The values of the "state" parameter correspond to the user's publishing status as follows:
| Enumeration Value | Description |
|---|---|
| ZegoPublisherState.NoPublish | Not publishing status, in this state before publishing. If a steady-state exception occurs during the publishing process, such as incorrect AppID, AppSign, or Token, or if other users are already publishing streams, publishing streams with the same stream ID will fail and will enter the not publishing status. |
| ZegoPublisherState.PublishRequesting | Requesting publishing status. After the publishing operation is successfully executed, it will enter the requesting publishing status. Usually, the UI interface is displayed through this status. If an interruption occurs due to poor network quality, the SDK will internally retry and will also return to the requesting publishing status. |
| ZegoPublisherState.Publishing | Publishing status. Entering this status indicates that publishing has been successful and users can communicate normally. |
The parameter "extendedData" is extended information attached to the status update. If using ZEGO's CDN content distribution network, after successfully publishing, the keys of the parameter content are "flv_url_list", "rtmp_url_list", "hls_url_list", corresponding to the playing URLs of flv, rtmp, and hls protocols respectively.
Playing Stream Status Change Callback
After successfully playing, developers can receive notifications of playing stream status changes through playerStateUpdate.
ZegoExpressEngine.instance().on("playerStateUpdate", (streamID, state, errorCode, extendedData) => {
console.info(`playerStateUpdate:streamID:${streamID}, state:${state}, errorCode:${errorCode}, extendedData:${JSON.stringify(extendedData)}`);
// When state is ZegoPlayerState.NoPlay and errcode is non-zero, it indicates publishing failure and no more retry publishing will be performed. At this time, a publishing failure prompt can be displayed on the interface;
// When state is ZegoPlayerState.PlayRequesting and errcode is non-zero, it indicates retrying publishing. If publishing is not successful within the retry time, a publishing failure notification will be thrown.
}Developers can roughly judge the user's playing network situation based on whether the "state" parameter is in "requesting playing status". The values of the "state" parameter correspond to the user's playing status as follows:
| Enumeration Value | Description |
|---|---|
| ZegoPlayerState.NoPlay | Not playing status, in this state before playing. If a steady-state exception occurs during the playing process, such as incorrect AppID, AppSign, or Token, it will enter the not playing status. |
| ZegoPlayerState.PlayRequesting | Requesting playing status. After the playing operation is successfully executed, it will enter the requesting playing status. Usually, the application interface is displayed through this status. If an interruption occurs due to poor network quality, the SDK will internally retry and will also return to the requesting playing status. |
| ZegoPlayerState.Playing | Playing status. Entering this status indicates that playing has been successful and users can communicate normally. |
Notification of Receiving Audio/Video First Frame
Publishing End Audio Capture First Frame Callback
You can receive audio first frame callbacks by registering publisherCapturedAudioFirstFrame. After successfully calling the publishing interface, you will receive this callback when the SDK captures the first frame of audio data.
When not publishing or not previewing, for the first publishing or first preview, that is, when the engine of the audio and video module inside the SDK starts, it will capture the local device's audio data and will receive this callback. Developers can judge whether the SDK really captures audio data based on this callback. If this callback is not received, it means the audio capture device is occupied or abnormal.
ZegoExpressEngine.instance().on("publisherCapturedAudioFirstFrame", () => {
console.info(`publisherCapturedAudioFirstFrame`);
}Publishing End Video Capture First Frame Callback
You can receive video first frame callbacks by registering publisherCapturedVideoFirstFrame. After successfully calling the publishing interface, you will receive this callback when the SDK captures the first frame of video data.
When not publishing or not previewing, for the first publishing or first preview, that is, when the engine of the audio and video module inside the SDK starts, it will capture the local device's video data and will receive this callback. You can judge whether the SDK really captures video data based on this callback. If this callback is not received, it means the video capture device is occupied or abnormal.
ZegoExpressEngine.instance().on("publisherCapturedVideoFirstFrame", (channel) => {
console.info(`publisherCapturedVideoFirstFrame:width:channel:${channel}`);
}Playing End Audio Receive First Frame Callback
Developers can listen to the playing end audio receive first frame callback by registering playerRecvAudioFirstFrame. After successfully calling the playing interface, you will receive this callback when the SDK plays the first frame of audio data.
ZegoExpressEngine.instance().on("playerRecvAudioFirstFrame", (streamID) => {
console.info(`playerRecvAudioFirstFrame:streamID:${streamID}`);
}Playing End Video Receive First Frame Callback
You can listen to the playing end receive video first frame callback by registering playerRecvVideoFirstFrame. After successfully calling the playing interface, you will receive this callback when the SDK plays the first frame of video data.
ZegoExpressEngine.instance().on("playerRecvVideoFirstFrame", (streamID) => {
console.info(`playerRecvVideoFirstFrame:streamID:${streamID}`);
}Playing End Render Video First Frame Callback
You can listen to the playing end render video first frame callback by registering playerRenderVideoFirstFrame. After successfully calling the playing interface, you will receive this callback when the SDK plays and renders the first frame of video data.
You can use this callback to calculate first frame time or update the UI components of the playing stream.
ZegoExpressEngine.instance().on("playerRenderVideoFirstFrame", (streamID) => {
console.info(`playerRenderVideoFirstFrame:streamID:${streamID}`);
}Video Resolution Change Callback
Capture Video Resolution Change Callback
You can listen to the capture video size change callback by registering publisherVideoSizeChanged. After successfully publishing, if the video capture resolution changes during publishing, you will receive this callback.
When not publishing or not previewing, for the first publishing or first preview, that is, when the engine of the audio and video module inside the SDK starts, it will capture the local device's video data, and the capture resolution will change at this time.
You can use this callback to remove local preview UI covers and similar operations. You can also dynamically adjust the preview view ratio based on the resolution of this callback.
ZegoExpressEngine.instance().on("publisherVideoSizeChanged", (width, height, channel) => {
console.info(`publisherVideoSizeChanged:width:${width}, height:${height}, channel:${channel}`);
}Playing Resolution Change Notification
You can receive playing resolution change notifications by registering playerVideoSizeChanged. After successfully playing, if the video resolution changes during playing, you will receive this callback. Users can adjust the display based on the final resolution of the stream.
- If the played stream only has audio data, this callback will not be received.
- If the publishing end triggers the SDK's internal flow control due to network issues, it may dynamically reduce the publishing end's encoding resolution, and this callback will also be received at this time. When the played audio and video stream is really rendered to the set UI playing interface, this callback will be triggered. Developers can use this callback notification to update or switch the UI components that really play the stream.
ZegoExpressEngine.instance().on("playerVideoSizeChanged", (streamID, width, height) => {
console.info(`playerVideoSizeChanged:streamID:${streamID},width:${width},height:${height}`);
}