Display Subtitles
Feature Overview
This document introduces how to use ZEGO's client subtitle component to display subtitles for corresponding users/audio streams in a streaming (typewriter-style) manner during voice or video calls.
- Types of content that the subtitle component can display:
- User speech subtitles: Streaming display of user speech text content, with support for forward error correction.
- Translation subtitles: Display of translation text results.
- Scope of subtitle component display:
- Room-level subtitles for all users/audio streams
- User/stream-level subtitles
Core Concepts
The core fields involved in the subtitle component are described as follows:
| Field | Type | Description |
|---|---|---|
| Timestamp | Number | Timestamp, in seconds |
| SeqId | Number | Packet sequence number, may be out of order, please sort messages by sequence number. In extreme cases, IDs may not be consecutive. |
| Round | Number | Conversation round, increases each time the user actively speaks |
| Cmd | Number | 201: ASR speech recognition text 202: LLM translated text |
| Data | Object | Specific content, different Cmd corresponds to different Data |
Different Cmd values correspond to different Data, as follows:
Subtitle Component Usage Guide
Prerequisites
- Basic functionality has been implemented according to the Quick Start documentation:
- Integrated ZEGO Express SDK to implement basic voice call functionality.
- Enabled cloud real-time speech recognition and configured
SubtitleType(default is 0) to1,2, or3to deliver subtitles via room signaling.
You must use the ZEGO Express SDK version optimized for Cloud ASR from the Download SDK and Demo page, otherwise subtitles will not display properly.
Using the Subtitle Component
Display Partial Subtitles Only (Optional)
You can filter by UserId to display subtitles only for certain users or streams. Taking displaying only other users' translated subtitles as an example.
Custom Subtitle Implementation (Not Recommended)
The client can obtain room custom messages with method as liveroom.room.on_recive_room_channel_message by listening to the onRecvExperimentalAPI callback.
Determine the message type based on the Cmd field, and get the message content based on the Data field.
Notes
- Message sorting: Data received through room custom messages may be out of order and needs to be sorted by SeqId field.
- Streaming text processing:
- ASR text delivers full text each time. Messages with the same MessageId need to completely replace previous content.
- LLM text delivers incremental text each time. Messages with the same MessageId need to be sorted and accumulated for display.
- Memory management: Please clean up completed message caches in a timely manner, especially when users engage in long conversations.