Configuring TTS
Function Introduction
To match different personas and scenarios, you may need to:
- Select different text-to-speech (TTS) vendors, such as Volcano Engine, MiniMax, Aliyun, etc.
- Configure different voices.
- Customize the audio of TTS, such as volume, speed, tone, etc.
- Special rules can filter the content for TTS. For example, in "(happily) The weather is really nice today", the content inside the parentheses will be filtered out.
Prerequisites
- Enable AI Agent service
- Enable corresponding TTS vendor service:
- Method 1: Experience directly with the zego_test account.
- Method 2: Purchase TTS service through ZEGO. Please contact ZEGOCLOUD sales to obtain an account and authentication information.
- Method 3: Purchase TTS service on your own and obtain key information, etc.
Usage Method
Currently, TTS related parameters can be set through 4 interfaces:
| Interface | Description |
|---|---|
| Register Agent | Set vendor, voice, speed, etc. parameters. |
| Create Agent Instance Create Digital Human Agent Instance | Set vendor, voice, speed, etc. parameters. Note If not set, the TTS parameters carried by the registered Agent ( RegisterAgent ) will be used by default. |
| Update Agent Instance | Set voice, speed, etc. parameters. Note Does not support modifying the FilterText parameter. |
TTS Parameters Description
| Parameter Name | Type | Required | Description |
|---|---|---|---|
| Vendor | String | Yes | Text-to-speech (TTS) service provider. Optional values:
Note This parameter cannot be updated when updating the agent instance. |
| Params | Object | Yes | TTS configuration parameters, in JSON object format. Contains app parameters (for authentication) and other parameters (for adjusting TTS effects). Please refer to the Params parameter description below. |
| FilterText | Array of Object | No | Filter out text within specified punctuation marks from the content input to TTS (usually the content returned by the LLM or the Text parameter value of the SendAgentInstanceTTS interface), and then perform speech synthesis. For example, if the content inside the parentheses in "(happily) Zego Technology welcomes you!" is filtered out and then synthesized, set it to [{"BeginCharacters": "(", "EndCharacters": ")"}]Note FilterText is an Object array. Each Object contains two string type parameters: BeginCharacters and EndCharacters. Note This parameter cannot be updated when updating the agent instance. |
| TerminatorText | String | No | Can be used to set the termination text of TTS. If the content input to TTS (usually the content returned by the LLM or the Text parameter value of the SendAgentInstanceTTS interface) contains content matching the TerminatorText string, the content from the TerminatorText string (inclusive) will not be synthesized for this round of TTS. Note The bidirectional streaming can only set one character. The maximum length is 4 characters. |
| CharacterFilter | Array of String | No | The specified string in the content input to TTS (usually the content returned by the LLM or the Text parameter value of the SendAgentInstanceTTS interface) does not participate in speech synthesis. Note Each string in the array represents a string to be filtered out, and each string does not exceed 2 characters. Note This parameter cannot be updated when updating the agent instance. |
Params Description
| Parameter Name | Type | Required | Description |
|---|---|---|---|
| app | object | Yes | Used for TTS service authentication. The structure of the app parameter required varies depending on the value of Vendor. See the app parameter instructions for each vendor below. |
| Other Params | - | No | Besides the app parameter, you can also provide additional TTS configuration parameters to further customize the speech synthesis effect. These parameters are directly passed through to the associated TTS service provider.Refer to the official documentation for each vendor (by the value of Vendor) for detailed parameter information:
|
The definitions of the app parameter and other TTS parameters vary by vendor. Please refer to the parameter instructions for each vendor below.
Filtering text content before speech synthesis
Note
The filtering content feature is optional and can be set according to your actual needs.
In some scenarios, you may need to filter the text content generated by the LLM before speech synthesis. For example, in a companion app, it may require the LLM to use parentheses to represent emotion or tone in the conversation, but these contents are only used for subtitles displayed to users and do not need to be synthesized:
(happily) You are welcome to ZEGOCLOUD!Here "(happily)" does not need to be synthesized.
There are three ways to control text filtering, respectively:
Filter text content between specified punctuation marks using FilterText.
For example, to filter the content inside parentheses in "(happily) You are welcome to ZEGOCLOUD!", set it to:
"TTS":{
....
"FilterText": [
{
"BeginCharacters": "(",
"EndCharacters": ")"
}
]
.....
}Filter text content after a specific string using TerminatorText.
For example, to filter the content after "#" in "Your are welcome to ZEGOCLOUD! #2025-01-01.", set it to:
"TTS":{
....
"TerminatorText": "#"
.....
}Filter specific strings using CharacterFilter.
For example, to filter the symbols "-" and "*" in "- Tomorrow 10am meeting", set it to:
"TTS":{
....
"CharacterFilter": ["-","*"]
.....
}