ZEGO Avatar SDK 2.0: Take Your Metaverse Immersion to the Next Level

“As a creator of voice-driven facial animations, every project is a unique challenge that requires attention to detail and creativity. The goal is to bring the Avatar to life with natural-looking expressions that perfectly complement the voiceover. It takes a lot of work and effort to ensure that each facial animation is synchronized with the voice and captures the right emotion. Utilizing the latest technology and tools, I strive to optimize every aspect of the animation process, from the initial design to the final touches. My ultimate goal is to create an immersive and engaging experience for the viewer, where they forget that they are interacting with a virtual Avatar and feel like they are talking to a real person.”

– ZEGOCLOUD AI Team

With ZEGO Avatar SDK, developers can easily incorporate a 3D Avatar creator into their apps, eliminating the need for external apps. The 1.0 version of the SDK boasts impressive capabilities, such as automatic Avatar generation, manual Avatar creation, facial expression mirroring, speech simulation, and gesture & body pose recognition. For a refresher on how to put these features to use, check out How to Create Your Own Avatar and How to Create an Avatar from a Photo.

The ZEGO Avatar SDK 2.0 has taken animation and synchronization technologies to the next level. 3 major updates come with version 2.0. Let’s explore them.

Text-to-Speech (TTS)

AI can recognize written text and make the Avatar display the corresponding mouth shape and speaking manner while utilizing the ability of text-to-speech to play the corresponding audio. Thus, you will get a text-to-speech Avatar or talking Avatar by simply inputting text.

Key Capabilities

• Users can input up to 1000 characters of text continuously. With the ZEGO Avatar SDK 2.0, the input text can be automatically segmented into sentences based on semantic punctuation. For each sentence, users can generate corresponding voice and facial expressions.
• Users can adjust the speech speed and volume and choose from multiple voice packages to support different scenario requirements.
• Extensive AI model analysis and training have optimized facial expressions and voice mapping. Therefore, the Avatar’s expressions are highly accurate, achieving a 90% consistency with real people in speaking scenarios based on the input text.

New Opportunities

ZEGO Avatar SDK 2.0’s TTS feature opens up new gaming, entertainment, e-learning, and telehealth opportunities. The most common scenarios for a text-to-speech avatar include the following:

• Virtual Customer Service – Combining with a knowledge base to provide simulated human-like responses.
• Virtual Hosts – With preset phrases, they can appropriately greet and interact with viewers.
• Virtual Actor in the Video – The TTS avatar can be utilized in various introduction/explanatory videos.

The Talking Avatar Guide – Android – iOS

Upgraded Motion Capture and Mapping Capabilities

Users can quickly and conveniently experience full-body motion capture mapping through their phone camera without additional motion capture devices. This creates vivid interactions and reduces barriers to adopting new technology.

With ZEGOCLOUD’s unique low-latency, high-precision 3D human key point modeling and advanced position encoding design, ZEGO Avatar achieves a synchronized virtual and real effect, allowing for more natural body movements. This capability can be utilized in various scenarios, such as virtual live streaming, online fitness, and e-learning, providing users with a seamless experience.

AR Avatar

ZEGO Avatar SDK 2.0 allows the creation of virtual images seamlessly blended with real-time camera video using a headgear model. Users can create a more personalized and unique virtual Avatar as their digital identity to explore the virtual world.

ZEGO Avatar SDK 2.0 is now available

ZEGO AvatarSDK 2.0 is now available to developers, providing them with advanced tools and capabilities to enhance virtual experiences. The upgraded features include text-to-speech capability for easy customization of Avatars, improved motion capture and mapping for natural and fluid body movements, and an AR-driven feature for personalized virtual Avatars.

With these cutting-edge features, ZEGO Avatar SDK 2.0 takes metaverse immersion to the next level. It’s an essential tool for developers who want to create innovative and engaging virtual experiences for their users.