AI Effects for Your Audio and Video Apps

Video calling and live broadcasting has steadily risen in popularity over the past ten years to rank among the most widely used entertainment mediums. Numerous apps are available. However, have you ever considered integrating them into your audio and video application? If so, this article will show you ZEGOCLOUD AI Effects, a service that provides various AI-powered real-time video effects.

Why ZEGOCLOUD AI Effects for Audio & Video Apps

Various AI-powered real-time video effects, such as face beautification, AR effects, picture segmentation, and more, are offered via the ZEGOCLOUD AI Effects SDK. You can use it for various purposes, including live social and entertainment streaming, online learning, and camera tools.

The main features are:

• Face beautification: skin smoothing, improvement of skin tone, eyes brightening, teeth whitening, elimination of eye bags and dark circles, elimination of freckles, elimination of acne, and more.
• Virtual face makeup: lipstick, eyeliner, eyeshadow, mascara, eyebrow pencils, foundation, blusher, and more.
• Face shape retouching: precise facial critical point recognition and 3D face models, including slimming of the cheekbones, chin, and nose, expanding of the eyes, adjusting the distance between the eyes, changing the form of the lips, and more
• Segmentation of images: several picture segmentation options, including head, hair, green screen, and portrait segmentation
• Filters: a wide variety of stunning picture and video filters, including ones with the following styles: light, soft, Japanese, woodland, fresh, and more
• AR results (coming soon): AR special effects, such as face stickers, 2D/3D stickers, and stickers for the environment, are used, all of which have exceptional durability to create good results when used with complicated backdrops, lighting changes, and elongated poses.

The Advantages of ZEGOCLOUD AI Effects SDK

The advantages of ZEGOCLOUD SDK for face filters and AI effects are the following:

• Robust facial recognition that can correctly detect 148 facial key points after extensive human face data analysis and training
• Leading-edge AI algorithms enable high-precision human body identification and placement algorithms that work well in challenging situations like selfies and video filming.
• Stable tracking of portrait features keeps track of portrait characteristics even when the subject’s face or body turns or moves swiftly.
• Precise virtual makeup with realistic-looking virtual face cosmetics thanks to extensive training and iterative facial keypoint recognition fine-tuning
• Complete compatibility with iOS, Android, Windows, and macOS

When to Use ZEGOCLOUD AI Effects

You can integrate the cutting-edge ZEGOCLOUD AI Effects to serve users in several scenarios:

• Live to broadcast social and entertainment streaming, live game streaming, video conferencing, and more; ZegoEffects can enhance your live streaming apps with powerful features like real-time face detection, face beautification, video filters, and many others.
• Photography: it enables the creation of a wide range of camera applications with outstanding beauty features and thought-provoking stickers
• Short videos: create mobile video recording and short video apps that let users record and edit videos uniquely.
• Video creation: as a powerful tool for professional video creation, it offers video filters, stickers, and many more features to speed up and lower the cost of your video production.

You can try face filters and other excellent tools with the demo of AI Effects SDK, available for iOS and Android.

How to Integrate the ZEGOCLOUD AI Effects SDK

Before starting, there are some prerequisites to be met. We will consider integrating AI Effects SDK on Android devices.

Ensure that your development environment meets the following criteria:

• at least Android Studio 2.1
• Android SDK 25 or later, Android SDK Platform-Tools 25.x.x, Android SDK Build-Tools 25.0.2
• Android 5.0 or later-running hardware or emulator (we recommend you use an actual device)

Step 1: Create a new project

1. Launch Android Studio, and click Create New Project in the Welcome to Android Studio window.

If you have a project already opened, select File > New > New Project.

2. In the Configure your project window, fill in the project name and the location to save the project.

3. Use the default settings for other fields, click Next, and click Finish to complete the project creation.

Step 2: Import the SDK

Currently, the SDK supports these Application Binary Interfaces (ABIs): arm64-v8a, armeabi-v7a, and x86_64.

To import the SDK, do the following:

1. Contact the ZEGO team to get the latest version of the SDK.
2. Extract the SDK package to your project directory, for example, app/libs.

3. Open the app/build.gradle file, and add references to the SDK.

• In the defaultConfig block, add the ndk block to specify the supported ABIs.

ndk {
    abiFilters'armeabi-v7a','arm64-v8a','x86_64'
}

• In the android block, add the sourceSets block to specify the directory where the SDK library files are located.

NOTE:

The libs directory in the following sample code is just an example. Use the actual path according to your project setup.

sourceSets {
    main {
        jniLibs.srcDirs = ['libs']
    }
}

• Add the following line to import all JAR files under the libs directory in the dependencies block.

1 implementation fileTree(dir:’libs’, include: [‘*.jar’])

Step 3: Add app permissions

Add the following permissions in the app/src/main/AndroidManifest.xml file to access the devices according to your app’s requirements.

CAUTION:

Request runtime permissions: Android 6.0 requires specific necessary permissions to be obtained at runtime. So, besides declaring permissions in your app’s AndroidMainfest.xml file, you also need to add the following code to request the permissions at runtime.

Note: requestPermissions is a method of the Activity.

String[] permissionNeeded = { "android.permission.CAMERA" };

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
    if (ContextCompat.checkSelfPermission(this, "android.permission.CAMERA") != PackageManager.PERMISSION_GRANTED) {
        requestPermissions(permissionNeeded, 101);
    }
}

Step 4: Prevent code obfuscation

In the proguard-rules.pro file, add the following line to prevent the classes of the ZegoEffects SDK from being obfuscated.

-keep class **.zego.**{*;}

For better insight and precise code transcription, please refer to the documentation page directory on this link.

Step 5: Import resources and models

Remember that before you can use the AI features of the ZegoEffects SDK, you need to import the resources or models required for these features.

• Import the related resources first to use features such as skin tone enhancement, teeth whitening, cheek blusher, and stickers.
• Import the related models first to use features such as eye enlarging, face slimming, and portrait segmentation.

It’s a very straightforward procedure; you will find the implementation steps on this link.