Audio and Video Track Replacement

Feature Overview

In browsers, media streams are composed of audio tracks and video tracks. When there are two or more media streams locally, ZEGO Express Web SDK can freely combine the audio and video tracks contained in media streams.

For example, a streamer needs to play background music. The streamer end Captures media stream A through camera and microphone, and also loads an MP3 music media stream B through the <audio> tag. After replacing the audio track in media stream A with the audio track from media stream B, the audience playing media stream A can hear the background music but cannot hear the streamer's voice.

Example Source Code Download

Please refer to Download Example Source Code to get the source code.

For related source code, please check files in the "src/Examples/Others/MediaTrackReplacement" directory.

Prerequisites

Before implementing audio and video track replacement functionality, please ensure:

• A project has been created in ZEGOCLOUD Console, and valid AppID and Server address have been obtained. For details, please refer to Console - Project Information.
• ZEGO Express SDK has been integrated into the project, and basic audio/video streaming functionality has been implemented. For details, please refer to Quick Start - Integration and Quick Start - Implementation.

Usage Steps

Publish Camera Feed

Call the createZegoStream interface, set the camera attribute to create camera and microphone Capture source data, and obtain a ZegoLocalStream instance object localStream.

Call the startPublishingStream interface to push the local stream to the remote end (ZEGO server). Set the publishing video attributes through the videoCodec parameter. Only "VP8" or "H264" can be passed in. The default value is "H264".

// Creating a stream is an asynchronous process. You need to use await syntax. Here await cannot be removed.
const localStream = await zg.createZegoStream({camera: {video: true, audio: true}});
// publishStreamId is customized, needs to be unique
// publishOption see interface documentation for details
const result = zg.startPublishingStream(publishStreamId, localStream, publishOption);

// Preview
// Preview the video, and mount the playback component to the page's component container DOM.
localStream.playVideo(document.querySelector("#local-video"));

// Creating a stream is an asynchronous process. You need to use await syntax. Here await cannot be removed.
const localStream = await zg.createZegoStream({camera: {video: true, audio: true}});
// publishStreamId is customized, needs to be unique
// publishOption see interface documentation for details
const result = zg.startPublishingStream(publishStreamId, localStream, publishOption);

// Preview
// Preview the video, and mount the playback component to the page's component container DOM.
localStream.playVideo(document.querySelector("#local-video"));

Load Background Music

Load background music through the <audio> tag.

<!--Load relative path mp3 music-->
<audio id="customAudio" crossOrigin="anonymous" loop preload playsinline controls src="../../../assets/16183688734997.mp3"></audio>

<!--Load relative path mp3 music-->
<audio id="customAudio" crossOrigin="anonymous" loop preload playsinline controls src="../../../assets/16183688734997.mp3"></audio>

Replace Microphone with Background Music

Call the startCaptureCustomAudio interface to replace the microphone with background music. Call the updatePublishingStream interface to synchronously update the publishing audio, replacing the microphone with background music.

// Listen to background music toggle button
$('#CustomAudio').on('change', async function({ target }) {
    // Selected case
    if (target.checked) {
        // Play background music
        $('#customAudio')[0].play()
        try {
            // Capture third-party background music
            await localStream.startCaptureCustomAudio({
                source: $('#customAudio')[0]
            })
            // Update publishing audio of publishing stream localStream
            await zg.updatePublishingStream(localStream, 1)
        } catch (err) {
            console.log(err);
        }
    }
});

// Listen to background music toggle button
$('#CustomAudio').on('change', async function({ target }) {
    // Selected case
    if (target.checked) {
        // Play background music
        $('#customAudio')[0].play()
        try {
            // Capture third-party background music
            await localStream.startCaptureCustomAudio({
                source: $('#customAudio')[0]
            })
            // Update publishing audio of publishing stream localStream
            await zg.updatePublishingStream(localStream, 1)
        } catch (err) {
            console.log(err);
        }
    }
});

Replace Camera with Screen Sharing

Call the startCaptureScreen interface to replace the camera with screen sharing. Call the updatePublishingStream interface to synchronously update the publishing video, replacing the camera with screen sharing.

try {
    // Capture screen sharing
    await localStream.startCaptureScreen()
    // Update publishing video of publishing stream localStream
    await zg.updatePublishingStream(localStream, 0)
} catch (err) {
    console.log(err);
}

try {
    // Capture screen sharing
    await localStream.startCaptureScreen()
    // Update publishing video of publishing stream localStream
    await zg.updatePublishingStream(localStream, 0)
} catch (err) {
    console.log(err);
}