How to customize video capture rotation method?
This document will introduce you how to customize the video capture rotation method.
For v2.23.0 and later versions, if you need to customize the video capture rotation method, to avoid conflicts with other video orientation modes, please do not call setAppOrientationMode|\blank to set modes other than custom mode.
Overview
If you want to achieve a customized screen rotation effect during mobile video calls or live streaming, you can refer to the following three common implementation methods:
- Fixed portrait
- Fixed landscape
- Auto-switch between portrait and landscape
In the following examples, the rendering mode of the "receiving end" is the default "ZegoViewModeAspectFit" (proportional scaling, may have black borders). For details, please refer to ZegoViewMode.
Fixed portrait: Means the sending end fixes the video capture in portrait mode. When the receiving end device is in portrait mode, the video displayed fills the device screen in portrait mode. When the receiving end device is in landscape mode, the video displayed has a certain rotation angle relative to the sending end image (the following example shows a 90-degree counterclockwise rotation).
- If the device rotation direction is locked:
- If the device rotation direction is not locked:
Fixed landscape: Means the sending end fixes the video capture in landscape mode. When the receiving end device is in landscape mode, the video displayed fills the device screen in landscape mode. When the receiving end device is in portrait mode, the video displayed has a certain rotation angle relative to the sending end image (the following example shows a 90-degree counterclockwise rotation).
- If the device rotation direction is locked:
- If the device rotation direction is not locked:
Auto-switch between portrait and landscape: Provides video rotation functionality. Users can rotate the video 90, 180, or 270 degrees counterclockwise relative to the phone's upright position as needed. This helps users obtain the desired video rendering effect based on the video scenario. After video rotation, it will automatically adjust to adapt to the encoded image resolution.
The interfaces called by the above three methods have differences. For detailed descriptions, please refer to the Implementation Process in this document.
Download Sample Source Code
Please refer to Download Sample Source Code to get the source code.
For related source code, please check the files in the "/ZegoExpressExample/Examples/CommonFeatures/VideoRotation" directory.
Prerequisites
Before implementing video capture rotation, please ensure:
- You have integrated ZEGO Express SDK in your project and implemented basic real-time audio and video functionality. For details, please refer to Quick Start - Integrate SDK and Quick Start - Implement Video Call.
- You have created a project in ZEGOCLOUD Console and applied for a valid AppID and AppSign. For details, please refer to "Project Information" in Console - Project Management.
Implementation Process
Fixed Portrait
iOS
When using portrait live streaming or video calls, the width resolution should be smaller than the height resolution. Assuming it is "360 × 640", you need to implement the function through the following steps.
- Call the createEngine interface to create an SDK engine instance. For details, please refer to "Create Engine" in Quick Start - Implementation Process.
ZegoEngineProfile *profile = [[ZegoEngineProfile alloc] init];
// Please get through official website registration, format: 1234567890
profile.appID = appID;
// Please get through official website registration
profile.appSign = appSign;
// For general scenario access, please choose the appropriate scenario based on your actual situation
profile.scenario = ZegoScenarioDefault;
// Create engine and register self as eventHandler callback. If you don't need to register callback, the eventHandler parameter can be passed nil. You can call "-setEventHandler:" method later to set callback
[ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];- (Optional) Call the setVideoConfig interface to set the encoding resolution. The default value is "360 × 640". You can adjust it as needed. If you keep the default value, skip this step. The resolution setting for portrait live streaming is as follows:
ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.encodeResolution = CGSizeMake(360, 640);
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];- Call the loginRoom interface to login to a room. For details, please refer to "Login Room" in Quick Start - Implementation Process.
ZegoUser *user = [ZegoUser userWithUserID:@"test_userid"];
[[ZegoExpressEngine sharedEngine] loginRoom:@"test_roomid" user:user];- Call the startPreview interface to start local preview to display the local image. previewView is UIView (iOS) or NSView (macOS).
ZegoCanvas *previewCanvas = [ZegoCanvas canvasWithView:previewView];
[[ZegoExpressEngine sharedEngine] startPreview:previewCanvas]；- Call the startPublishingStream interface to start publishing stream and push the local audio and video stream to the real-time audio and video cloud. For details, please refer to "Publishing Stream" in Quick Start - Implementation Process.
[[ZegoExpressEngine sharedEngine] startPublishingStream:@"test_streamid"];- Use fixed portrait mode for live streaming or video calls.
Android
When using portrait live streaming or video calls, the width resolution should be smaller than the height resolution. Assuming it is "360 × 640", you need to implement the function through the following steps.
- Call the createEngine interface to create an SDK engine instance. For details, please refer to "Create Engine" in Quick Start - Implementation Process.
// Create engine, general scenario access, and register self as eventHandler callback
// If you don't need to register callback, the eventHandler parameter can be passed null. You can call "setEventHandler:" method later to set callback
ZegoEngineProfile profile = new ZegoEngineProfile();
profile.appID = ; // Please get through official website registration, format: 1234567890L
profile.appSign = ; // Please get through official website registration, format: "1234567890"
profile.scenario = ZegoScenario.DEFAULT; // For general scenario access, please choose the appropriate scenario based on your actual situation
profile.application = getApplication();
engine = ZegoExpressEngine.createEngine(profile, null);- (Optional) Call the setVideoConfig interface to set the encoding resolution. The default value is "360 × 640". You can adjust it as needed. If you keep the default value, skip this step.
ZegoVideoConfig config = new ZegoVideoConfig();The resolution setting for portrait live streaming is as follows:
config.setEncodeResolution(360, 640);
ZegoExpressEngine.getEngine().setVideoConfig(config);- Call the loginRoom interface to login to a room. For details, please refer to "Login Room" in Quick Start - Implementation Process.
ZegoExpressEngine.getEngine().loginRoom("test_roomid", new ZegoUser("test_userid"), null);- Call the startPreview interface to start local preview to display the local image. textureViewLocalPreview is a TextureView object in the layout.
ZegoExpressEngine.getEngine().startPreview(new ZegoCanvas(textureViewLocalPreview));- Call the startPublishingStream interface to start publishing stream and push the local audio and video stream to the real-time audio and video cloud. For details, please refer to "Publishing Stream" in Quick Start - Implementation Process.
ZegoExpressEngine.getEngine().startPublishingStream("test_streamid");- Use fixed portrait mode for live streaming or video calls.
Fixed Landscape
iOS
When using landscape live streaming or video calls, the width resolution should be larger than the height resolution. Assuming it is "640 × 360", you need to implement the function through the following steps.
- Call the createEngine interface to create an SDK engine instance. For details, please refer to "Create Engine" in Quick Start - Implementation Process.
ZegoEngineProfile *profile = [[ZegoEngineProfile alloc] init];
// Please get through official website registration, format: 1234567890
profile.appID = appID;
// Please get through official website registration
profile.appSign = appSign;
// For general scenario access, please choose the appropriate scenario based on your actual situation
profile.scenario = ZegoScenarioDefault;
// Create engine and register self as eventHandler callback. If you don't need to register callback, the eventHandler parameter can be passed nil. You can call "-setEventHandler:" method later to set callback
[ZegoExpressEngine createEngineWithProfile:profile eventHandler:self];- Call the setVideoConfig interface to set the encoding resolution to "640 × 360".
The resolution setting for landscape live streaming is as follows:
ZegoVideoConfig *videoConfig = [[ZegoVideoConfig alloc] init];
videoConfig.encodeResolution = CGSizeMake(640, 360);
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];- Call the setAppOrientation interface to set the video orientation. If rotating 90 degrees counterclockwise, pass UIInterfaceOrientationLandscapeLeft. If rotating 90 degrees clockwise, pass UIInterfaceOrientationLandscapeRight.
[[ZegoExpressEngine sharedEngine] setAppOrientation:UIInterfaceOrientationLandscapeLeft];- Call the loginRoom interface to login to a room. For details, please refer to "Login Room" in Quick Start - Implementation Process.
ZegoUser *user = [ZegoUser userWithUserID:@"test_userid"];
[[ZegoExpressEngine sharedEngine] loginRoom:@"test_roomid" user:user];- Call the startPreview interface to start local preview to display the local image. previewView is UIView (iOS) or NSView (macOS).
ZegoCanvas *previewCanvas = [ZegoCanvas canvasWithView:previewView];
[[ZegoExpressEngine sharedEngine] startPreview:previewCanvas]；- Call the startPublishingStream interface to start publishing stream and push the local audio and video stream to the real-time audio and video cloud. For details, please refer to "Publishing Stream" in Quick Start - Implementation Process.
[[ZegoExpressEngine sharedEngine] startPublishingStream:@"test_streamid"];- Use fixed landscape mode for live streaming or video calls.
Android
When using landscape live streaming or video calls, the width resolution should be larger than the height resolution. Assuming it is "640 × 360", you need to implement the function through the following steps.
- Call the createEngine interface to create an SDK engine instance. For details, please refer to "Create Engine" in Quick Start - Implementation Process.
// Create engine, general scenario access, and register self as eventHandler callback
// If you don't need to register callback, the eventHandler parameter can be passed null. You can call "setEventHandler:" method later to set callback
ZegoEngineProfile profile = new ZegoEngineProfile();
profile.appID = ; // Please get through official website registration, format: 1234567890L
profile.appSign = ; // Please get through official website registration, format: "1234567890"
profile.scenario = ZegoScenario.DEFAULT; // For general scenario access, please choose the appropriate scenario based on your actual situation
profile.application = getApplication();
engine = ZegoExpressEngine.createEngine(profile, null);- Call the setVideoConfig interface to set the encoding resolution to "640 × 360".
ZegoVideoConfig config = new ZegoVideoConfig();The resolution setting for landscape live streaming is as follows:
config.setEncodeResolution(640, 360);
ZegoExpressEngine.getEngine().setVideoConfig(config);- Call the setAppOrientation interface to set the video orientation. If rotating 90 degrees counterclockwise, pass ZegoOrientation.ORIENTATION_90. If rotating 90 degrees clockwise, pass ZegoOrientation.ORIENTATION_270.
ZegoExpressEngine.getEngine().setAppOrientation(ZegoOrientation.ORIENTATION_90);- Call the loginRoom interface to login to a room. For details, please refer to "Login Room" in Quick Start - Implementation Process.
ZegoExpressEngine.getEngine().loginRoom("test_roomid", new ZegoUser("test_userid"), null);- Call the startPreview interface to start local preview to display the local image. textureViewLocalPreview is a TextureView object in the layout.
ZegoExpressEngine.getEngine().startPreview(new ZegoCanvas(textureViewLocalPreview));- Call the startPublishingStream interface to start publishing stream and push the local audio and video stream to the real-time audio and video cloud. For details, please refer to "Publishing Stream" in Quick Start - Implementation Process.
ZegoExpressEngine.getEngine().startPublishingStream("test_streamid");- Use fixed landscape mode for live streaming or video calls.
Auto-switch between Portrait and Landscape
iOS
If the App is set to rotate the video image based on changes in gravity during live streaming or video calls, you need to listen to the screen rotation events of the corresponding platform and rotate the video direction accordingly.
When developers use SDK capture, the switching between portrait and landscape is mainly done by calling the setVideoConfig interface to set the encoding resolution and calling the setAppOrientation interface to set the video orientation.
- Listen to device orientation changes.
[[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(orientationChanged:) name:UIDeviceOrientationDidChangeNotification object:[UIDevice currentDevice]];- In the callback of device orientation changes, modify the corresponding encoding resolution and set the video orientation.
- (void)orientationChanged:(NSNotification *)notification {
UIDevice *device = notification.object;
ZegoVideoConfig *videoConfig = [[ZegoExpressEngine sharedEngine] getVideoConfig];
UIInterfaceOrientation orientation = UIInterfaceOrientationUnknown;
switch (device.orientation) {
// Note that UIInterfaceOrientationLandscapeLeft is equal to UIDeviceOrientationLandscapeRight (and vice versa).
// This is because rotating the device to the left requires rotating the content to the right.
case UIDeviceOrientationLandscapeLeft:
orientation = UIInterfaceOrientationLandscapeRight;
videoConfig.encodeResolution = CGSizeMake(640, 360);
break;
case UIDeviceOrientationLandscapeRight:
orientation = UIInterfaceOrientationLandscapeLeft;
videoConfig.encodeResolution = CGSizeMake(640, 360);
break;
case UIDeviceOrientationPortrait:
orientation = UIInterfaceOrientationPortrait;
videoConfig.encodeResolution = CGSizeMake(360, 640);
break;
case UIDeviceOrientationPortraitUpsideDown:
orientation = UIInterfaceOrientationPortraitUpsideDown;
videoConfig.encodeResolution = CGSizeMake(360, 640);
break;
default:
// Unknown / FaceUp / FaceDown
break;
}
[[ZegoExpressEngine sharedEngine] setVideoConfig:videoConfig];
[[ZegoExpressEngine sharedEngine] setAppOrientation:orientation];
}When developers use custom video capture, after listening to device orientation changes, you can refer to the following two methods to implement auto-switch between portrait and landscape:
- Handle video frame data yourself: In the callback of device orientation changes, rotate the captured video frame data, and then pass the processed data to the SDK through the sendCustomVideoCapturePixelBuffer interface.
- Handle video frame data through SDK: In the callback of device orientation changes, before passing the captured video frame data to the SDK, set the "rotation" in ZegoVideoEncodedFrameParam according to the actual orientation, call the sendCustomVideoCaptureEncodedData interface to pass the video frame data and orientation setting parameters to the SDK.
Android
If the App is set to rotate the video image based on changes in gravity during live streaming or video calls, you need to listen to the screen rotation events of the corresponding platform and rotate the video direction accordingly.
When developers use SDK capture, the switching between portrait and landscape is mainly done by calling the setVideoConfig interface to set the encoding resolution and calling the setAppOrientation interface to set the video orientation.
Listen to the "onConfigurationChanged" callback triggered by the screen rotation event in the corresponding Activity. In the event processing callback, modify the corresponding encoding resolution and inform the SDK of the App's orientation.
@Override
public void onConfigurationChanged(Configuration newConfig) {
super.onConfigurationChanged(newConfig);
// Checks the orientation of the screen
ZegoVideoConfig videoConfig = ZegoExpressEngine.getEngine().getVideoConfig();
ZegoOrientation orientation = ZegoOrientation.ORIENTATION_0;
if(Surface.ROTATION_0 == this.getWindowManager().getDefaultDisplay().getRotation()){
orientation = ZegoOrientation.ORIENTATION_0;
videoConfig.setEncodeResolution(360, 640);
}else if(Surface.ROTATION_180 == this.getWindowManager().getDefaultDisplay().getRotation()){
orientation = ZegoOrientation.ORIENTATION_180;
videoConfig.setEncodeResolution(360, 640);
}else if(Surface.ROTATION_270 == this.getWindowManager().getDefaultDisplay().getRotation()){
orientation = ZegoOrientation.ORIENTATION_270;
videoConfig.setEncodeResolution(640, 360);
}else if(Surface.ROTATION_90 == this.getWindowManager().getDefaultDisplay().getRotation()){
orientation = ZegoOrientation.ORIENTATION_90;
videoConfig.setEncodeResolution(640, 360);
}
ZegoExpressEngine.getEngine().setAppOrientation(orientation);
ZegoExpressEngine.getEngine().setVideoConfig(videoConfig);
}When developers use custom video capture, after listening to device orientation changes, you can refer to the following two methods to implement portrait and landscape switching:
- Handle video frame data yourself: In the callback of device orientation changes, rotate the captured video frame data, and then pass the processed data to the SDK through the sendCustomVideoCaptureTextureData interface.
- Handle video frame data through SDK: In the callback of device orientation changes, before passing the captured video frame data to the SDK, set the "rotation" in ZegoVideoEncodedFrameParam according to the actual orientation, call the sendCustomVideoCaptureEncodedData interface to pass the video frame data and orientation setting parameters to the SDK.
If the recorded live stream cannot be played, the possible reason is: Since the encoding resolution is modified when switching between portrait and landscape, some third-party players have poor compatibility with videos with modified resolutions, which may cause playback failures. Therefore, it is generally not recommended to modify the resolution when switching between portrait and landscape during live streaming or video calls.