探索iOS自定义ijkplayer画中画播放
iOS提供AVPictureInPictureController用于画中画播放控制，但是只能绑定AVPlayer使用，对于开发者来说挺头痛的。在iOS 15.0后呼之欲出，支持SampleBufferDisplayLayer自定义数据源显示图层，意味着我们可以自定义第三方播放器实现画中画。以ijkplayer为例，让我们一起探索如何支持画中画播放。
目录
一、判断画中画支持
二、后台播放配置
1、配置AudioSession
2、配置后台模式
三、画中画生命周期
1、从全屏进入画中画
2、退出画中画回到全屏
3、生命周期
四、画中画创建流程
1、初始化
2、提供start和stop方法
3、实现画中画代理方法
4、更新缓冲区图层
5、更新播放状态
6、避坑指南
一、判断画中画支持
我们可以通过isPictureInPictureSupported() 判断当前设备是否支持画中画。
二、后台播放配置
1、配置AudioSession
在application创建时，配置AudioSession为playback模式，代码如下：
func application(_ application: UIApplication,didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {let audioSession = AVAudioSession.sharedInstance()do {try audioSession.setCategory(.playback)} catch {print("Setting category to AVAudioSessionCategoryPlayback failed.")}return true
}2、配置后台模式
勾选“Audio, Airplay, Picture in Picture”选项，如下图所示：
三、画中画生命周期
1、从全屏进入画中画
启动画中画时，首先执行willStartPictureInPicture()方法，此时动画还未开始。等到初始化完毕，执行didStartPictureInPicture()方法，完成从全屏到画中画的切换，进入后台状态。
2、退出画中画回到全屏
退出画中画时，从后台状态恢复app的UI，执行willStopPictureInPicture()方法。最终完成退出，执行didStopPictureInPicture()方法，回到全屏播放。
3、生命周期
画中画完整生命周期包括：从全屏到画中画，最终退出画中画回到全屏。即上面两个生命周期的结合，如下图所示：
四、画中画创建流程
1、初始化
初始化AVSampleBufferDisplayLayer，包括设置frame、opaque、position、videoGravity、controlTimebase。示例代码如下：
- (AVSampleBufferDisplayLayer *) createDisplayLayer
{AVSampleBufferDisplayLayer *layer;layer = [[AVSampleBufferDisplayLayer alloc] init];layer.frame = self.bounds;layer.opaque = YES;layer.position = CGPointMake(CGRectGetMidX(self.bounds), CGRectGetMidY(self.bounds));layer.videoGravity = AVLayerVideoGravityResizeAspect;CMTimebaseRef controlTimebase;CMTimebaseCreateWithSourceClock( CFAllocatorGetDefault(), CMClockGetHostTimeClock(), &controlTimebase);layer.controlTimebase = controlTimebase;return layer;
}使用AVSampleBufferDisplayLayer来初始化ContentSource，前面先判断设备是否支持画中画。示例代码如下：
- (BOOL)initPictureInPicture:(AVSampleBufferDisplayLayer *)displayLayer {if ([AVPictureInPictureController isPictureInPictureSupported] == NO) {NSLog(@"Sorry, don't support PictureInPicture mode!");return NO;}if (@available(iOS 15.0, *)) {_displayLayer = displayLayer;_contentSource = [[AVPictureInPictureControllerContentSource alloc]initWithSampleBufferDisplayLayer:displayLayer playbackDelegate:self];return YES;} else {return NO;}
}2、提供start和stop方法
开始画中画，包括初始化AVPictureInPictureController、设置delegate代理、设置播放速率、启动。。示例代码如下：
- (void)startPictureInPicture {if (@available(iOS 15.0, *)) {if (_pipController != nil) {return;}_pipController = [[AVPictureInPictureController alloc] initWithContentSource:_contentSource];_pipController.delegate = self;CMTimebaseSetRate(_displayLayer.controlTimebase, 1);CMTimebaseSetTime(_displayLayer.controlTimebase, CMTimeMake([_delegate getCurrentTime], 1));// 延时启动，否则可能会启动失败dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(100 * NSEC_PER_MSEC)), dispatch_get_main_queue(), ^{if ([self->_pipController isPictureInPicturePossible]) {[self->_pipController startPictureInPicture];NSLog(@"startPictureInPicture...");}});}
}结束画中画， 示例代码如下：
- (void)stopPictureInPicture {if (_pipController != nil && [_pipController isPictureInPictureActive]) {[_pipController stopPictureInPicture];}
}3、实现画中画代理方法
实现AVPictureInPictureControllerDelegate代理方法如下：
- (void)pictureInPictureControllerWillStartPictureInPicture:(AVPictureInPictureController *)pictureInPictureController {NSLog(@"pictureInPictureControllerWillStart...");
}- (void)pictureInPictureControllerDidStartPictureInPicture:(AVPictureInPictureController *)pictureInPictureController {// 回调画中画开始[_delegate onStartPictureInPicture:nil];
}- (void)pictureInPictureController:(AVPictureInPictureController *)pictureInPictureControllerfailedToStartPictureInPictureWithError:(NSError *)error {// 画中画开始失败[_delegate onStartPictureInPicture:error];
}- (void)pictureInPictureControllerWillStopPictureInPicture:(AVPictureInPictureController *)pictureInPictureController {NSLog(@"pictureInPictureControllerWillStop...");
}- (void)pictureInPictureControllerDidStopPictureInPicture:(AVPictureInPictureController *)pictureInPictureController {[self stopPictureInPicture];// 回调画中画结束[_delegate onStopPictureInPicture];
}- (void)pictureInPictureController:(AVPictureInPictureController *)pictureInPictureControllerrestoreUserInterfaceForPictureInPictureStopWithCompletionHandler:(void (^)(BOOL restored))completionHandler {completionHandler(true);// 恢复全屏界面
}4、更新缓冲区图层
在ijk解码出来后，封装成CVPixelBuffer，然后封装CMSampleBufferRef，存入displayLayer。示例代码如下：
-(void) enqueueBuffer:(CVPixelBufferRef)pixelBuffer
{if (!_displayLayer || !pixelBuffer)return;@autoreleasepool {CMSampleBufferRef sampleBuffer = nil;CMVideoFormatDescriptionRef format = nil;// 设置dts、ptsCMSampleTimingInfo timeInfo = {.presentationTimeStamp = kCMTimeInvalid, .decodeTimeStamp = kCMTimeInvalid};OSStatus status = CMVideoFormatDescriptionCreateForImageBuffer(kCFAllocatorDefault, pixelBuffer, &format);status = CMSampleBufferCreateForImageBuffer(kCFAllocatorDefault, pixelBuffer, true, NULL, NULL,format, &timeInfo, &sampleBuffer);if (format != nil) {CFRelease(format);}if (sampleBuffer == nil || status != noErr) {NSLog(@"sampleBuffer error!");return;}CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, YES);CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);// 设置立即渲染CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);[_displayLayer enqueueSampleBuffer:sampleBuffer];CFRelease(sampleBuffer);if (_displayLayer.status == AVQueuedSampleBufferRenderingStatusFailed) {NSLog(@"displayLayer error=%@", [_displayLayer error]);}}
}5、更新播放状态
实现AVPictureInPictureSampleBufferPlaybackDelegate代理方法，更新播放状态、seek进度。示例代码如下：
- (void)pictureInPictureController:(AVPictureInPictureController *)pictureInPictureController setPlaying:(BOOL)playing {_isPlaybackPaused = !playing;// 回调播放/暂停[_delegate setPlaying:playing];if (@available(iOS 15.0, *)) {[pictureInPictureController invalidatePlaybackState];}if (playing == NO) {_playClickTime = [[NSDate date] timeIntervalSince1970] * 1000;}
}- (CMTimeRange)pictureInPictureControllerTimeRangeForPlayback:(AVPictureInPictureController *)pictureInPictureController {// 计算time range：0～durationNSTimeInterval durationTime = [_delegate getDurationTime];return CMTimeRangeMake(kCMTimeZero, CMTimeMake(durationTime * 1000, 1000));
}- (BOOL)pictureInPictureControllerIsPlaybackPaused:(AVPictureInPictureController *)pictureInPictureController {return _isPlaybackPaused;
}- (void)pictureInPictureController:(AVPictureInPictureController *)pictureInPictureControllerdidTransitionToRenderSize:(CMVideoDimensions)newRenderSize {}- (void)pictureInPictureController:(AVPictureInPictureController *)pictureInPictureController skipByInterval:(CMTime)skipIntervalcompletionHandler:(void (^)(void))completionHandler {// 回调seek[_delegate seekTo:CMTimeGetSeconds(skipInterval)];if (@available(iOS 15.0, *)) {// seek开始会暂停播放，seek成功后恢复播放状态if (([[NSDate date] timeIntervalSince1970] * 1000 - _playClickTime) < 200) {[_delegate setPlaying:YES];}// 更新displaylayer的controlTimebase时钟，从而更新进度CMTimebaseSetTime(_displayLayer.controlTimebase, CMTimeMake([_delegate getCurrentTime] * 1000, 1000));// 更新播放状态[_pipController invalidatePlaybackState];}
}ijkplayer提供play、seek、getCurrentPosition、getDuration、isPlaying等接口，给自定义的画中画Controller调用：
- (BOOL)isPlaybackPaused
{return [self isPlaying] == NO;
}- (void)setPlaying:(BOOL)playing
{if (playing) {[self play];} else {[self pause];}
}- (void)seekTo:(NSTimeInterval)relativePosition
{NSTimeInterval position = [self currentPlaybackTime] + relativePosition;[self setCurrentPlaybackTime:position];
}- (NSTimeInterval)getCurrentTime
{return [self currentPlaybackTime];
}- (NSTimeInterval)getDurationTime
{return [self duration];
}6、避坑指南
(1) 延时启动：大概要延时100ms左右启动AVPictureInPictureController，否则可能启动画中画失败，有点坑爹
(2) 设置时钟：需要设置dts、pts，设置立即渲染。坑爹+1
(3) 更新播放状态：seek开始时会暂停播放，等到seek成功要恢复播放状态，坑爹+2
本文来自互联网用户投稿，文章观点仅代表作者本人，不代表本站立场，不承担相关法律责任。如若转载，请注明出处。 如若内容造成侵权/违法违规/事实不符，请点击【内容举报】进行投诉反馈！