[SWIFT] Picture-in-picture starting from iOS14

1.First of all

Picture-in-picture, which was already introduced on the iPad, is now compatible with iOS. I will describe the flow of introduction to the application from now on.

2. About picture-in-picture

By introducing picture-in-picture, video playback will be possible on other apps.

In addition, the application supports picture-in-picture, If autoplay is enabled, it will start when you press the home button.

2-1. About automatic playback of picture-in-picture

If your application supports picture-in-picture There is a behavior setting in the settings app when you press the home button while watching.

2-1-1. Items added to the settings app

I don't think the average iPhone user is aware of it. It is added to the place where you dive about 2 layers of the setting application.

item iOS14 iPadOS13
hierarchy Settings → General → Picture in Picture Settings → Home screen and Dock → Multitasking
item name Play picture-in-picture automatically Picture in picture
Explanatory text When turned on, the overlay will continue to play the video even when you press the home button. the same

2-1-2. Behavior introduction of supported apps

Next, I investigated how the behavior changes depending on whether the picture-in-picture autoplay setting is On / Off. I also checked if there was any other wire that would start picture-in-picture.

Behavior of TV application (genuine apple) </ b> https://apps.apple.com/jp/app/apple-tv/id1174078549

operation Autoplay ON Autoplay OFF
Press the home button Transition to the home screen and start picture-in-picture Transitions to the home screen, playback stops, and audio stops * 1
Press the picture-in-picture button Transition to the home screen and start picture-in-picture Transition to the home screen and start picture-in-picture
Swipe down on the playback screen Viewing stops Viewing stops
  • 1 ・ ・ ・ Voice can be resumed from the control center </ b>.

Twitch behavior </ b>

operation Autoplay ON Autoplay OFF
Press the home button Transition to the home screen and start picture-in-picture Transitions to the home screen, playback stops, and audio stops * 1
Press the picture-in-picture button Transition to the home screen and start picture-in-picture Transition to the home screen and start picture-in-picture
Swipe down on the playback screen Proprietary mini player starts Proprietary mini player starts
  • 1 ・ ・ ・ Voice can be resumed from the control center </ b>.

2-1-3. Behavior summary

To get the user to start picture-in-picture Operate with the home button and prepare a picture-in-picture button.

The behavior when the home button is pressed follows OS settings </ b>. When you press the picture-in-picture button, start picture-in-picture </ b> as usual.

3. Actually introduce

3-1. Prerequisites

There are AVPlayerView, AVPlayerViewController, and AVPlayerLayer to implement the player. However, this time we will describe the implementation using AVPlayerLayer.

3-2. Corresponding items for introduction

3-2-1. Check if picture-in-picture is supported

Check it because it may not be supported by the OS such as iOS12.

3-2-2. Start preparing for playback of picture-in-picture

Inform the OS that the picture-in-picture will be played and prepare.

3-2-3. Wait until the picture-in-picture becomes playable

Wait for the OS to be ready for picture-in-picture playback.

3-2-4. Start playing picture-in-picture

The start command is executed only when starting with the picture-in-picture button.

3-3. View sample app

https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player

SamplePlayer.swift


func setupPictureInPicture() {
    // Ensure PiP is supported by current device.
    if AVPictureInPictureController.isPictureInPictureSupported() {
        // Create a new controller, passing the reference to the AVPlayerLayer.
        pictureInPictureController = AVPictureInPictureController(playerLayer: playerLayer)
        pictureInPictureController.delegate = self
        
        pipPossibleObservation = pictureInPictureController.observe(\AVPictureInPictureController.isPictureInPicturePossible,
                                                                    options: [.initial, .new]) { [weak self] _, change in
            // Update the PiP button's enabled state.
            self?.pictureInPictureButton.isEnabled = change.newValue ?? false
        }
    } else {
        // PiP isn't supported by the current device. Disable the PiP button.
        pictureInPictureButton.isEnabled = false
    }
}

@IBAction func togglePictureInPictureMode(_ sender: UIButton) {
    if pictureInPictureController.isPictureInPictureActive {
        pictureInPictureController.stopPictureInPicture()
    } else {
        pictureInPictureController.startPictureInPicture()
    }
}

AVPictureInPictureController.isPictureInPictureSupported() Corresponds to 3-2-1. Here, it is determined whether the target device supports picture-in-picture.

AVPictureInPictureController(playerLayer: playerLayer) Corresponds to 3-2-2. I am creating an AVPictureInPictureController. By creating this AVPictureInPictureController, it will be automatically pressed when the home button is pressed. Picture-in-picture playback will begin.

AVPictureInPictureController.isPictureInPicturePossible Corresponds to 3-2-3. There is a several seconds delay </ b> between the generation of AVPictureInPictureController and its activation. Since it changes dynamically, it seems that the sample application uses observe. If you are using Rx, I think it is possible to monitor the parameters with Rx.

pictureInPictureController.startPictureInPicture() Corresponds to 3-2-4. Requests a start when the picture-in-picture button is pressed after it is ready. It is not necessary when the home button is pressed to start automatic playback.

3-4. Detailed notes, etc.

Create AVPictureInPictureController

As described in the previous section, there is a delay from the generation of AVPictureInPictureController to its activation. Therefore, if the process is started after the button is pressed, the response will be poor.

isPictureInPicturePossible becomes true but may not start

Currently under investigation </ b> When AVPictureInPictureController is generated, it becomes true for a moment, but immediately returns to false. It may be a phenomenon that occurs only in my environment, but I will describe it just in case.


The above is a rough outline of picture-in-picture. It can be introduced in more than a dozen lines, so why not consider it?

Recommended Posts