ScreenShare with iOS

This article describes how to build an iOS application with device level screen-sharing.

How does it work?

Before Dolby Interactivity APIs iOS SDK 2.3, screen-sharing on iOS was available only inside the application. The iOS SDK 2.3 introduces a new way of sharing the screen at the device level. It uses the ReplayKit2 framework and the Broadcast Upload extension made available in iOS 11. Additionally, the iOS SDK 2.3 introduces the VoxeetScreenShareKit that manages the screen broadcast.

The following workflow describes the screen-share process:

1. The presenter enables screen-sharing on the iPhone to record the device screen. This activates ReplayKit 2 which starts recording the screen and creating media samples.

2. Broadcast Upload extension retrieves the recorded media samples in real time. It encodes them, creates a video stream, and uploads this stream to the broadcast service. The extension gets the CMSampleBuffer object and transfers the object to the VoxeetScreenShareKit.

3. VoxeetScreenShareKit uploads the CMSampleBuffer object to the shared repository called App Group Shared Data.

4. VoxeetSDK retrieves the CMSampleBuffer from the App Group Shared Data repository.

5. VoxeetSDK uses the CMSampleBuffer to receive the shared content, create a screen-share stream, and send the stream to the conference.

screen-share

API reference

The iOS SDK 2.3 updates the startScreenShare API, which adds a new broadcast parameter. The parameter is a Boolean that specifies whether the shared screen is used inside (false) or outside (true) the application.

    .startScreenShare(broadcast: boolean,completion: ((_ error: NSError?) -> Void)?)

Implementation guide

The following steps describe how to implement the screen-sharing feature into your application.

Note: Before using the screen-sharing option, configure the required settings using Xcode.

Prerequisites:

  • A valid Apple developer account
  • A valid Apple provision file supporting app groups
  • Apple iOS 12 and later versions

1. Download VoxeetScreenShareKit using Carthage dependency manager, CocoaPods dependency manager, or Voxeet GitHub.

binary "https://raw.githubusercontent.com/voxeet/voxeet-screensharekit-ios/master/VoxeetScreenShareKit.json"
  • CocoaPods dependency manager
pod 'VoxeetScreenShareKit'

2. Open your project in Xcode and click the Signing & Capabilities tab to access the configuration page.

3. Select the + Capability and navigate to the App Groups. Then select the valid App Groups.

screen-share-setup

4. Click the + button on the bottom of the left side menu to create a new target.

5. In the pop-up window, select the Broadcast Upload extension and click Next.

screen-share-setup2

6. In the new target, navigate to the didFinishLaunchingWithOptions method, where the Dolby Interactivity APIs iOS SDK is initialized in the AppDelegate.swift.

import VoxeetSDK

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
    var window: UIWindow?

    func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
        ...
        VoxeetSDK.shared.initialize(...)
        VoxeetSDK.shared.appGroup = "YOUR_APP_GROUP"
        VoxeetSDK.shared.preferredExtension = "YOUR_BROADCAST_EXTENSION_BUNDLE_ID"

        return true
    }
}

Replace the following phrases mentioned in the code:

  • YOUR_APP_GROUP with the App Group name.
  • YOUR_BROADCAST_EXTENSION_BUNDLE_ID with the Bundle Identifier, which you can find in the Signing & Capabilities tab, in the Signing section.

7. Click the Signing & Capabilities tab, and navigate to App Groups.

8. Click the + icon on the bottom of the App Groups section to create a new App Group in your extension and set its name.

screen-share-setup3

9. Click the Build Phases tab, and navigate to the Link Binary With Libraries.

10. Set the status for VoxeetScreenShareKit.framework to Required.

screen-share-setup4

11. Click the Build Settings tab, and set the iOS Deployment Target to iOS 11.0.

screen-share-setup5

12. Open the SampleHandler.swift file that was automatically created with the Broadcast Upload extension. Replace YOUR_APP_GROUP mentioned in the code with the App Group name. Copy the sample code from this file to your iOS application.

import ReplayKit
import VoxeetScreenShareKit
class SampleHandler: RPBroadcastSampleHandler, VoxeetScreenShareKitDelegate {
    private var screenShareService: VoxeetScreenShareKit?
    override func broadcastStarted(withSetupInfo setupInfo: [String : NSObject]?) {
        screenShareService = VoxeetScreenShareKit(appGroup: "YOUR_APP_GROUP")
        screenShareService?.delegate = self
        screenShareService?.broadcastStarted(withSetupInfo: setupInfo)
    }
    override func broadcastPaused() {
        screenShareService?.broadcastPaused()
    }
    override func broadcastResumed() {
        screenShareService?.broadcastResumed()
    }
    override func broadcastFinished() {
        screenShareService?.broadcastFinished()
    }
    override func processSampleBuffer(_ sampleBuffer: CMSampleBuffer, with sampleBufferType: RPSampleBufferType) {
        screenShareService?.processSampleBuffer(sampleBuffer, with: sampleBufferType)
    }
    func finishBroadcastWithError(error: Error) {
        self.finishBroadcastWithError(error)
    }
}

13. Create a Screen Share button calling the startScreenShare method inside the application.

VoxeetSDK.shared.conference.startScreenShare(broadcast: true)

14. Tap the created Screen Share button to see the screen-recording window. Select your created project from the available options to share your screen.

screen-share-setup6