Introduction

The Dolby Interactivity APIs provide a platform for unified communications and collaboration. Using in-flow communication you can combine voice, video, and messaging as an integrated solution for your own applications in a way that is cohesive to your end-users. This is in contrast to out-of-app communications where users must exit your application and turn to third-party stand-alone tools.

For example, with the Interactivity Client SDKs you can build:

  • A video conference application for your employee portal
  • A broadcast webinar application for presentations to grow your audience
  • A 1-on-1 help desk service for your specialized customers

Let's review a few key concepts for how in-flow communications can be incorporated into your application.

Components

Dolby Interactivity APIs provide client software development kits (SDKs) that provide support for building desktop and mobile experiences. These client libraries provide the functionality you need for building an application.

The Client SDKs support:

  • Web browsers (desktop and mobile) utilizing WebRTC for wide compatibility without requiring your end-users to install any plugins
  • Mobile native iOS and Android libraries to embed in your projects for more control

See the Supported Environments section for more specific details on the platforms and operating systems.

To further enable building applications Dolby Interactivity APIs provide:

  • REST APIs that allow you to monitor conference activities
  • Webhooks to react to events that may occur during a conference
  • UXKit to help get started with building a user interface with a plug-and-play reference solution

You can find additional documentation for these components in the Reference documentation. You may also find some of our Developer Tools helpful as you get started building.

How It Works

The Dolby Interactivity APIs are offered as a Communications Platform as a Service (CPaaS). The Dolby media server allows you to focus on building your own client applications for real-time audio and video communications.

Using one of our client SDKs you can build an application that:

  1. Create and initialize a conference
  2. Invite other users to join
  3. Manage attendees
  4. Coordinate with messages between clients

These steps are described in more detail with code samples in the Creating a Basic Audio Conference Application getting started tutorial.

CPaaS

This process is managed by the platform over a websocket connection between each instance of the client app and the Dolby Interactivity Platform media servers. This connection uses UDP or TCP as necessary to maintain client-server synchronization in contrast to peer-to-peer solutions.

The platform provides a Selective Forwarding Unit (SFU) that receives media streams from each individual participant. The media server then decides how to best route video in HD (720p30) as H.264 or VP8 back to the other participants in a 1:1 or multi-party conference.

The platform also provides a Multi-point Conferencing Unit (MCU) that is powered by Dolby Voice, an award-winning audio communications technology. Audio is delivered as an Opus stream or with the Dolby Voice Codec (DVC) depending on the client platform to provide improvements in voice quality and clarity.

This includes features such as:

  • Dynamic audio leveling
  • Advanced spatial audio
  • Noise and echo reduction
  • Optimized bandwidth utilization
  • Advanced network resilience to maintain audio quality in challenging network conditions

With the REST APIs, applications can also use HTTP Live Streaming (HLS) and Real-Time Messaging Protocol (RTMP) to achieve scale through media broadcasting platforms that support those protocols.

Features

The SDKs define some data models such as Conference and Participant for use, but puts you, the developer, in control of choosing which features to use while you build a user interface to suit your application needs. Explore the Client SDK documentation to learn more about these options.

Devices

Dolby has one of the most extensive device databases when it comes to understanding media capabilities. The Interactivity Platform leverages this deep knowledge of user media such as screens, microphones and cameras to provide wide compatibility and performance. This enables functionality such as device selection, muting and unmuting the microphone, enabling or disabling the video camera, etc.

For information on how to use these features, see the MediaDeviceService section.

Presentations

Content sharing enables users to present what is on their screen to all participants in the conference. Content sharing is important for collaboration, as it allows for presentations, peer review, etc. Dolby Interactivity APIs support screen share (powerpoint) and video sharing.

For more information on how to use these features, see the Presenting section.

Recording

Recording provides flexibility to create a permanent record of a conference or broadcast. These recordings can be captured live, generated on-demand, or captured from webhook events.

Additionally, you can create a mixer recording layout which is a specific app to customized with user-defined mixer layouts.

For more information on how to use these features, see the Recording section.

Messaging

To keep multiple client applications in sync, the Dolby Interactivity APIs provide message broadcasting to send data to all conference participants. This could be JSON, XML, or a simple string that triggers behavior on other clients.

For more information on how to use these features, see the CommandService documentation.

Streaming / Broadcasting

Dolby Interactivity APIs support streaming conferences through HLS and RTMP protocols. You can start a stream using a REST API call and monitor the status listening to Webhook events.