For those of you who built your spatial audio solution on High Fidelity, we know many are looking for new solutions, and taking the time to shift your product roadmaps to use new technologies as soon as you can. Dolby.io is here to provide a smooth transition from your High Fidelity application to our Communications APIs for those of you looking for an alternative spatial audio provider.
To migrate your spatial audio applications without any interruption, we suggest using Dolby.io Spatial Audio as your provider as soon as possible. This post serves as a guide to help you with this endeavor. One crucial difference Dolby.io Communications has compared to High Fidelity however, is that Dolby.io provides the cloud communication software solution alongside the spatial audio solution, meaning there is no need to integrate with a separate provider for video and voice conferences. We will walk through setting this feature up in this guide as well.
Register for a Dolby.io account
You can sign up for a free Dolby.io account here. This account will provide a free credit just for signing up, giving you plenty of minutes to trial your solution with no credit card needed.
Create a new Application and obtain credentials
To obtain your API key and secret, you need to create a new application in the Dolby.io dashboard. Each new app will create a new API key and secret pair, letting you rotate and separate credentials between projects as needed.
While we are here, we will want to enable Spatial Audio, as it is currently in Open Beta. To do this, we will need to join the Open Beta program by following these instructions.
Download the Dolby.io Communications SDK
Using the Dolby.io SDKs makes it easy to use our REST APIs, and manage your conference programmatically. Here is a comparison table showing platforms supported between High Fidelity and Dolby.io:
| Platform | High Fidelity | Dolby.io Communications |
|---|---|---|
| Web – JavaScript | ✅ | ✅ |
| Android – Java & Kotlin | ❌ | ✅ |
| iOS – Swift | ✅ | ✅ |
| React-Native | ❌ | ✅ |
| Cordova | ❌ | ✅ |
| Electron | ❌ | ✅ |
| C++ | ✅ | ✅ |
You can use the Dolby.io SDK with the following code in HTML:
<script
src="https://unpkg.com/@voxeet/[email protected]/dist/voxeet-sdk.js"
type="text/javascript"
></script>For other languages, see our documentation.
Set up the conferencing capabilities
Follow this documentation how to set up cloud conferencing with Dolby.io Communications. As this guide is focused on migrating spatial audio capabilities, we will not go in depth into these in this guide, so we suggest taking a look at how to set up your environment.
Differences between Dolby.io and High Fidelity
Before we integrate spatial audio into our conference, we should understand how spatial audio works on Dolby.io in comparison to High Fidelity. In High Fidelity, spatial audio is configured as a shared audio scene between all users in a conference, akin to a room. In this scenario, each client sets its own position only, which reduces the calls needed for each client but it also restricts the use cases that can be supported.
Dolby.io instead uses a unique audio scene for each participant, meaning we need to communicate the spatial positions between different clients for a consistent spatial experience, but gives us more control on what can be done, such as Audio/Video congruence, which lets you hear the audio for all the other participants from positions related to the video layout surrounding the local participant. This also means that when sharing spatial position, we should be sending relative values based on the scene configuration, so that we see the appropriate amount of positional change no matter the scene size of each participant, instead of sending static values.
This difference in logic should be considered when migrating platforms to understand where in your code you will need to manage your positional changes. For this guide, we will focus on migrating away from the audio-only social spaces typical of High Fidelity as an example to build off of.
Enable Spatial Audio capabilities
Now that your conference has been set up, we can add in the Spatial Audio capabilities. To do this, we will follow the documentation to enable Spatial Audio for your conference. Note that Dolby Voice must be enabled for Spatial Audio to work. We can modify our conference creation methods with the following parameters:
// Create a Dolby Voice conference
const createOptions = {
alias: "spatial",
params: {
dolbyVoice: true
}
};
const conference = await VoxeetSDK.conference.create(createOptions);
// Join the Dolby Voice conference using spatial audio and requesting stereo on the downlink
const joinOptions = {
preferRecvMono: false,
spatialAudio: true
};
await VoxeetSDK.conference.join(conference, joinOptions);Configure the Spatial Audio scene
Continue along the documentation to configure the scene and scale for Spatial Audio to be heard in. This is going to help us determine the size of the workspace being used, as well as how volume scales as distance between participants changes, as we are already working with visual elements from the cloud conferencing. The environment configuration lets the developer continue to work with the positions that are being used to draw the participants. The developer does not need to convert to another coordinate system.
A typical use case is setting the scale based on the window of the application being used to ensure everyone will be heard, as a participant who is one meter away will be at full volume. A participant who is 100 meters away will not be heard. The scene will help determine what default directions people will be heard in, so making the top of the screen be heard from in front, and so on. The following example will create a scene with dimensions 4 meters by 3 meters.
// Prepare the audio scene
// window.innerWidth and window.innerHeight give me the dimensions of the window, which we then convert to a 4x3 meter scene.
const scale = { x: window.innerWidth / 4, y: window.innerHeight / 3, z: 1 };
const forward = { x: -1, y: 0, z: 0 };
const up = { x: 0, y: 0, z: 1 };
const right = { x: 0, y: 1, z: 0 };
VoxeetSDK.conference.setSpatialEnvironment(scale, forward, up, right);Change spatial direction
If your application has dynamic positioning of speakers and/or listeners, you might want to be able to change the Spatial Direction. This lets us rotate the user, so that we can keep participants in the same position, while changing the direction that they face spatially. To do this, we can use the setSpatialDirection() method.
In High Fidelity, you may have used the HighFidelityAudio.Quaternion() method, for example:
const myDirection = HighFidelityAudio.Quaternion.fromEulerAngles({ "pitchDegrees": xDirection; "rollDegrees": yDirection; "yawDegrees": zDirection }),
hifiCommunicator.updateUserDataAndTransmit({
position: HighFidelityAudio.Point3D());,
orientation: myDirection,
}));While in Dolby.io:
const spatialDirection = {
x: xDirection,
y: yDirection,
z: zDirection,
};
VoxeetSDK.conference.setSpatialDirection(VoxeetSDK.session.participant, spatialDirection);Setting spatial position
Setting the position of a participant is how we can determine where a participant in placed in the scene, so we can see how the audio changes in relation to others.
For example, in High Fidelity this could look like the following for a single participant, where we only need to modify the position once, as the scene is shared between all participants:
const myPosition = HighFidelityAudio.Point3D({ "x": xPosition, "y": yPosition, "z": zPosition }));
hifiCommunicator.updateUserDataAndTransmit({
position: myPosition,
orientation: HighFidelityAudio.Quaternion(),
}));While in Dolby.io, we need to set the positions for all the participants. Here is how we would set the position only for the local participant:
const spatialPosition = {
x: xPosition,
y: yPosition,
z: zPosition
};
// Update position locally
VoxeetSDK.conference.setSpatialPosition(VoxeetSDK.session.participant, spatialPosition);
// Send position with command service.
await VoxeetSDK.command.send({
action: 'updatePosition',
participantId: VoxeetSDK.session.participant.id,
position: spatialPosition
});To set the position for all users, each user needs to send their position to the other users. See Communicating spatial changes to other users below for examples of how to do this.
Communicating spatial changes to other users
As mentioned earlier, we will need to communicate these spatial environment changes to other users, as Dolby.io uses a unique audio scene per participant, instead of a shared one. We also need to make sure we are sending the relative position over so that it can be properly converted to the correct scale based on the participant’s scene. There a few ways to do this, and the correct one will likely depend on your project’s environment. Here are a few examples:
Cloud
If this is a cloud hosted environment, your backend might already have all the positions for the participants, which you can then use to get the position updates natively by referencing that database. Set the positions manually accordingly that way.
Command Service
If you are not storing the positions already, we can use the Dolby.io Command Service instead. The concept here is the following:
- A participant’s spatial position changes
- This event triggers a command message to all participants
- This message sends a data payload to all participants with the updated position
- Participants listen for Command Service messages, and update the position accordingly
In a code example, this might look like the following to send a Command Service update, containing the participant that changed position and the position changed in the payload:
// Note: you can optionally add in UI position changes in payload as well
await VoxeetSDK.command.send({
action: 'updatePosition',
participantId: VoxeetSDK.session.participant.id,
position: {
x: relativeXPosition,
y: relativeYPosition,
z: relativeZPosition
}
});And then to receive it:
VoxeetSDK.command.on("received", (participant, message) => {
const jMessage = JSON.parse(message);
if (jMessage.action === 'updatePosition') {
const participantId = jMessage.participantId;
const position = jMessage.position; // Convert position to scale as needed
const participant = VoxeetSDK.conference.participants.get(participantId);
console.log(`Set Spatial Position for ${participantId}`, position);
VoxeetSDK.conference.setSpatialPosition(participant, position);
// Update the position on the UI here
}
});As seen from our Spatial Audio demo here.
You can also see an example of how the Command Service works here.
Troubleshooting
If you are running into issues, we have a frequently asked questions page you can find here. We also suggest taking a look at our Models for a more in depth explanation of how to configure Spatial Audio in your application.
Next Steps
Dolby.io is here to support the High Fidelity community for all of your Spatial Audio needs. We hope this guide provides you a good starting point for migrating your High Fidelity application to Dolby.io.
Our team is available to assist in your process. If you have any questions or run into any roadblocks, please contact our sales or support team – we’d love to help.
If you would like to see an example of how an application might put all of these together, take a look at our demo app here. And to read more, see the new Spatial Features enabled in our new SDK version here. Happy building!
