Most video conferencing solutions weren’t built for creative purposes. Tools like these were created with business meetings in mind, not providing interactive experiences for a creative community. If you want your app to stand out by providing a premium experience, the music capture mode with Dolby.io voice and video calls will certainly delight your fans.
Capturing Music in Web Browser Video Chats
Don’t Stop the Music
WebRTC provides cross-platform accessibility to users whether on a PC, Mac, or Linux without requiring a download or installation of software. This can be convenient for quickly getting on a voice call, but the default Opus codec is common for how audio is processed for voice over IP (VoIP) applications. With Dolby Voice, we’ve upgraded the experience and switched the underlying codec to be able to support many advanced audio features like spatial audio, noise suppression, and the recently announced music mode.
A new function was introduced in recent releases of the Dolby.io Communications Web SDK allowing you to call setCaptureMode(). This can be used with one of three different options or modes for how audio is captured: standard, unprocessed, or music.
The standard mode is good for voice discussions. Like a stereotypical parent, standard mode will stop that infernal racket by considering music to be part of the background noise. For use cases such as music lessons, interactive performances where somebody is singing or playing an instrument, or playing a pre-recorded performance the noise suppression is too aggressive.
The unprocessed value disables all special audio processing. If the creative intent is to just send the audio as captured, this might be a mode to choose for those use cases such as being captured in an ideal studio environment. The audio is unfiltered, unprocessed, and as natural sounding as the microphone and room acoustics allow.
The music value for capturing audio is something special. It not only protects the music from being affected by standard voice processing techniques, it enhances the music to create something even better for anybody listening to the call. The Dolby.io music mode will process the music live to boost and optimize your volume. You’ll hear every instrument with automatic dynamic EQ to adapt to the music being played, doing stereo widening to upmix and expand your sound beyond the mono audio capture of a typical microphone. There will be minimal noise suppression for environments that may have static noise like electrical hum, amp buzz, or fan noise, but handled to ensure music is captured and delivered in high quality.
The standard audio stream is 32kHz / 32kbps which is good for voice but with music mode a high quality stream is transmitted at 48kHz / 96kbps.
Getting Started with Music Capture Mode
Dream On
If you follow the Web SDK Getting Started Guide you can get a basic video chat up and running. That project was kept pretty basic, sticking with static HTML and pulling in the SDK from a CDN so that you could more easily adapt it to whatever framework or build tools you want to use in your own projects later.
Start by cloning the final source-code and running a basic HTTP server.
git clone https://github.com/dolbyio-samples/comms-sdk-web-getting-started.git
cd comms-sdk-web-getting-started
npx live-serverView the final/index.html and edit the file to add a setting to use the Dolby Voice Web Codec (DVWC). The dvwc setting is not enabled by default when joining a conference so must be turned on.
const joinOptions = {
- constraints: { audio: true, video: true }
+ constraints: { audio: true, video: true },
+ dvwc: true,
};This setting must be set to true for the client application where music is being captured and transmitted. It is recommended your users use a Chromium browser on the desktop for these capture sessions. The setting is not necessary for any client apps being used by participants who will listen and receive the high quality music audio stream.
Making a Music Capture Mode Toggle for Video Chat
Weapon of Choice
To do this, we first need to add an HTML control and the switch in Bootstrap is a common widget for this use case.
<div class="container px-4 py-4 mt-3">
<div class="form-check form-switch">
<input class="form-check-input" type="checkbox" role="switch" id="btn-music">
<label class="form-check-label" for="btn-music">Music Capture Mode</label>
</div>
</div>When the switch is on, we want to enable music capture mode. When the switch is off, we want to return to the standard behavior of boosting dialog and suppressing background noise. We can do this by adding an onclick handler that checks whether or not the switch is enabled and calls setCaptureMode() appropriately.
This should be put someplace in your main() function, such as immediately after handleConferenceFlow().
document.getElementById("btn-music").onclick = async (event) => {
if (VoxeetSDK.conference.current) {
if (event.target.checked) {
VoxeetSDK.audio.local.setCaptureMode({mode: "music"});
} else {
VoxeetSDK.audio.local.setCaptureMode({mode: "standard"});
}
} else {
// provide user with feedback and/or maintain state of toggle if not in a conference
}
}We’ve taken care to check that the session is open and a current conference has been joined, otherwise there will be no audio to capture. Similarly, in your final app you may want to set disabled on the btn-music element until somebody joins a conference to maintain a consistent state for your users.
What Dolby.io Music Capture Mode Sounds Like Turned On
I Love Rock n Roll
Here’s a sample of the before and after musical performances to give you an idea of the impact music capture mode has on performances to make a webinar, music lesson, or interactive music event really stand out.
Handling Echo Cancellation During Playback
Rhythm is Gonna Get You
When in a live video conference, a participant’s microphone can pick up audio coming through speakers and attempt to re-transmit the same signal back into the conference. Echo cancellation is a special type of processing that is enabled by default in Dolby.io sessions to prevent these audio echoes from happening. The additional audio processing required for this purpose can sometimes impact the quality of music. Depending on the specific environment and equipment in use by participants (we recommend headphones), it can be helpful to turn the echoCancellation off. This configuration is set in the modeOptions like this:
VoxeetSDK.audio.local.setCaptureMode({mode: "music", modeOptions: {echoCancellation: "off"}});There are some limitations on browser support for sending and/or receiving music streams, so see the music mode and troubleshooting guides for additional details on where and how you can best experience music capture mode.
Sound Check for Music Capture
Video Killed the Radio Star
Audio testing can be a bit tricky to test with video calls as a developer. While working on a proof of concept, you may want to consider recording the conference in order to hear the results of music capture mode. This approach can be quite handy when nobody else wants to join as a remote participant to listen to Christmas Carols throughout the day.
I sound much better with music mode.
Once More, With Feeling
Joy to the World
Focus attention on the audio that matters most for your use case. Dolby.io lets you decide what kind of audio capture you want to allow your users to use in a conference, whether voice, background effects, or music. Review the Music Mode Guide to learn more about how this feature works.
Take a look at some of our other Music solutions and add music capture mode to your next release. If you have specific browser compatibility or platform needs, reach out to our sales and product team to let us know what you found so we can share more from our roadmap and prioritize your needs.
