The Dolby.io team has been speaking with a number of people about how they work with media and how they create media data. These folks have ranged in skill from beginner to advanced but share a common appreciation for high-quality audio. The beginner may still be interested in programmatic content classification, clipping detection, and loudness. The more advanced user may be trying to solve a very specific audio workflow problem such as evaluating the signal-to-noise ratio as a time series or extracting Musical key, instrument, and genre identification from a music file.
It is with this range of users and use cases in mind that the Media Analyze API exists.
What Does It Do?
The Analyze API takes media as input and programmatically generates audio data from the underlying signal, creating datasets that consist of the following information:
- General media Info
- Clipping sections
- Loudness (time-series)
- Bandwidth
- Signal-to-noise ratio
- Content classification (time-series)
- Musical key, instrument, and genre identification
The API adheres to the principles of RESTful architecture, meaning it is language and platform agnostic. For example, sign up for a Dolby.io developer account to get your API Key and you can run something like this:
curl -X POST https://api.dolby.com/media/analyze
--header "x-api-key: $DOLBY_API_KEY"
--data '{
"input": "https://archive.org/download/OrsonWellesMrBruns/381030_64kb.mp3"
}'You’ll get a response back with a job_id:
{"job_id":"b6ab4237-9109-4acb-82f4-5545ca8cd6fd"}This is an asynchronous task meaning your audio is being programmatically analyzed on the Dolby.io cloud. When it’s ready, we’ll be able to pick up the results. You do that by passing the same job id back to the API.
curl -X GET https://api.dolby.com/media/analyze?job_id=$YOUR_JOB_ID
--header "x-api-key: $DOLBY_API_KEY"We’ll get a JSON response to take a look at. What have we learned?
Check Basic Media Info
This section includes details about the container such as duration, kind, and size. Insight into the audio structure like bitrate, channels, codec, and sample rate. There are many tools that do this, but with the Analyze API you have a powerful option for mobile applications or platform integrations.
You get back a JSON response with audio and container attributes defined like the following:
"media_info": {
"container": {
"kind": "mp4",
"duration": 10801.645,
"bitrate": 79674,
"size": 107575636
},
"audio": {
"codec": "aac",
"channels": 2,
"channel_order": "L R",
"sample_rate": 44100,
"duration": 10801.621223993765,
"bitrate": 78286
}
}Review Loudness Profile
To deliver content that conforms to broadcasting standards like A/85 and R.128 you may need to do some analysis of your media or user-generated content. If you are delivering to popular media platforms like Amazon, Apple, SoundCloud, Spotify, Vimeo, and Youtube you can prepare your content for the best results and get a straightforward pass/fail validation test.
If you want to know if your audio is within range, the JSON response like this may be useful:
"loudness": {
"measured": -15.27,
"range": 4.31,
"gating_mode": "speech",
"sample_peak": -0.0,
"true_peak": 0.07,
"time_series": [
[
0.0,
-120.0,
-4.23,
-4.22
],
[
1.0,
-120.0,
-8.06,
-7.95
],
...
]Identify Noise, Clipping, and Silence
To understand how easy it is to listen to your content, it can be useful to understand the level of noise present. If you are mixing content from many sources it can be important to understand if there is silence as a lead-in, at the end of a recording, or periodically placed markers throughout.
If there is a lot of silence or clipping you may need to make some audio cuts and run noise suppression to reduce static background noise:
"bandwidth": 11197
"noise": {
"snr_average": 82.42,
"level_average": -101.87
},
"clipping": {
"num_sections": 0,
"sections": []
},
"silence": {
"processed_region": {
"audio": {
"silence": {
"percentage": 5.3,
"num_sections": 2,
"sections": [
{
"section_id": "si_1",
"start": 3.08,
"duration": 2.08,
"channels": [
"ch_0"
]
},
{
"section_id": "si_2",
"start": 8.3,
"duration": 3.2,
"channels": [
"ch_0",
"ch_1"
]
}
]
}
}
}
}Is the Dolby.io Analyze API for you?
We certainly hope so and if there is more data you are looking to learn about your media let us know. Our Analyze API tells you much more about your audio than just the metadata.
If you are ready to include analysis as part of your quality control process in a media workflow, a gating step for user-generated content, or to help search across an archive of media this API can help.
Pick your favorite client language and library and get started with the Analyze Media Quick Start guide which will walk you through the steps or check out this blog here which highlights how you can use Analyze for a Serverless Media Workflow.
