UPDATE: Unreal Engine WebRTC Player Plugin for Millicast is now publicly available: https://docs.dolby.io/streaming-apis/docs/unreal-player-plugin
Looking for a guide on building with the Dolby.io Millicast Plugins? Check out this Unreal Engine 5 Streaming Plugin tutorial.
This plugin enables you to play a real time stream from Millicast in your Unreal Engine game. You can configure your credentials and configure your game logic using an Unreal object and then render the video in a texture2D.
WebRTC is, hands-down, the streaming protocol of choice for creating real-time interactivity online. It’s highly adaptable, open source, offers end-to-end encryption, and can deliver less than 500 ms of latency to any browser and any Android or iOS device-no client-side plug-ins or software needed.
So, it was no surprise to us that Epic Games chose to add WebRTC support Unreal Engine with their Pixel Streaming Plugin.
Unreal Engine delivers real-time, interactive streams with WebRTC
Unreal Engine is a powerful real-time 3D creation tool that enables creators across industries to deliver cutting-edge content, interactive experiences, and immersive virtual worlds. These industries include Gaming, Architecture, Film & Television, Broadcast & Live Events, Animation, Product Design, Simulation and much more.
Traditionally, users experience their Unreal Engine application on the same local device that runs the gameplay logic and renders the game world to the screen.
With the Pixel Streaming Plugin, users can run their Unreal Engine application remotely, encoding the rendered output into a media stream that can be broadcast and viewed through standard web browsers and devices in real-time using WebRTC. Users can also control the experience from their browsers adding interactivity through keyboard, mouse, and touch events emitted from the player web page back to Unreal Engine.
Sounds amazing. What’s the catch?
Well, no catch. Unless you need the ability to reliably scale the WebRTC stream to dozens, hundreds or even thousands of viewers around the world. WebRTC has historically not been able to scale as it was focused on voIP and Peer-to-Peer use cases which limited viewership to only a few concurrent viewers.
That is exactly why we built Millicast. We envisioned the need for a simple service that offers the fastest, most reliable and highest quality WebRTC streaming across the globe, all through a simple API.
So now all we need is the ability to connect the output from the Pixel Streaming Plugin to Millicast that can serve WebRTC at-scale. Easier said than done, since media ingest with WebRTC has always been a mess. The lack of a standard signaling protocol has made it difficult to implement end-to-end workflows for WebRTC. Imagine offering your hand to someone and expecting a shake, but receiving a fist bump or a high five instead. The new WebRTC HTTP Ingest Protocol (WHIP), developed by our CTO Sergio Garcia Murillo, puts an end to the confusion. WHIP hides all the WebRTC complexities, providing a way for encoders and media servers to know exactly the kind of “handshake” to expect. No more awkward fist bumps catching you on the chin.
Pixel Streaming + WHIP + Millicast = GAME OVER
Millicast recently added support for WHIP inside Unreal Engine’s Pixel Streaming Plugin, enabling new exciting remote use cases that will make interactivity at-scale possible inside Unreal Engine worlds.
The Pixel Streaming plug-in running inside Unreal Engine establishes a connection using WHIP with the Millicast servers. The Millicast servers then deliver standard WebRTC with near-zero latency to remote viewers through their web browser or a client-side SDK. Streaming starts immediately and can scale to over a million viewers as needed.
This video demonstrates WHIP’s low-latency performance with side-by-side screens. On the left you see Unreal Engine connecting locally, and on the right is the WebRTC/WHIP stream going from Madrid to US-West and back in less than 200 milliseconds. The Unreal Engine content is being rendered off-screen, streamed via WHIP to Millicast servers, and controlled via an external Pixel Streaming client.
It’s as simple as that. Again, your viewers do not need to install or download anything. That’s the beauty of the WebRTC standard, the Millicast service and WHIP’s simple HTTP request/response approach.
We have also built a MillicastPlayer Plugin to enable playback of a WebRTC stream inside of Unreal Engine. Now it’s possible to capture WebRTC in your web or mobile browser and send that stream through Millicast for playback within Unreal Engine. You can then render that to a texture for simple playback within your game environment.
Making remote experiences feel real
Unreal Engine, WHIP and Millicast combine to enable real-time collaboration between co-workers, clients, and friends anywhere, making the remote experience real. Everything from designing video game levels with dispersed teams to customizing architectural plans live during client meetings to customers like Precision OS that exemplify how virtual reality is changing the face of medical education through faster, more thorough, more portable training for new surgeons:
The benefits include:
- Sub-second latency: Millicast ingests WHIP and delivers standard WebRTC with latency between 200–500ms end-to-end, so viewers experience streaming events truly in real time.
- Universal compatibility: Viewers can play Millicast streams in any browser or device that supports WebRTC-with no client-side software or plug-in required.
- Immersive interactivity: Unreal Engine users can view real-time streams at-scale, and also interact with Unreal Engine via keyboard, mouse, touch, and even their own custom HTML5 UI.
- Secure streaming: Enable end-to-end encryption on the Millicast platform to protect your streams from being “hijacked” on the network, and use the Subscribe token to restrict viewership.
Want to explore how you can use Unreal Engine, the PixelStreaming Plugin and Millicast for your own streaming project? Contact the Millicast team.
