Enabling a Seamless Multimedia Experience with RemoteFX Media Streaming in Windows Server 2012 and Windows 8

Hi, I am Alvin Lau, a Software Development Engineer in Test (SDET) from the Remote Desktop Virtualization team. Today I want to talk about how Windows Server 2012 and Windows 8 enable a seamless multimedia experience for all content types on all networks.

Previously, Windows Server 2008 R2 and Windows 7 introduced Windows Multimedia Redirection, which provided a great experience when playing multimedia content by using Windows Media Player (WMP) in a remote session. Since Multimedia Redirection was based on redirecting the native media stream to the client, the experience was great for supported video formats on LAN networks. However, there were a few key shortcomings:

1. Multimedia Redirection did not work for non-WMP content. Non-WMP content such as Flash, Silverlight, and QuickTime did not have the same experience as WMP content.
2. Multimedia Redirection required that the client computer be able to decode and render the original media stream. This limited the operating system that the client computer could run, and the client computer needed to be powerful enough to decode HD video files.
3. Bandwidth usage was dependent on the bitrate of the original video stream. Depending on the bitrate of the original video, the bandwidth usage could vary from as low as 400 Kbps to as high as 12 Mbps or more. In addition to the problem of unpredictable bandwidth, users might or might not have a good experience on bandwidth constrained networks—imagine sending a 5600 Kbps HD video down a 1000 Kbps WAN link.
4. The user experience for both WMP content and non-WMP content was poor in low bandwidth, high latency, and lossy networks—the frame rate was too low and the frame jitter was high.

To address these shortcomings, Windows Server 2012 and Windows 8 introduced RemoteFX Media Streaming, which uses host-side rendering techniques (which provide broad video format support) in combination with the industry standard H.264 codec (which benefits online media streaming) to seamlessly redirect video content.

RemoteFX Media Streaming works in 3 simple steps

1. On the server side, a combination of algorithmic techniques, heuristics, and application-provided hints are used to detect regions of the screen that contain video.
2. The regions that contain video are then encoded by using the H.264 codec (or the RemoteFX Progressive Codec, which is used when the H.264 codec is not available on the client, a video has a low frame rate, etc.). Audio content, if available, is encoded by using the AAC codec. The encoded audio and video content is sent over the network to the client computer.
3. Finally, the client decodes the audio and video content, attempts to synchronize the audio and video, and renders the content in a manner that is seamless with the rest of the graphics content (non-video applications, desktop, etc.).

This is a simplification of all the complexities in the system–but should give you an idea of how it works.

Benefits of RemoteFX Media Streaming

• Supports all video formats and media applications. RemoteFX Media Streaming detects and redirects any and all media content including but not limited to Silverlight, Flash, QuickTime, and HTML5 videos. Users have a consistent experience irrespective of video formats or video applications.
• Lowers bandwidth usage by up to 90 percent. Compared to Remote Desktop Protocol 7.0 in Windows 7, RemoteFX Media Streaming consumes up to 90% less bandwidth. The actual difference in bandwidth depends on the size and frame rate of the video.
  
• Adapts to changing network and client conditions. RemoteFX Media Streaming leverages RemoteFX Network Auto-Detect and client capabilities detection to dynamically change the encoding bitrate and frame rate to provide the best possible user experience.
• Delivers better frame rate in WAN networks. Thanks to adaptive media streaming and the fact that RemoteFX Media Streaming uses the User Datagram Protocol (UDP), users even on WAN networks have a very good experience. The following chart shows the frame rate that we see in our tests on some of the WAN network configurations that we test with.
  

In summary, RemoteFX Media Streaming delivers a smooth playback experience for all video content even on WAN networks.

Take a look at the following video which shows Windows 7 and Windows 8 remote desktops side by side. In both cases the client computer is connected to the remote desktops by using a WAN link with 2 Mbps bandwidth, 250ms round-trip latency, and 0.5% random loss.

Does Windows Server 2012 still support Multimedia Redirection?

Windows Media Redirection still provides the best experience on LAN networks for supported video formats. Windows Server 2012 and Windows 8 automatically detect these conditions and turn on Multimedia Redirection. If these conditions are not met (for example, if the user is on a WAN network), the system automatically switches to use RemoteFX Media Streaming, even for WMP content. The determination of whether or not the user is on a WAN network is made by looking at the end-to-end latency—if the connection latency is above 30ms, Multimedia Redirection will not be used.

Configuration requirements

RemoteFX Media Streaming is enabled along with the rest of Remote Desktop Services, and no additional configuration is required. This is in line with our core philosophy for RemoteFX in Windows Server 2012, which is to make things simple and have features just work out of the box. However, if a user is connecting to a virtual desktop that has a RemoteFX virtual graphics processing unit (vGPU) installed, the host Hyper-V server must have a non-server core setup due to the fact that media codecs are not available in a server core setup.

Thanks for reading, and I hope you find this post useful. I am looking forward to your feedback.

If you have specific troubleshooting questions, go to the RDS web forum at http://social.technet.microsoft.com/Forums/en-US/winserverTS/threads; many of my fellow product group members and community experts watch and respond to questions on the web forum on a regular basis.