Microsoft RemoteFX: The problem we are solving …

Microsoft RemoteFX: The problem we are solving …

  • Comments 17

My name is Tad Brockway.  I am the Product Unit Manager for the Remote Desktop Virtualization (RDV) Calista (now RemoteFX) engineering team. We’re located on the Microsoft Silicon Valley campus in Mountain View, California.  I have been passionate about desktop centralization for many years, going back even before I joined the Microsoft Remote Desktop Virtualization team in 1998.  Prior to joining Microsoft, I was a UNIX developer.  We didn’t call the scenario desktop centralization at that time.  We called it X Windows ;)

I feel fortunate to now be running the engineering team that resulted from the acquisition of Calista Technologies in January of 2008.  This is the engineering team that is delivering RemoteFX, a set of technologies that will change the way that the industry thinks about Windows desktop centralization, remote graphics, and thin client computing scenarios.  This blog post explains our goals for remoting, where we started, and the basic approach we’ve taken.

The Promise of VDI

The promise of Virtual Desktop Infrastructure (VDI) is that end-user desktops can be centralized in such a way as to move complexity and state from the desktop into the datacenter. To execute on this promise, we needed to allow people to use a broad range of endpoint devices without compromising on the end-user experience. To this end, we are developing a remoting approach that complements traditional graphics remoting capabilities and works for endpoint devices ranging from PCs to the most lightweight of thin clients.

Client-Centric Remoting

Traditionally, graphics remoting protocols like RDP have approached remoting in a client-centric way.  These protocols intercept graphics on the host device and then efficiently forward the intercepted graphics ‘primitives’ (e.g., “Draw Rectangle”, “Draw Line”) to the client device.  The client endpoint renders the primitives using a client-side counterpart for each graphics intercept point on the host.   

This style of remoting is client-centric because the architecture relies heavily on the rendering capabilities of the client software and hardware.  There are benefits to client-centric remoting.  Chiefly, the bandwidth utilization is very good for graphics types that can be intercepted high in the software stack and sent as primitives.  But, when the client and host don’t both support a particular graphics type, either the application fails to run properly or the two sides negotiate down to a least common denominator graphics construct: a bitmap.  Bitmaps require more bandwidth than primitives.  For example, the primitive representation of ‘Draw Line’ would simply include the x, y coordinates for the line start and the line finish. The bitmap representation of the line would have to describe, minimally, the X and Y coordinates for every single point on the line.

If you have a powerful client device with a rich software stack and your host has all the right graphics intercept points, you can have a great user experience over a relatively low-bandwidth connection with a client-centric graphics solution.  But, if you have a less complex client device and/or are missing some important graphics intercept points on the host, client-centric remoting will result in gaps in the experience, such as choppy video or missing graphics.

Client-centric remoting originated when there was limited bandwidth from the datacenter to the end-user desktop and when the vast majority of applications were developed on top of the same Windows graphics APIs, GDI

Host-Centric Remoting

Today, bandwidth is less expensive and in many places widely available.  Today’s modern Windows desktop includes rich media and 3D graphics content.  Additionally, a wide array of graphics types (for example, Silverlight, Adobe Flash, DirectX, Aero Glass, Windows Media, etc.) is now relevant to Windows users. These changing conditions call for the addition of a new model that can support all graphics types, including 3D, by sending highly compressed bitmaps to the endpoint device in an adaptive manner. We call this host-centric remoting.

You can ensure a consistent end-user experience for a wide array of devices if you follow the VDI model and enable movement of a large portion of the client software and hardware into the datacenter. With host-centric remoting, all the graphics can be intercepted on the host at a very low layer in the software stack. All graphics are rendered on the host into a single frame buffer (a temporary holding station for graphical updates) that represents the end-user display.  Changes to the frame buffer are sent to the client at a frame rate that dynamically adapts to network conditions and the client’s ability to consume the changes. The changes are sent to the client endpoint as highly compressed bitmaps by using an encoding scheme optimized for Windows desktop content. The basic graphics requirement for the client endpoint is that it supports the ability to decode and display the highly compressed bitmaps that it receives from the host. At a minimum, the client needs the decoder counterpart to the encoder that was used on the host as well as a basic graphics display capability.

The downside to host-centric remoting is that it requires more bandwidth than client-centric remoting. However, it delivers a consistent experience for every aspect of the modern Windows desktop projected out to an amazing diversity of client devices.

It’s Additive!

If you have a client device with a rich software stack and advanced processing capabilities, client-centric remoting makes sense. But, to completely deliver on the promise of VDI, you also need host-centric remoting.

RemoteFX is Microsoft’s first big step into the world of host-centric remoting. But, the obvious relationship between client-centric remoting and host-centric remoting is that you need both sets of capabilities in your remoting protocol. We are adding RemoteFX as a new capability or ‘payload’ to the RDP platform, while continuing to support and enhance our existing client-centric model. We offer our customers the best of both worlds in one solution. Host-centric remoting takes advantage of environments with ample bandwidth for content- and device-independent remoting, and client-centric remoting works well for network-constrained environments with richer, more powerful client devices. The fundamentals of RDP are unchanged. RDP includes the same authentication, encryption, device redirection, and transport capabilities independent of the remoting model being leveraged.

This has been a quick overview to explain our goals and the models we’ve employed to meet them. Please check back with this blog as we will talk more about what makes RemoteFX as part of RDP so special. 

Leave a Comment
  • Please add 5 and 7 and type the answer here:
  • Post
  • Do you now a time.line when RemteFX will be available in WIndows 7 as a host? So I can use the new RemoteFX to make a RDP connection to my Windows 7 client...

  • 1. How to join RemoteFX partner program? what's the advantage?

    2. Any more detailed tech. documents on RemoteFX, e.g., what's the requirement of client beside your specific decoder? can we define and use our own codec in host/client side?


  • With the opportunity to update RDP 7 clients now, please support Server 2003 and XP x64. Unless the TS team is dropping even 32-bit XP support like the IE9 and Windows Live teams?

  • Does this mean high-end video cards on the server?

  • When does this RemoteFX on RDS session host server work? Do you have to virtualize the session hosts under Hyper-v or does it work on bare metal without Hyper-v?

  • I am so happy that RDP will address this problem, however it needs to take care of client who are low on the bandwidth and have latency. Will RDP have lossful compression? I have seen some specs about this but till now the data compression is always lossless.

    I also would like to know how to join RemoteFX partner program and what are the advantage?

  • Ericom Software, a provider of Terminal Services and VDI solutions, plans to support Microsoft RemoteFX when it’s

    released by Microsoft, as it will complement Ericom Blaze RDP acceleration. Ericom Blaze accelerates RDP up to 25x

    and is ideal for remote users connecting over slow WANs who need to access graphics-rich content and applications.  

    RemoteFX delivers a premium user experience for LAN users accessing rich media content and 3D applications, from

    virtual and session-based desktops.

    The combination of RemoteFX and Blaze within PowerTerm WebConnect - Ericom's unified VDI and terminal services access

    solution will deliver the best user experience for all types of users and connections.

    Read more about Blaze and download a free evaluation at:

    Or view a video demo at:


  • I didn't see any coverage of directdraw, and how it's handled over newer RDP.

    How does DD work in the newer RDP?

    My problem now: XP Pro sp3 client (or win7 for that matter).  Server 2003 r2.  App on host-server processes camera video.  App on host-server requires hardware directx support (YUV surface format), no RGB fallback (color conversion cost is too high).

    Question:  Does the newer RDP support YUV offscreen surfaces?

    I suppose this will still be too slow, going by the DX TS example I've come across (block bouncing around in a square), even if the directdraw emulation does YUV format.   DD is all the application needs, since it's simply blt'ing a YUV frame.  I don't see how D2D could improve that limited and specific need.

    The current plan is to bypass RDP completely and use TCP (http specifically since this could be over a WAN) to get the YUV data from the server app to the client, instead of RDP.

    I'll repeat the only real question:

    Question:  Does the newer RDP support YUV offscreen surfaces?  I only see emulated DD being used when in a TS session. (Hardware DX9 is there on the server, as is the driver - it all works fine when not in a TS session).

  • What OS and GFX capability is required on the endpoint for the SW decode approach?  Are there specific DX requirements?

  • Yeah when will remote fx be availble for windows 7 as a host!!!!!!!

    Server 2008 is all very well for buisiness but I want play games on my PC media center in my living room on my plasma with an xbox 360 contoller redirected. I want my PC in my study with all my games installed on to be a the host! I don't want Server 2008 and nor do I need it.

  • "Does this mean high-end video cards on the server"

    I'm current researching this as well. As stated in the handbook "Top 5 Reasons to Upgrade" for Remote Desktop Services: "Enhanced Bitmap Acceleration – 3D applications including DirectX 9, 10 & 11 applications, as well as rich media content such as Silverlight or Flash WILL RENDER ON THE SERVER and will be remoted using enhanced bitmap acceleration. No matching graphics stacks are required on the client."

    I take this to mean the server must be able to support the graphics requirements for the application in question. The client pc doesn't require the same graphics capabilities. However, in my own 'testing' of RemoteApp; I'm attempting to use RD Web Access with new high-end 3D graphic games.

    All related RD services seem to be functioning fine, and are properly licensed. After installing the game to the server, I'm able to successfully add the game to the RemoteApp Programs List. From a client machine, I log into the RD Web Access site and click the icon for my game. I can see it start to launch, the splash screens appear like the game is starting to run, then I run into errors... ALl errors are graphics related and specific to the game im launching.  Given that the Handbook says, "WILL RENDER ON THE SERVER" I'm guessing this is my problem because my Server doesn't have a designated graphics card capable of running the game.

    I've been testing this out for 2 days. Installing/uninstalling/reconfiguring. I've looked up the error messages and codes on various forums but no posts really seem to fit this situation. As best as I can tell, at this point, it seems my errors are caused by the lack of graphics support on the server.

    I have played both test games on ther CLient PC just fine when installed locally as normal.

    SERVER SPECS: TYAN S2912,, 2x(Opteron (dual-core),, 2x(4GB per cpu),,Windows Server 2008 R2 Ent. x64, Integrated 32mb Grfx,  No Graphics Card.

    Client PC: ECS BLack Ed. mobo,, AMD Phenom 3-core,, 6GB ddr2,,Geforce 250 GTS 1GB,, WIndows 7 Ult. x64

  • After a little more research I found something nice on "Windows Server TechCenter" from

    "...Windows Server 2008 R2 with SP1 has been tested for up to 12 virtual machines per GPU, for a total of 24 virtual machines on two physical GPUs, providing the necessary dedicated video memory..."

  • At first i have to say that I'm a very keen User and Reseller of Hyper-V Server since the first release

    BUT !


    Please do not deaktivate the local Hyper-V Server Screen after installing RemoteFX. I had the Problem that RDP was not responding a few days after activating RemoteFX.

    The complete Access to all Hyper-V over Network was lost !

    The only way to gain Access to the Server was to deinstall RemoteFX from the Recovery Console of the HyperV - Server. Which was very tricky....

    I costs about 2 Day to regain access to the Hyper-V Server !

  • At first i have to say that I'm a very keen User and Reseller of Hyper-V Server since the first release

    BUT !


    Please do not deaktivate the local Hyper-V Server Screen after installing RemoteFX. I had the Problem that RDP was not responding a few days after activating RemoteFX.

    The complete Access to all Hyper-V over Network was lost !

    The only way to gain Access to the Server was to deinstall RemoteFX from the Recovery Console of the HyperV - Server. Which was very tricky....

    I costs about 2 Day to regain access to the Hyper-V Server !

  • Issues with my virtual remote desktop session host server not picking up the synthetic RemoteFX driver.

    From the hypervisors side everything looks good and i can add the RemoteFX adapter in Hyper-V Manager

    The virtual machine starts fine, but i have two unknown devices in the device manager with a code 28

    Both Hypervisor and VM are Server 2008 R2 SP1 Enterprise

    I should mention that I'm running vWorkspace on the VM as well


Page 1 of 2 (17 items) 12