RemoteFX Multi-Touch for Windows 8

RemoteFX Multi-Touch for Windows 8

Rate This
  • Comments 12

Hi, my name is Elton Saul. I am a developer on the Remote Desktop Virtualization (RDV) team. Our team is responsible for developing Microsoft RemoteFX in the Windows Server 2012 and Windows 8 release.

Windows 8 is designed to be used with a keyboard, mouse, and touch. However, from its inception through to the Windows 7 timeframe, RemoteFX has only supported the keyboard and mouse as input devices. Because Windows 8 will be used on devices without a physical keyboard and mouse, the ability to use remote touch over a RemoteFX connection is essential to ensure a fluid and seamless user experience when connecting to Windows 8 devices. For this reason, RemoteFX in Windows 8 now supports rich multi-touch remoting. This feature is known as RemoteFX Multi-Touch.

How does RemoteFX Multi-Touch work?

Rich multi-touch remoting involves more than simply sending touch gestures (such as tap, pinch, zoom, and pan) to a remote computer. Instead we transmit touch frames that contain rich metadata from client to server. Each transmitted touch frame describes the complete set of touch contacts that are active at a given moment in time (across all active touch digitizers), and includes details such as the on-screen location of each contact, the bounding rectangle associated with each contact, and timing information relative to the previous frame.

Advantages of RemoteFX Multi-Touch

What are some of the advantages of RemoteFX touch remoting?

  • Touch frames: Because all of the active touch contacts are sent remotely as frames, contacts in the remote session can be injected as a single logical group, resulting in strong cohesiveness and synchronization between injected contacts.
  • Contact rectangle: Sending the contact rectangle associated with a contact to the remote session can help the Windows targeting subsystem figure out what control the user was trying to touch.
  • Relative timing: Intra-frame timing information helps us to correctly pace touch frame injection on the server, resulting in more fluid touch interaction with remote objects.

How many simultaneous contacts can we send remotely from client to server? For Windows 8 we have a limit of 256 active contacts, which should be more than adequate for most humans. By the way, we also support touch remoting in multi-monitor Remote Desktop and RemoteApp scenarios. That means you can connect up to 16 touch monitors, and the touch interaction with each monitor will be sent remotely.

How to get started

To use remote touch over RemoteFX connections, you need to run the Remote Desktop Connection client (to establish a remote desktop or RemoteApp connection) on a Windows 8 operating system, or use the Windows Store Remote Desktop app, on a computer with a connected touch digitizer, and then simply connect to a Windows Server 2012 RD Session Host or RD Virtualization Host server running Windows 8 virtual machines.

Want to see a demo? Take a look at the following video that runs through some touch remoting scenarios:

We hope you enjoy using RemoteFX to get a great remote touch experience, and we look forward to hearing from you.

NOTE: Questions and comments are welcome. However, please DO NOT post a request for troubleshooting by using the comment tool at the end of this post. Instead, post a new thread in the RDS & TS forum at http://social.technet.microsoft.com/Forums/en-US/winserverTS/threads.

Leave a Comment
  • Please add 7 and 7 and type the answer here:
  • Post
  • For demo purposes we like to connect a Windows 7 computer with a mutli touch enabled input device to a windows 8 computer via rdp. Unfortunately gestures are not transmitted to the Windows 8 os. Would RemoteFX help us in our scenario? What has to be done on the Windows 7 device to get it running?

    SvenWS

  • @SvenWS

    RemoteFX Multi-Touch remoting only kicks in when connecting from a Windows 8 (or Windows Server 2012) system (which has a touch digitizer attached) to a Windows 8 (or Windows Server 2012) system.

  • Elton,

    Can you give me any insight on how you are capturing or hooking the touch inputs on the client machine?  I am trying to update some software that records user interaction to Windows8, and I would really like to capture this new form of user input.

    Wayne

  • The client uses the Windows 8 pointer input API, as described here: msdn.microsoft.com/.../hh454904.aspx

    This mostly just involves handling WM_POINTER* messages in a program's message loop. There is a little twist if you want to maximize the accuracy of your input capturing though; in some situations frames can be dropped if you're handling window messages alone, but using functions such as GetPointerTouchInfoHistory (msdn.microsoft.com/.../hh454891.aspx) you can get this information back (and more detailed data about the geometry of the touch inputs besides).

    Hope this helps!

  • Will there be support added for pressure metadata from styluses (Wacom, N-Trig, etc.) in an update to RemotFX.  Multitouch worked well, but the a stylus worked like a regular mouse pointer in OneNote MX in a test I did recently with N-Trig hardware on the client end.

    I was impressed though that a vanilla desktop VM install of Windows 8 automatically switched to tablet / touch mode upon RDP connection from a tablet device.  Glad to see those features mainlined.

  • So lets say I want to remote in via a tablet, like my Asus Transformer, using Log me in, or something. Could I still have touch input?

  • Let's say:

    I have a desktop that runs windows 8, but it doesn't have a touch-screen.

    I connect my tablet via remote desktop.

    Will I be able to control my desktop as if it had a touch screen? Or am I limited to just using my touch to control the mouse?

  • Nick's question is very interesting. Is touch data forwarded when using remote desktop from a Surface RT to control a physical desktop computer running Windows 8 Pro?

  • Remote to a windows 8 desktop from RT or windows 8 Tablet, perfect.

    Definitily want to be able to do that. Loads of legacy PCs upgraded to windows 8 will never have touch screens.

    When's the preview?

  • Will there be a client for android?

    Windows phones are horrible.

  • is posible to install, and where can download?? :))

  • Is there any way to turn off this feature? When I remote into my work PC from home, application-switching and tab-switching both exhibit latency of 3-5 seconds. I am almost certain this is the result of RemoteFX, since I can vastly improve performance by remoting into an intermediate computer that does not support this feature.

    To clarify what I mean, I remote from the home PC into a Windows Server 2008 session, and once there, remote into a Windows 8 machine. By using the double-hop, desktop performance (including app/tab switching) is fast.

Page 1 of 1 (12 items)