Handling Multi-touch and Mouse Input in All Browsers

IEBlog

Windows Internet Explorer Engineering Team Blog

Handling Multi-touch and Mouse Input in All Browsers

  • Comments 19

Touch interaction with Web sites and apps has the opportunity to improve their usability and ubiquity as the Web and Windows 8 Metro style apps play a key role on tomorrow’s touch-enabled devices.

This post explains how Web developers can use the new IE10 pointer event model along with the iOS touch event model and the W3C mouse event model (as extended) to create a cross-browser, common-code handler for pointer, touch, and mouse input.

A little background: I’m fortunate to have a Samsung 700T Windows Developer Preview tablet PC. With it, I’ve been able to enjoy the IE Test Drive multi-touch demos Touch Effects and Lasso Birds. Like me, you may have noticed that Lasso Birds works across different devices and browsers in addition to IE10. For example, its multi-touch works on iOS devices. In this post, I’ve taken some of the patterns from Lasso Birds and generalized and extended them to include older versions of browsers.

The result of my experimentation is below. It should work in your browser. A discussion of the coding patterns and lessons learned follows below the demo.

The Code

The basic algorithm for drawing with the mouse model is straightforward:

var drawingStarted = false;

function DoEvent(eventObject) {

if (eventObject.type == "mousedown") {

drawingStarted = true;

startDraw(eventObject.pageX, eventObject.pageY);

}

else if (eventObject.type == "mousemove") {

if (drawingStarted) {

extendDraw(eventObject.pageX, eventObject.pageY);

}

}

else if (eventObject.type == "mouseup") {

drawingStarted = false;

endDraw();

}

}

The only change needed to make this work with IE10’s pointer events is to add awareness that multiple pointers can be down at the same time, each identified by a different pointerId value. The IE10 pointer model fires separate MSPointerDown, MSPointerMove, and MSPointerUp events for each pointer that changes state.

var drawingStarted = {};

function DoEvent(eventObject) {

// the following is needed in developer preview to stop implicit pan/zoom

if (eventObject.preventManipulation)

eventObject.preventManipulation();

 

var pointerId = eventObject.pointerId;

if (eventObject.type == "MSPointerDown") {

drawingStarted[pointerId] = true;

startDraw(pointerId, eventObject.pageX, eventObject.pageY);

}

else if (eventObject.type == "MSPointerMove") {

if (drawingStarted[pointerId]) {

extendDraw(pointerId, eventObject.pageX, eventObject.pageY);

}

}

else if (eventObject.type == "MSPointerUp") {

delete drawingStarted[pointerId];

endDraw(pointerId);

}

}

Adapting the original mouse model to Apple’s iOS touch event model requires that you iterate through the list of changedTouches for each touchstart, touchmove, and touchend event because, in the iOS model, state changes that occur at the same time are bundled into one event. Like the IE10 pointer model, a unique identifier identifies each touch point.

var drawingStarted = {};

function DoEvent(eventObject) {

eventObject.preventDefault(); // without this, instead of drawing, you pan

for (var i = 0; i < eventObject.changedTouches.length; ++i) {

var touchPoint = eventObject.changedTouches[i];

var touchPointId = touchPoint.identifier;

if (eventObject.type == "touchstart") {

drawingStarted[touchPointId] = true;

startDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

else if (eventObject.type == "touchmove") {

if (drawingStarted[touchPointId]) {

extendDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

}

else if (eventObject.type == "touchend") {

delete drawingStarted[touchPointId];

endDraw(touchPointId);

}

}

}

Merging these three individual algorithms requires noting the differences between the event names and the unique pointer identifier attribute names and accounting for the lack of an identifier in the mouse model.

In the merged model, below, I also add a check that the “move” position has actually changed because the IE10 pointer model streams MSPointerMove events with the same x, y position when a touch point is held down but not moved. By filtering out these redundant moves, I eliminate calls to extendDraw() that do nothing. I implemented this check by storing the last x,y position from a start or move in the lastXY object and, by checking that a lastXY entry exists for a particular id, lastXY replaces the drawingStarted object used in the previous two examples.

var lastXY = {};

function DoEvent(eventObject) {

// stop panning and zooming so we can draw

if (eventObject.preventManipulation)

eventObject.preventManipulation();

 

// we are handling this event

if (eventObject.preventDefault)

eventObject.preventDefault();

 

// if we have an array of changedTouches, use it, else create an array of one with our eventObject

var touchPoints = (typeof eventObject.changedTouches != 'undefined') ? eventObject.changedTouches : [eventObject];

for (var i = 0; i < touchPoints.length; ++i) {

var touchPoint = touchPoints[i];

// pick up the unique touchPoint id if we have one or use 1 as the default

var touchPointId = (typeof touchPoint.identifier != 'undefined') ? touchPoint.identifier : (typeof touchPoint.pointerId != 'undefined') ? touchPoint.pointerId : 1;

 

if (eventObject.type.match(/(down|start)$/i)) {

// process mousedown, MSPointerDown, and touchstart

lastXY[touchPointId] = { x: touchPoint.pageX, y: touchPoint.pageY };

startDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

else if (eventObject.type.match(/move$/i)) {

// process mousemove, MSPointerMove, and touchmove

if (lastXY[touchPointId] && !(lastXY[touchPointId].x == touchPoint.pageX && lastXY[touchPointId].y == touchPoint.pageY)) {

lastXY[touchPointId] = { x: touchPoint.pageX, y: touchPoint.pageY };

extendDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

}

else if (eventObject.type.match(/(up|end)$/i)) {

// process mouseup, MSPointerUp, and touchend

delete lastXY[touchPointId];

endDraw(touchPointId);

}

}

}

The examples above specifically ignore the issues of registering to receive the events or ensuring that they apply to the drawing target. Making this work for real and with all browsers—including versions of Internet Explorer before IE9—requires a bit more work. Interested parties can peruse the final version of my multi-browser, multi-touch drawing class here.

Of particular note is a change between Windows Developer Preview and Windows Consumer Preview. To improve the performance of touch interactions, it is now required that you add to the intended touch target a CSS property that disables the default touch actions. The CSS syntax is:

#touchTarget {

-ms-touch-action: none;

}

In the code that powers the demo above, this property is set in JavaScript using code similar to:

if (typeof touchTarget.style.msTouchAction != 'undefined')

touchTarget.style.msTouchAction = "none";

A future blog post will describe the new -ms-touch-action property in more detail.

Code for All Input

By coding for touch alongside mouse, Web developers can assure their sites work with all browsers—whether desktop, tablet, or phone.

—Ted Johnson, Graphics Program Manager Lead, Internet Explorer

2 April 2012—
Updated this post to reflect changes between Windows Developer Preview and Windows Consumer Preview. I thank reader Ike Starnes for reminding me that this post needed updating.
—Ted Johnson

  • Loading...