Eternal Coding

Using Visual Studio’s Javascript Memory Analysis tool to find memory leaks on your Windows 8 Javascript app

2013-04-24T08:24:28Z

Javascript is a wonderful language for developing Windows 8 apps. As a untyped, dynamic language, Javascript is really flexible but sometimes if you are not careful you may leave some memory leaks in your code.

But do not worry, Visual Studio includes a great tool called Javascript Memory Analysis in order to help you find out where the leaks are.

Using the Javascript Memory Analysis tool

This tool is available in the Debug menu of your Visual Studio:

Launching the tool will launch your app and will show this screen:

The first part of the window is dedicated to the real-time monitoring of the memory used by the application:

Right after you can find a button (Take Heap Snapshot) used to capture the instantaneous state of the memory heap (The control also indicates the size of the heap and the number of active objects):

By clicking on the heap size, a more detailed view is generated:

This view presents a sorted list of the objects by retained size, which means the objects consuming the most memory that are potentially easiest to free would be listed first.

For instance, in my app, I can see that the UrzaGatherer object retains almost 60 MB (which is normal because it handles a huge list of cards):

The view can also display:

a list of types sorted by occurrence count
a list of root objects (from the garbage collector point of view)
a list of DOM objects sorted by retained size
a list of WinRT objects sorted by retained size

All of these views can potentially help you find inconsistent memory usage.

Comparing snapshots

Each time you click on Take Heap Snapshot button, the system generates a new dump of the heap for further analysis. Furthermore, each snapshot indicates the difference of the heap size and objects count against previous snapshot.

This is the key for finding memory leaks.

For instance, in my app, I will take a first snapshot, navigate to a given page, then go back to home page and finally take a second snapshot. Doing this I will be able to see if the given page produces memory leaks.

We can see here that between the two snapshots, the heap size has grown of 6.12 MB and more than 53k objects were generated and still alive.

By clicking on the heap size difference (+6.12 MB), you can view a detail of the heap size compared to previous snapshot:

The Dominators view indicates that new objects remain when we come back to the home page. You can use this view to identify unwanted objects.

In my case, the first line seems suspicious. By right-clicking on it, I can go to the Root view to see which function retains my objects:

We can see that the object is retained by a function called ImportFromTappedOut.

After looking for this function in my code, it turns out that this function is a global function which takes a function as parameter:

ImportFromTappedOut = function (deckName, custom) {

The custom parameter is used by this function to call some specific user code. The main problem is that because the ImportFromTappedOut is a global function it will retain all the code referenced by the function affected to the custom parameter and in my case I used an anonymous function that references a lot of local DOM objects:

 Tools.ImportFromTappedOut("test", function () {
     var select = document.querySelector("#addToDeck");
     var count = document.querySelector("#addToDeckCount").value;

     listView.selection.getItems().then(function (items) {
         appBar.hide();
         appBar.sticky = false;
         listView.selection.clear();
…code continue…

And this exactly the problem described by the tool:

The solution is easy: I just have to call importFromTappedOut with a null function when I leave the page in order to allow the garbage collector to collect all retained objects (1.25 MB here).

I clearly encourage you to use this tool to detect and eradicate memory leaks. It is simple and really powerful and can help you get rid of memory problems.

More about the tool

Others articles you may find useful:

Now with Javascript: Reading PDF and XPS on your Windows 8 application using WinRT

2013-04-22T07:10:54Z

Following my previous article on “Reading PDF and XPS on your Windows 8 application using WinRT”, I would like to share with you the same example but with Javascript code.

Many of you sent me a mail to get this code, so feel free to grab it here (I would like to thank Mehdi Lahlou for his original version):

http://www.catuhe.com/msdn/js.PDFReader.zip

The principle is the same as for C#: I use a FlipView control that uses a custom template to generate.

The FlipView is nothing but a standard WinJS control:

 <div data-win-control="WinJS.UI.FlipView" id="flipView">
 </div>

To open the PDF file, we can use the following code (standard WinRT code):

openPDF: function(filename) {
    return Windows.ApplicationModel.Package.current.installedLocation.getFileAsync(filename).then(function(file) {
        return Windows.Storage.FileIO.readBufferAsync(file).then(function(fileBuffer) {
            return MuPDFWinRT.Document.create(fileBuffer, MuPDFWinRT.DocumentType.pdf, 
                               Windows.Graphics.Display.DisplayProperties.logicalDpi);
        });
    });
},

Please note that the MuPDFWinRT component used here is strictly the same as the one used by C#

Once the PDF is opened we can configure the FlipView using this code:

this.openPDF("test.pdf").then(function (pdfDocument) {
    var array = [];
    for (var i = 0; i < pdfDocument.pageCount; i++) {
        array.push(i);
    }
    var dataList = new WinJS.Binding.List(array);
    var flipViewDOM = document.getElementById("flipView");
    var flipView = flipViewDOM.winControl;
    var width = flipViewDOM.clientWidth;
    var height = flipViewDOM.clientHeight;
    
    flipView.itemDataSource = dataList.dataSource;
    flipView.itemTemplate = (function (itemPromise) {
        return itemPromise.then(function (item) {
            // root element for the item
            var canvas = document.createElement("canvas");
            
            canvas.style.width = width;
            canvas.style.height = height;

            var canvasContext = canvas.getContext("2d");

            that.drawPage(pdfDocument, canvasContext, item.index);
            return canvas;
        });
    });
});

Each page will be generated using a canvas stretched to fill the FlipView. Each canvas is then filled using this code:

drawPage: function (pdfDocument, canvasContext, index) {
    var size = pdfDocument.getPageSize(index);

    canvasContext.canvas.width = size.x;
    canvasContext.canvas.height = size.y;

    var imageData = canvasContext.createImageData(size.x, size.y);

    var current = new Int32Array(size.x * size.y);

    pdfDocument.drawPage(index, current, 0, 0, size.x, size.y, false);

    // from  ARGB to ABGR
    for (var i = 0; i < current.length; i++) {

        var val = current[i];

        var cursor = i * 4;

        imageData.data[cursor] = (val >> 16) & 0xFF; //r
        imageData.data[cursor + 1] = (val >> 8) & 0xFF; //g
        imageData.data[cursor + 2] = val & 0xFF; //b
        imageData.data[cursor + 3] = 255; // a
    }            
    canvasContext.putImageData(imageData, 0, 0);

Using pdfDocument.drawPage, you can fill the image data object created by the canvas. You must then invert pixel color components to switch from ARGB to ABGR and finally paste the result Inside the canvas.

And voila!

Reading PDF and XPS on your Windows 8 application using WinRT

2013-04-15T09:20:40Z

PDF and XPS file formats are widely used across the world and you could need one day to display them inside your application.

Today I would like to share with you a simple way to do so by using an open source solution: MuPDF (a multiplatform lightweight PDF and XPS viewer).

MuPDF is already working on a WinRT solution you can grab on this GIT repo: http://git.ghostscript.com/?p=user/mvrhel/mupdf.git;a=summary

I also used the work of Libreliodev and especially their advanced port of MuPDF for WinRT.

The result is a simple but really useful Windows 8 Modern UI app that is able to display a PDF/XPS file: http://www.catuhe.com/msdn/pdfreader.zip

The solution is based on two projects:

MuPDF.winRT: a C++/CX WinRT component built upon MuPDF
PDFReader: A C#XAML project that used the WinRT component to display a PDF

For this article I created a C#/XAML application. But because MuPDF.winRT is a WinRT component, you can also use it from a HTML5/CSS3/Javascript app!

Using the WinRT component, you just have to use the following line to create a Document (which is the class used to interact with a PDF or a XPS):

pdfDocument = Document.Create(readBuffer, DocumentType.PDF, (int)Windows.Graphics.Display.DisplayProperties.LogicalDpi);

To draw a page, you just have to use the previous pdfDocument object and call the DrawPage method:

pdfDocument.DrawPage(PageNumber, buf, 0, 0, width, height, false);

This method draws a given page inside a IBuffer. To create it you just have to instantiate a WriteableBitmap and the inner PixelBuffer:

var size = Document.GetPageSize(PageNumber);
var width = (int)(size.X * ZoomFactor);
var height = (int)(size.Y * ZoomFactor);

var image = new WriteableBitmap(width, height);
IBuffer buf = new Buffer(image.PixelBuffer.Capacity);
buf.Length = image.PixelBuffer.Length;

The buffer is then copied back to the WriteableBitmap:

using (var stream = buf.AsStream())
{
    await stream.CopyToAsync(image.PixelBuffer.AsStream());
}

And finally the result is displayed on screen:

You can even display two pages on the same image:

await Task.Run(() => pdfDocument.DrawFirtPageConcurrent(PageNumber, buf, width, height));
await Task.Run(() => pdfDocument.DrawSecondPageConcurrent(PageNumber + 1, buf, width, height));

Using the mouse or the touch you can obviously navigate inside the document or zoom on pages. And with the power of vectorized PDF, you will always have the best display quality:

Optical zoom (bitmap based)

Vectorized zoom

To display these images, I used a simple FlipView so that I do not have to bother about gestures. The DataTemplate used for every page is based on a ScrollViewer in order to automaticaly handle zoom in and zoom out:

<FlipView Name="flipView">
    <FlipView.ItemTemplate>
        <DataTemplate>
            <ScrollViewer
                ZoomMode="Enabled" 
                HorizontalScrollMode="Auto"
                VerticalScrollMode="Auto"
                VerticalSnapPointsType="None"
                HorizontalSnapPointsType="None"
                HorizontalScrollBarVisibility="Auto"
                VerticalScrollBarVisibility="Auto"
                MinZoomFactor="1"            
                MaxZoomFactor="4">
                <Image Source="{Binding Image}" Width="{Binding Width}" Height="{Binding Height}" 
                       Stretch="Uniform" HorizontalAlignment="Center" Margin="0"/>
            </ScrollViewer>
        </DataTemplate>
    </FlipView.ItemTemplate>
</FlipView>

Simple, isn’t it ?

Faites partie du futur Instagram ou Facebook :)

2013-04-10T11:18:00Z

Chers développeurs, je vous propose d'inviter ceux d'entre vous qui sont intéressés à une après-midi de rencontres avec des diplômés de l'Ecole Supérieure de Commerce de Paris.

Ces diplômés ont monté des projets, trouvé et mis en place le design, fais le business plan, réalisé les études de marchés et parfois même commencé le marketing.

Il ne leur manque que VOUS :)

Que vous soyez motivés pour partager votre temps perso avec eux ou que vous ayez envie de rejoindre le projet en tant que CTO, venez assister à leurs pitchs et faites vos courses.

Si vous êtes intéressés, envoyez moi un mail. Le pitch aura lieu le 18/04 à Spark en mode "cosy".

WinRT component: How to generate a GUID?

2013-03-25T13:39:23Z

For your Windows 8 applications, you may need to generate a unique identifier (for instance, if you want to uniquely identify your users).

To do so, here is a small C++ WinRT component (I decided not to use C# to make the component light-weighted because it is pure native code and so you do not need to load the CLR to use it).

#pragma once
#include <Objbase.h>
using namespace Platform;

namespace UrzaBox
{
    public ref class Tools sealed
    {
    public:
        Tools();

        static Platform::String^ GetNewID()
        {
            GUID result;
            HRESULT hr = CoCreateGuid(&result);

            if (SUCCEEDED(hr))
            {
                Guid gd(result);
                return gd.ToString();
            }

            throw Exception::CreateException(hr);
        }
    };
}

Create a custom user control using JavaScript and WinJS for Windows 8

2013-03-19T13:08:31Z

As you may know, you can easily develop native applications for Windows 8 using JavaScript and WinJS.

Among a lot of other things, WinJS allows you to create custom control in order to factorize your UI.

Today, I would like to show you how to create this kind of control and to do so, please let me introduce you with the final version of the control:

As you can see, this is a 3D carousel that can be used to display a long list of items alongside a great (if you like it) mirror.

The sample Visual Studio 2012 project can be downloaded here.

(If you want to know how to achieve the same goal but with C# and XAML, you can read this article written by my colleague Sébastien Pertus: http://blogs.msdn.com/b/mim/archive/2013/03/19/create-a-custom-user-control-using-xaml-and-c-for-windows-8.aspx)

So you may now wonder how you can develop such a beautiful (!!) control. The response will be decomposed on 6 parts:

How to set up the layout?
Declaring and instancing the control
Using templates
Using data binding
Integrating user inputs and animations
Adding events
Optimizations

How to set up the layout?

The control itself is built using a grid (CSS3 Grid) defined as the “carousel-viewport”. Inside the viewport there are two items: one called the “carousel-container” to handle all items and one called “carousel-mirror” for the mirror (obviously).

Every item is built inside a “carousel-item-container” and contains two parts: a “carousel-item” for the standard part and a “carousel-mirror-item” for the mirror part.

You can then use the magic of CSS in order to completely style every part of the control:

.carousel-item {    
    transition: transform 0.1s ease;
}

.carousel-item:hover {
    transform: scale(0.95);
}

For instance, with these rules, I added a cool hover effect to my items. Simple, isn’t it?

Declaring and instancing the control

To declare a custom control, you just have to declare a WinJS class:

var carousel = WinJS.Class.define(
    function(element, options) {
    // Constructor
    },
    {// Instance members
    }, 
    {// Static members
    });

The first argument of the WinJS.Class.define function is the constructor of you control (used to instantiate your class), the second one is the list of instance members and the last one is the list of static members.

Inside the constructor, you have to define the root DOM elements you want to use. The instance members can be properties or public functions you want to make available.

For our control, the constructor will produce the layout with the following code:

var carousel = WinJS.Class.define(
    function(element, options) {
        this._element = element || document.createElement("div");
        this._element.winControl = this;
        this._selectedIndex = -1;

        // Root element
        this._element.style.display = "-ms-grid";
        this._element.style.msGridRows = "1fr";
        this._element.style.msGridColumns = "1fr";
        this._element.style.msTouchAction = "none";
        WinJS.Utilities.addClass(this._element, "carousel-viewport");

        // Items element
        this._itemsRoot = document.createElement("div");
        this._element.appendChild(this._itemsRoot);
        this._itemsRoot.msGridRow = "1";
        this._itemsRoot.msGridColumn = "1";
        this._itemsRoot.style.display = "-ms-grid";
        this._itemsRoot.style.msGridRows = "1fr";
        this._itemsRoot.style.msGridColumns = "1fr";
        this._itemsRoot.style.perspectiveOrigin = "50% 50%";
        WinJS.Utilities.addClass(this._itemsRoot, "carousel-container");

        // Mirror
        this._mirror = document.createElement("div");
        this._mirror.style.msGridRow = "1";
        this._mirror.style.msGridColumn = "1";
        this._mirror.style.msGridRowAlign = "stretch";
        this._mirror.style.msGridColumnAlign = "stretch";
        this._mirror.style.backgroundColor = "Black";
        WinJS.Utilities.addClass(this._mirror, "carousel-mirror");

The constructor is also responsible for initialization options:

maxVisibleItems: Count of items visible on each side of the current item
depth: Depth of each item relatively to previous
animationSpeed : Speed of the animation when you change the current item
mirrorEnabled: Define if the mirror is visible
perspective: Value indicating the amount of perspective

These options are defined by the user during the instantiation of the control:

<div id="carousel" data-win-control="DarkStone.Controls.Carousel" data-win-options="{mirrorEnabled: true}">
</div>

The instance of your class can be retrieved using the following code:

var carousel = element.querySelector("#carousel").winControl;

Every exposed properties can also be manipulated by the client. Here is for instance, how a slider can control the perspective:

element.querySelector("#perspective").onchange = function(evt) {
    carousel.perspective = element.querySelector("#perspective").value;
};

And the result:

In addition to the previous properties, our control also exposes some other useful members (always defined with the WinJS.Class.define function):

var carousel = WinJS.Class.define(
    function(element, options) {
        // Code hidden for readability
    },
{
    _doBinding: function () {
        // Code hidden for readability
    },
    refreshLayout: function () {
        // Code hidden for readability
    },
    element: {
        get: function () { return this._element; }
    },
    itemRender: { // Main item template
        set: function (value) {
            this._itemRender = value;
            this._doBinding();
        }
    },
    mirrorRender: { // Mirror template
        set: function (value) {
            this._mirrorRender = value;
            this._doBinding();
        }
    },
    itemsSource: { // Data source
        set: function (source) {
            this._source = source;
            this._doBinding();
        }
    },
    selectedIndex: { // Selected index
        get: function () { return this._selectedIndex; },
        set: function (index) {
            if (index < 0 || index >= this._items.length || index == this.selectedIndex)
                return;
            this._selectedIndex = index;
            this.refreshLayout();
        }
    },
    // more code…       
 });

Using templates

WinJS also provides a way to generate item content based on templates. For our control, I decided to use two templates: one for the item and one for the mirror.

The user can define his templates like this:

<div id="templateDiv" data-win-control="WinJS.Binding.Template">
    <img data-win-bind="src: imageUrl" class="templateImg" />
</div>
<div id="mirrorTemplate" data-win-control="WinJS.Binding.Template">
    <div class="mirrorContainer">
        <img data-win-bind="src: imageUrl" class="templateImg" />
        <div class="opacityMask"></div>
    </div>
</div>

The templates are retrieved from the web page:

var template = this._itemRender.winControl;
var mirrorTemplate= this._mirrorRender.winControl;

And used to create the DOM content:

template.render(data, item);

These templates are used by a function called createItem which takes a data from the data source and generate DOM objects using templates:

// Create item
var createItem = function (data) {
    // Creating DOM element and setting correct position
    var root = document.createElement("div");
    root.style.msGridRow = "1";
    root.style.msGridColumn = "1";
    root.style.msGridRowAlign = "center";
    root.style.msGridColumnAlign = "center";
    root.style.msTouchAction = "none";

    root.style.display = "-ms-grid";
    root.style.msGridRows = "1fr";
    root.style.msGridColumns = "1fr";
    root.style.opacity = 0;
    WinJS.Utilities.addClass(root, "carousel-item-container");


    that._itemsRoot.appendChild(root);

    var item = document.createElement("div");
    root.appendChild(item);

    WinJS.Utilities.addClass(item, "carousel-item");

    // Template
    template.render(data, item);

    if (mirrorTemplate) {
        // Clone for mirror
        var clone = document.createElement("div");
        mirrorTemplate.render(data, clone);
        WinJS.Utilities.addClass(clone, "carousel-mirror-item");
        root.appendChild(clone);
        clone.style.transform = "translateY(" + item.clientHeight + "px) rotateX(180deg)";
    }
    
    that._itemsRoot.removeChild(root);

    return root;
};

This function generates the correct class for each part of the item. This will allow you to skin them easily. For instance, using the following templates:

<div id="templateDiv" data-win-control="WinJS.Binding.Template">
</div>
<div id="mirrorTemplate" data-win-control="WinJS.Binding.Template">
</div>

And the following CSS rules:

.carousel-item {    
    width: 600px;
    height: 600px;
    background-color: red;
}

.carousel-mirror-item {
    width: 600px;
    height: 600px;
    background-color: blue;
}

You will obtain the following result:

Using data binding

The user can define data using a property called itemsSource:

itemsSource: { // Data source
    set: function (source) {
        this._source = source;
        this._doBinding();
    }
},

You can use a simple array or a WinJS.Binding.List:

var carousel = element.querySelector("#carousel").winControl;
var list = new WinJS.Binding.List();

list.push({ imageUrl: "/assets/pic01.jpg" });
list.push({ imageUrl: "/assets/pic02.jpg" });
list.push({ imageUrl: "/assets/pic03.jpg" });
list.push({ imageUrl: "/assets/pic04.jpg" });
list.push({ imageUrl: "/assets/pic05.jpg" });
list.push({ imageUrl: "/assets/pic06.jpg" });

carousel.itemsSource = list;

Once the itemsSource is set, the control just have to browse it and call createItem for each element.

If you use a WinJS.Binding.List, the control can also handle two events to dynamically support insertion and deletion of items:

// Events
if (usingBindingList) {
    this._source.addEventListener("itemremoved", function (evt) { // On item removed
        var previous = that._items.splice(evt.detail.index, 1)[0];

        that._itemsRoot.removeChild(previous);

        if (evt.detail.index < that.selectedIndex) {
            that.selectedIndex--;
        } else if (Math.abs(that.selectedIndex - evt.detail.index) <= that.maxVisibleItems) {
            that.refreshLayout();
        }
    });
    this._source.addEventListener("iteminserted", function (evt) { // On item inserted
        var current = evt.detail.index;
        var newOne = createItem(that._source.getAt(current));

        that._items.splice(evt.detail.index, 0, newOne);

        if (Math.abs(that.selectedIndex - current) <= that.maxVisibleItems) {
            that.refreshLayout();
        }
    });
}

Using this events, if the user executes this code:

element.querySelector("#button1").onclick = function () {
    list.splice(0, 1); // Removing first item
};

element.querySelector("#button2").onclick = function () {
    list.push({ imageUrl: "/assets/pic01.jpg" });  // Adding new item
    list.push({ imageUrl: "/assets/pic02.jpg" }); // Adding new item
};

The list will behave like this:

Integrating user inputs and animations

To respond to user inputs, you just have to handle pointer events. The pointer events (more about here) are used to unify mouse, touch and pens inside a single event model. So for our control, you just have to add the following code to handle all kind of inputs:

// Pointer events
this._element.onmspointerdown = function (evt) {
    initialOffset = evt.clientX;

    leftButtonDown = evt.buttons == 1;

    that._pageChanged = false;
};

this._element.onmspointermove = function (evt) {
    if (!leftButtonDown)
        return;

    var moveThreshold = 40; // 40px for a slide

    if (Math.abs(evt.clientX - initialOffset) > moveThreshold) {
        if (evt.clientX < initialOffset) {
            that.selectedIndex++;
        } else {
            that.selectedIndex--;
        }
        that._pageChanged = true;
        leftButtonDown = false;
    }
};

this._element.onmspointerup = function (evt) {
    leftButtonDown = false;
};

The principle is simple: if the user moves his finger/mouse/pen over more than 40 pixels, we can change the current item according to the direction of the movement.

To create a good looking control, you must ensure that every update of the layout is animated (fast and fluid!). To do so, you may have noticed that every “carousel-item-container” has a CSS style to define transitions:

root.style.transition = "transform " + that._animationSpeed + "s ease-in-out, opacity " + that._animationSpeed + "s linear";

This transition defines that every change on the transform and opacity properties is not immediate but animated from its original value to the final value during a specific duration.

If you want to learn more about CSS3 Transitions, you can go just here.

To take advantage of this great feature, the code just have to change the transformation and the opacity of each item.

The transformation itself (CSS 3D Transforms) allows elements styled with CSS to be transformed in two-dimensional or three-dimensional space.

The transform matrix is built using this code:

var depth = (current == this.selectedIndex) ? 0 : -this._depth;
var rotation = (current == this.selectedIndex) ? 0 : ((current < this.selectedIndex) ? -45 : 45);
var offset = (current == this.selectedIndex) ? 0 : ((current < this.selectedIndex) ? -this._baseWidth : this._baseWidth);

// Transform
item.style.transform = "rotateY(" + rotation + "deg) translate3d(" + 
                        (((current - this.selectedIndex) * this._baseWidth) + offset) + "px, 0px, " + depth + "px)";

To summarize, the items are transformed according to the following schema:

The problem you can face when dealing with CSS 3D Transforms is about zIndex. Indeed, when you move objects using matrices, something objects may overlap in a wrong order.

For resolving this issue, you just have to correctly organize the zIndex value of your items:

// Z-Index
var total = this._items.length;
item.style.zIndex = (current == this.selectedIndex) ? total + 1 : total - Math.abs(current - this.selectedIndex);

Using this code, you can ensure that items are rendered in the correct order (depending on their distance to the selected item).

The opacity is computed using this code:

// Opacity
var currentOpacity;
if (current == this.selectedIndex) {
    currentOpacity = 1;
    
} else {
    currentOpacity = 1.0 - (Math.abs(current - this.selectedIndex) / this._maxVisibleItems);
}

Every item has an opacity proportional to its distance from the current item:

Adding events

Obviously, a good control must raise events. For instance, if an user select an item, it would be nice to raise a “selectedindexchanged”.

For our control, first of all you have to “dispatch” the event with the following code:

selectedIndex: { // Selected index
    get: function () { return this._selectedIndex; },
    set: function (index) {
        if (index < 0 || index >= this._items.length || index == this.selectedIndex)
            return;
        this._selectedIndex = index;
        this.refreshLayout();

        this.dispatchEvent("selectedindexchanged", {
            selectedIndex: index
        });
    }
},

When the user changes the selectedIndex, we just have to raise the event with dispatchEvent function of the control.

But the control will not natively provide this function. To add it, you just have to call a small WinJS function:

WinJS.Class.mix(DarkStone.Controls.Carousel,
    WinJS.Utilities.createEventProperties("selectedindexchanged"),
    WinJS.UI.DOMEventMixin);

With only these lines, you can add event listener on your control like any other control:

carousel.addEventListener("selectedindexchanged", function() {

});

Optimizations

Finally, just to be sure our control works well on very low end hardware, we must ensure that only visible items are effectively handled by the DOM.

In order to achieve this goal, we just have to remove items from the DOM when their opacity is inferior or equal to 0. Obviously you must re-inject them when they become visible again:

// Opacity
var currentOpacity;
if (current == this.selectedIndex) {
    currentOpacity = 1;
    
    if (!isPresentInsideDOM) {
        this._itemsRoot.appendChild(item);
    }
} else {
    currentOpacity = 1.0 - (Math.abs(current - this.selectedIndex) / this._maxVisibleItems);

    // Virtualization
    if (currentOpacity <= 0 && isPresentInsideDOM) {
        this._itemsRoot.removeChild(item);
    } else if (currentOpacity > 0 && !isPresentInsideDOM) {
        this._itemsRoot.appendChild(item);
    }
}

Using this code, we are now ready to handle thousands of items even on a low end devices.

The control is now complete, feel free to use it in your own app

Going further

If you want to read more about some subjects I addressed in this article, here are some links you may find useful:

Complete code

The complete code can be downloaded here or you can copy/paste from here:

(function () {

    var carousel = WinJS.Class.define(
        function(element, options) {
            this._element = element || document.createElement("div");
            this._element.winControl = this;
            this._selectedIndex = -1;

            // Root element
            this._element.style.display = "-ms-grid";
            this._element.style.msGridRows = "1fr";
            this._element.style.msGridColumns = "1fr";
            this._element.style.msTouchAction = "none";
            WinJS.Utilities.addClass(this._element, "carousel-viewport");

            // Items element
            this._itemsRoot = document.createElement("div");
            this._element.appendChild(this._itemsRoot);
            this._itemsRoot.msGridRow = "1";
            this._itemsRoot.msGridColumn = "1";
            this._itemsRoot.style.display = "-ms-grid";
            this._itemsRoot.style.msGridRows = "1fr";
            this._itemsRoot.style.msGridColumns = "1fr";
            this._itemsRoot.style.perspectiveOrigin = "50% 50%";
            WinJS.Utilities.addClass(this._itemsRoot, "carousel-container");

            // Mirror
            this._mirror = document.createElement("div");
            this._mirror.style.msGridRow = "1";
            this._mirror.style.msGridColumn = "1";
            this._mirror.style.msGridRowAlign = "stretch";
            this._mirror.style.msGridColumnAlign = "stretch";
            this._mirror.style.backgroundColor = "Black";
            WinJS.Utilities.addClass(this._mirror, "carousel-mirror");

            this._items = [];
            this._source = null;
            this._itemRender = null;
            this._mirrorRender = null;
            this._pageChanged = false;
            this._baseWidth = 0;

            // Default values
            this._maxVisibleItems = 15;
            this._depth = 100;
            this._animationSpeed = 0.3;
            this._mirrorEnabled = false;
            this.perspective = 800;

            // Options
            if (options) {
                this.depth = options.depth || this._depth;
                this.animationSpeed = options.animationSpeed || this._animationSpeed;
                this.perspective = options.perspective || this.perspective;
                this._mirrorEnabled = options.mirrorEnabled || this._mirrorEnabled;
                this._maxVisibleItems = options.maxVisibleItems || this._maxVisibleItems;
            }

            if (this._mirrorEnabled) {
                this._itemsRoot.appendChild(this._mirror);
            }

            // Pointer events
            var that = this;
            var initialOffset = 0;
            var leftButtonDown = false;

            this._element.addEventListener("MSPointerDown", function (evt) {
                initialOffset = evt.clientX;

                leftButtonDown = evt.buttons == 1;

                that._pageChanged = false;
            }, false);

            this._element.addEventListener("MSPointerMove", function (evt) {
                if (!leftButtonDown)
                    return;

                var moveThreshold = 40; // 40px for a slide

                if (Math.abs(evt.clientX - initialOffset) > moveThreshold) {
                    if (evt.clientX < initialOffset) {
                        that.selectedIndex++;
                    } else {
                        that.selectedIndex--;
                    }
                    that._pageChanged = true;
                    leftButtonDown = false;
                }
            }, false);

            this._element.addEventListener("MSPointerUp", function () {
                leftButtonDown = false;
            }, false);
        },
    {
        _doBinding: function () {
            if (!this._source || !this._itemRender || (this._mirrorEnabled && !this._mirrorRender))
                return;

            var usingBindingList = (this._source.getAt !== undefined);

            var template = this._itemRender.winControl;
            var mirrorTemplate;

            if (this._mirrorEnabled && this._mirrorRender)
                mirrorTemplate = this._mirrorRender.winControl;

            var that = this;

            var onItemClick = function (evt) { // Selecting the current item
                var newOne = that._items.indexOf(evt.currentTarget.parentElement);
                if (!that._pageChanged) {
                    that.selectedIndex = newOne;
                }
            };

            // Browse data source
            for (var index = 0; index < this._items.length; index++) {
                var item = this._items[index];
                
                if (item.parentElement) {
                    this._itemsRoot.removeChild(item);
                }
            }
            this._items = [];

            // Create item
            var createItem = function (data, initalTransform) {
                var root = document.createElement("div");
                root.style.msGridRow = "1";
                root.style.msGridColumn = "1";
                root.style.msGridRowAlign = "center";
                root.style.msGridColumnAlign = "center";
                root.style.msTouchAction = "none";
                if (initalTransform)
                    root.style.transform = initalTransform;
                root.style.transition = "transform " + that._animationSpeed + "s ease-in-out, opacity " 
                                                     + that._animationSpeed + "s linear";
                root.style.display = "-ms-grid";
                root.style.msGridRows = "1fr";
                root.style.msGridColumns = "1fr";
                root.style.opacity = 0;
                root.onmousemove = function (evt) {
                    evt.preventDefault(); // Preventing picture manipulation
                };
                WinJS.Utilities.addClass(root, "carousel-item-container");

                that._itemsRoot.appendChild(root);

                var item = document.createElement("div");
                root.appendChild(item);

                item.onclick = onItemClick;
                WinJS.Utilities.addClass(item, "carousel-item");

                // Template
                template.render(data, item);

                if (mirrorTemplate) {
                    // Clone for mirror
                    var clone = document.createElement("div");
                    mirrorTemplate.render(data, clone);
                    WinJS.Utilities.addClass(clone, "carousel-mirror-item");
                    root.appendChild(clone);
                    clone.style.transform = "translateY(" + item.clientHeight + "px) rotateX(180deg)";
                }
                
                that._itemsRoot.removeChild(root);

                return root;
            };

            // Browse data source
            for (var index = 0; index < this._source.length; index++) {
                var data = usingBindingList ? this._source.getAt(index) : this._source[index];

                this._items.push(createItem(data));
            }

            this._selectedIndex = 0;
            this.refreshLayout();

            // Events
            if (usingBindingList) {
                this._source.addEventListener("itemremoved", function (evt) { // On item removed
                    var previous = that._items.splice(evt.detail.index, 1)[0];

                    if (previous.parentElement) {
                        that._itemsRoot.removeChild(previous);
                    }

                    if (evt.detail.index < that.selectedIndex) {
                        that.selectedIndex--;
                    } else if (Math.abs(that.selectedIndex - evt.detail.index) <= that.maxVisibleItems) {
                        that.refreshLayout();
                    }
                });
                this._source.addEventListener("iteminserted", function (evt) { // On item inserted
                    var current = evt.detail.index;
                    var newOne = createItem(that._source.getAt(current));

                    that._items.splice(evt.detail.index, 0, newOne);

                    if (Math.abs(that.selectedIndex - current) <= that.maxVisibleItems) {
                        that.refreshLayout();
                    }
                });
            }

            // Mirror
            this._mirror.style.transform = "translateY(" + this.element.clientHeight / 2 + "px)";
        },
        refreshLayout: function () {
            if (this._items.length == 0) {
                return;
            }

            var current = this.selectedIndex;
            var total = this._items.length;
            for (var index = 0; index < total; index++) {

                var item = this._items[current];
                var isPresentInsideDOM = (item.parentElement !== null);

                // Opacity
                var currentOpacity;
                if (current == this.selectedIndex) {
                    currentOpacity = 1;
                    
                    if (!isPresentInsideDOM) {
                        this._itemsRoot.appendChild(item);
                        isPresentInsideDOM = true;
                    }
                } else {
                    currentOpacity = 1.0 - (Math.abs(current - this.selectedIndex) / this._maxVisibleItems);

                    // Virtualization
                    if (currentOpacity <= 0 && isPresentInsideDOM) {
                        this._itemsRoot.removeChild(item);
                        isPresentInsideDOM = false;
                    } else if (currentOpacity > 0 && !isPresentInsideDOM) {
                        this._itemsRoot.appendChild(item);
                        isPresentInsideDOM = true;
                    }
                }

                // Setting opacity
                item.style.opacity = currentOpacity;

                if (isPresentInsideDOM) {
                    if (!this._baseWidth)
                        this._baseWidth = item.clientWidth;

                    var depth = (current == this.selectedIndex) ? 0 : -this._depth;
                    var rotation = (current == this.selectedIndex) ? 0 : ((current < this.selectedIndex) ? -45 : 45);
                    var offset = (current == this.selectedIndex) ? 0 : (
                                             (current < this.selectedIndex) ? -this._baseWidth : this._baseWidth);

                    // Transform
                    item.style.transform = "rotateY(" + rotation + "deg) translate3d(" 
                       + (((current - this.selectedIndex) * this._baseWidth) + offset) + "px, 0px, " + depth + "px)";

                    // Z-Index
                    item.style.zIndex = (current == this.selectedIndex) ? total + 1 : total
                       - Math.abs(current - this.selectedIndex);
                }

                current++;

                if (current == this._items.length)
                    current = 0;
            }
        },
        element: {
            get: function () { return this._element; }
        },
        itemRender: { // Main item template
            set: function (value) {
                this._itemRender = value;
                this._doBinding();
            }
        },
        mirrorRender: { // Mirror template
            set: function (value) {
                this._mirrorRender = value;
                this._doBinding();
            }
        },
        itemsSource: { // Data source
            set: function (source) {
                this._source = source;
                this._doBinding();
            }
        },
        selectedIndex: { // Selected index
            get: function () { return this._selectedIndex; },
            set: function (index) {
                if (index < 0 || index >= this._items.length || index == this.selectedIndex)
                    return;
                this._selectedIndex = index;
                this.refreshLayout();

                this.dispatchEvent("selectedindexchanged", {
                    selectedIndex: index
                });
            }
        },
        animationSpeed: {
            get: function () { return this._animationSpeed; },
            set: function (value) {
                this._animationSpeed = value;
            }
        },
        depth: {
            get: function () { return this._depth; },
            set: function (value) {
                if (this._depth === value)
                    return;

                this._depth = value;

                this._refreshLayout();
            }
        },
        perspective: {
            get: function () {
                return parseInt(this._itemsRoot.style.perspective.replace("px"));
            },
            set: function (value) {
                this._itemsRoot.style.perspective = value + "px";
            }
        },
        maxVisibleItems: {
            get: function () {
                return this._maxVisibleItems;
            },
            set: function (value) {
                if (this._maxVisibleItems === value) {
                    return;
                }

                this._maxVisibleItems = value;
                this._refreshLayout();
            }
        }
    });

    WinJS.Namespace.define("DarkStone.Controls", {
        Carousel: carousel
    });
    
    WinJS.Class.mix(DarkStone.Controls.Carousel,
        WinJS.Utilities.createEventProperties("selectedindexchanged"),
        WinJS.UI.DOMEventMixin);
})();

TechDays 2013 - Vidéos de la plénière développeurs et des sessions Coding4Fun et Geek is in da House

2013-03-08T07:23:14Z

Ben voila, tout est dans le titre

Développeurs c’est le chef

Coding4Fun 2013

Geek is in the house

Developing a WinRT component to create a video file using Media Foundation

2013-03-06T10:00:09Z

After creating a GIF file using my previous article, I propose you to use the power of WinRT to create a component in order to produce video file from your drawings (using for instance a canvas with Javascript).

To do so, we will use Media Foundation COM components. Media Foundation is the next generation multimedia platform for Windows that enables developers, consumers, and content providers to embrace the new wave of premium content with enhanced robustness, unparalleled quality, and seamless interoperability.

And as I said before, Media Foundation is based on COM components. But thanks to C++ projects for Windows Store, we can create a WinRT component on top of Media Foundation in order to use it from our Javascript or .NET projects.

This WinRT component will be called VideoGenerator.

The VideoGenerator component

The VideoGenerator class will have a main method called AppendNewFrame. This method is used to add a new frame to the video.

The class requires the following information to work:

Frame’s width
Frame’s height
Delay between frames (in milliseconds)

The class itself can be like the following:

public ref class VideoGenerator sealed
{

public:
    VideoGenerator(UINT32 width, UINT32 height, Windows::Storage::Streams::IRandomAccessStream^ stream,
                   UINT32 delay);
    virtual ~VideoGenerator();

    void AppendNewFrame(const Array<byte> ^videoFrameBuffer);
    void Finalize();
};

Obviously, this class also requires private data to work:

UINT32 videoWidth;
UINT32 videoHeight;
UINT32 fps;
UINT32 bitRate;
UINT32 frameSize;
GUID   encodingFormat;
GUID   inputFormat;

videoWidth is the width of a frame
videoHeight is the height of a frame
fps is the number of frames per second
bitRate is obviously the bits rate
frameSize is equal to videoWidth * videoHeight
encodingFormat defines the encoder to use (WMV for us)
inputFormat defines the format of the pixels sent to this class (RGB32)

Initialization

Starting with this information, here is the constructor:

VideoGenerator::VideoGenerator(UINT32 width, UINT32 height, 
                               Windows::Storage::Streams::IRandomAccessStream^ stream, UINT32 delay)
{
    videoWidth = width;
    videoHeight = height;
    fps = 25;
    bitRate = 400000;
    frameSize = videoWidth * videoHeight;
    encodingFormat = MFVideoFormat_WMV3;
    inputFormat = MFVideoFormat_RGB32;

    HRESULT hr = CoInitializeEx(NULL, COINIT_APARTMENTTHREADED);
    if (SUCCEEDED(hr))
    {
        hr = MFStartup(MF_VERSION);
        if (SUCCEEDED(hr))
        {
            hr = InitializeSinkWriter(stream);
            if (SUCCEEDED(hr))
            {
                initiated = true;
                rtStart = 0;
                rtDuration = (10000000 * delay) / 1000;
            }
        }
    }
}

This method needs to initialize COM engine (CoInitializeEx) and also needs to initialize Media Foundation (MFStartup).

To compile you will need the following includes:

#include <Windows.h>
#include <mfapi.h>
#include <mfidl.h>
#include <Mfreadwrite.h>
#include <mferror.h>
#include <wrl\client.h>
#include <memory>

Please note that the bigger the bitRate is the bigger the file will be.

The next step after initializing things is to create a sink writer (IMFSinkWriter) which is used to control writings to the output file.

The sink writer enables you to author media files by passing in uncompressed or encoded data. For example, you can use it to re-encode a video file, or to capture live video from a webcam to a file.

In our case we will use it by passing uncompressed data from .NET or javascript.

HRESULT VideoGenerator::InitializeSinkWriter(Windows::Storage::Streams::IRandomAccessStream^ stream)
{    
    ComPtr<IMFAttributes> spAttr;
    ComPtr<IMFMediaType>  mediaTypeOut;   
    ComPtr<IMFMediaType>  mediaTypeIn;           
    ComPtr<IMFByteStream> spByteStream;
    HRESULT hr = MFCreateMFByteStreamOnStreamEx((IUnknown*)stream, &spByteStream);

    if (SUCCEEDED(hr))
    {        
        MFCreateAttributes(&spAttr, 10);
        spAttr->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, true);

        hr = MFCreateSinkWriterFromURL(L".wmv", spByteStream.Get(), spAttr.Get(), &sinkWriter);
    }

    // Set the output media type.
    if (SUCCEEDED(hr))
    {
        hr = MFCreateMediaType(&mediaTypeOut);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);     
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetGUID(MF_MT_SUBTYPE, encodingFormat);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetUINT32(MF_MT_AVG_BITRATE, bitRate);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeSize(mediaTypeOut.Get(), MF_MT_FRAME_SIZE, videoWidth, videoHeight);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeOut.Get(), MF_MT_FRAME_RATE, fps, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeOut.Get(), MF_MT_PIXEL_ASPECT_RATIO, 1, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->AddStream(mediaTypeOut.Get(), &streamIndex);   
    }

    // Set the input media type.
    if (SUCCEEDED(hr))
    {
        hr = MFCreateMediaType(&mediaTypeIn);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeIn->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeIn->SetGUID(MF_MT_SUBTYPE, inputFormat);     
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeIn->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeSize(mediaTypeIn.Get(), MF_MT_FRAME_SIZE, videoWidth, videoHeight);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeIn.Get(), MF_MT_FRAME_RATE, fps, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeIn.Get(), MF_MT_PIXEL_ASPECT_RATIO, 1, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->SetInputMediaType(streamIndex, mediaTypeIn.Get(), NULL);   
    }

    // Tell the sink writer to start accepting data.
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->BeginWriting();
    }

    return hr;
}

Starting from a WinRT Stream, you have to call MFCreateMFByteStreamOnStreamEx method to convert it to a IMFByteStream object. Then using MFCreateSinkWriterFromURL you can grab the sink writer.

The following part of the code is dedicated to configure the input and output format of the sink writer accordingly to defined parameters (mediaTypeOut and mediaTypeIn objects).

The mediaTypeOut is responsible for:

MF_MT_MAJOR_TYPE: The main type of the file (Video here)
MF_MT_SUBTYPE: The subtype (WMV here)
MF_MT_AVG_BITRATE: The bit rate or the compression rate if you will (Defines the quality of the video)
MF_MT_INTERLACE_MODE: Interlace mode (Progressive for us)
MF_MT_FRAME_SIZE: The size of the frame (width and height)
MF_MT_FRAME_RATE: Frame per second
MF_MT_PIXEL_ASPECT_RATIO: Aspect ratio

The mediaTypeIn is responsible for:

MF_MT_MAJOR_TYPE: The main type of the file (Video here)
MF_MT_SUBTYPE: The subtype (RGB32 here)
MF_MT_INTERLACE_MODE: Interlace mode (Progressive for us)
MF_MT_FRAME_SIZE: The size of the frame (width and height)
MF_MT_FRAME_RATE: Frame per second
MF_MT_PIXEL_ASPECT_RATIO: Aspect ratio

Adding a new frame

Once we have a sink writer, you just have to append new frame by sending a byte array to the component:

void VideoGenerator::AppendNewFrame(const Platform::Array<byte> ^videoFrameBuffer)
{
    auto length = videoFrameBuffer->Length / sizeof(DWORD);
    DWORD *buffer = (DWORD *)(videoFrameBuffer->Data);
    std::unique_ptr<DWORD[]> target(new DWORD[length]);

    for (UINT32 index = 0; index < length; index++)
    {
        DWORD color = buffer[index];
        BYTE b = (BYTE)((color & 0x00FF0000) >> 16);
        BYTE g = (BYTE)((color & 0x0000FF00) >> 8);
        BYTE r = (BYTE)((color & 0x000000FF));

#if ARM
        auto row = index / videoWidth;
        auto targetRow = videoHeight - row - 1;
        auto column = index - (row * videoWidth);
        target[(targetRow * videoWidth) + column] = (r << 16) + (g << 8) + b;
#else
        target[index] = (r << 16) + (g << 8) + b;
#endif
    }

    // Send frame to the sink writer.
    HRESULT hr = WriteFrame(target.get(), rtStart, rtDuration);
    if (FAILED(hr))
    {
        throw Platform::Exception::CreateException(hr);
    }
    rtStart += rtDuration;
}

Before writing data, you have to prepare it because:

Data must be provided using RGB format and in my case (HTML5 canvas), data is stored using BGR format
On ARM devices (detected thanks to the define #ARM), data is stored starting from the last line (and not the first line) so you have to revert them on the Y axis

Then, when your data is ready, just call the following code to append it to the output file:

HRESULT VideoGenerator::WriteFrame(
    DWORD *videoFrameBuffer,
    const LONGLONG& rtStart,        // Time stamp.
    const LONGLONG& rtDuration      // Frame duration.
    )
{
    ComPtr<IMFSample> sample;
    ComPtr<IMFMediaBuffer> buffer;

    const LONG cbWidth = 4 * videoWidth;
    const DWORD cbBuffer = cbWidth * videoHeight;

    BYTE *pData = NULL;

    // Create a new memory buffer.
    HRESULT hr = MFCreateMemoryBuffer(cbBuffer, &buffer);

    // Lock the buffer and copy the video frame to the buffer.
    if (SUCCEEDED(hr))
    {
        hr = buffer->Lock(&pData, NULL, NULL);
    }
    if (SUCCEEDED(hr))
    {
        hr = MFCopyImage(
            pData,                      // Destination buffer.
            cbWidth,                    // Destination stride.
            (BYTE*)videoFrameBuffer,    // First row in source image.
            cbWidth,                    // Source stride.
            cbWidth,                    // Image width in bytes.
            videoHeight                // Image height in pixels.
            );
    }
    if (buffer.Get())
    {
        buffer->Unlock();
    }

    // Set the data length of the buffer.
    if (SUCCEEDED(hr))
    {
        hr = buffer->SetCurrentLength(cbBuffer);
    }

    // Create a media sample and add the buffer to the sample.
    if (SUCCEEDED(hr))
    {
        hr = MFCreateSample(&sample);
    }
    if (SUCCEEDED(hr))
    {
        hr = sample->AddBuffer(buffer.Get());
    }

    // Set the time stamp and the duration.
    if (SUCCEEDED(hr))
    {
        hr = sample->SetSampleTime(rtStart);
    }
    if (SUCCEEDED(hr))
    {
        hr = sample->SetSampleDuration(rtDuration);
    }

    // Send the sample to the Sink Writer.
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->WriteSample(streamIndex, sample.Get());
    }

    return hr;
}

The code creates a sample, fill it with your data and set time and duration before writing the sample to the sink writer.

Closing and cleaning

When you are done, you just have to call the Finalize method (and for the sake of completeness, I also add the destructor here):

VideoGenerator::~VideoGenerator()
{
    Finalize();
}

void VideoGenerator::Finalize()
{
    if (!initiated)
        return;

    initiated = false;
    sinkWriter->Finalize();
    MFShutdown();
}

Finalization is just about closing the sink writer and release Media Foundation API (MFShutdown).

Using VideoGenerator

VideoGenerator is a WinRT component so you can easily use it from .NET or Javascript. For instance, using Javascript, the client code can be something like that:

var picker = new Windows.Storage.Pickers.FileSavePicker();
picker.fileTypeChoices.insert(Flipflop.Tools.GetString("VideoFiles"), [".wmv"]);

picker.pickSaveFileAsync().then(function (file) {
    if (!file) {
        return;
    }
    file.openAsync(Windows.Storage.FileAccessMode.readWrite).then(function (stream) {
        var videoGenerator = new VideoTools.VideoGenerator(800, 600, stream, 16);

        for (var commandsIndex = 0; commandsIndex < 50; commandsIndex++) {
            var canvas = document.getElementById("canvas" + commandsIndex);
            var context = canvas.getContext("2d");

            var data = context.getImageData(0, 0, width, height);

            var bytes = data.data;
            videoGenerator.appendNewFrame(bytes);
        }

        videoGenerator.finalize();
        stream.close();
    });
});

Complete code

The complete VideoGenerator code can be found here.

Or copy/paste right here.

VideoGenerator.h

#pragma once
#include <Windows.h>
#include <mfapi.h>
#include <mfidl.h>
#include <Mfreadwrite.h>
#include <mferror.h>
#include <wrl\client.h>
#include <memory>

using namespace Platform;
using namespace Microsoft::WRL;

namespace VideoTools
{
    public ref class VideoGenerator sealed
    {
        UINT32 videoWidth;
        UINT32 videoHeight;
        UINT32 fps;
        UINT32 bitRate;
        UINT32 frameSize;
        GUID   encodingFormat;
        GUID   inputFormat;

        DWORD  streamIndex;
        ComPtr<IMFSinkWriter> sinkWriter;

        bool   initiated;

        LONGLONG rtStart;
        UINT64 rtDuration;

    private:
        HRESULT InitializeSinkWriter(Windows::Storage::Streams::IRandomAccessStream^ stream);
        HRESULT WriteFrame(DWORD *videoFrameBuffer, const LONGLONG& rtStart, 
                           const LONGLONG& rtDuration);

    public:
        VideoGenerator(UINT32 width, UINT32 height, 
                       Windows::Storage::Streams::IRandomAccessStream^ stream, UINT32 delay);
        virtual ~VideoGenerator();

        void AppendNewFrame(const Array<byte> ^videoFrameBuffer);
        void Finalize();
    };
}

VideoGenerator.cpp

#include "pch.h"
#include "VideoGenerator.h"

#pragma comment(lib, "mfreadwrite")
#pragma comment(lib, "mfplat")
#pragma comment(lib, "mfuuid")

using namespace VideoTools;

VideoGenerator::VideoGenerator(UINT32 width, UINT32 height, 
                               Windows::Storage::Streams::IRandomAccessStream^ stream, UINT32 delay)
{
    videoWidth = width;
    videoHeight = height;
    fps = 25;
    bitRate = 400000;
    frameSize = videoWidth * videoHeight;
    encodingFormat = MFVideoFormat_WMV3;
    inputFormat = MFVideoFormat_RGB32;

    HRESULT hr = CoInitializeEx(NULL, COINIT_APARTMENTTHREADED);
    if (SUCCEEDED(hr))
    {
        hr = MFStartup(MF_VERSION);
        if (SUCCEEDED(hr))
        {
            hr = InitializeSinkWriter(stream);
            if (SUCCEEDED(hr))
            {
                initiated = true;
                rtStart = 0;
                rtDuration = (10000000 * delay) / 1000;
            }
        }
    }
}

VideoGenerator::~VideoGenerator()
{
    Finalize();
}

void VideoGenerator::Finalize()
{
    if (!initiated)
        return;

    initiated = false;
    sinkWriter->Finalize();
    MFShutdown();
}

void VideoGenerator::AppendNewFrame(const Platform::Array<byte> ^videoFrameBuffer)
{
    auto length = videoFrameBuffer->Length / sizeof(DWORD);
    DWORD *buffer = (DWORD *)(videoFrameBuffer->Data);
    std::unique_ptr<DWORD[]> target(new DWORD[length]);

    for (UINT32 index = 0; index < length; index++)
    {
        DWORD color = buffer[index];
        BYTE b = (BYTE)((color & 0x00FF0000) >> 16);
        BYTE g = (BYTE)((color & 0x0000FF00) >> 8);
        BYTE r = (BYTE)((color & 0x000000FF));

#if ARM
        auto row = index / videoWidth;
        auto targetRow = videoHeight - row - 1;
        auto column = index - (row * videoWidth);
        target[(targetRow * videoWidth) + column] = (r << 16) + (g << 8) + b;
#else
        target[index] = (r << 16) + (g << 8) + b;
#endif
    }

    // Send frame to the sink writer.
    HRESULT hr = WriteFrame(target.get(), rtStart, rtDuration);
    if (FAILED(hr))
    {
        throw Platform::Exception::CreateException(hr);
    }
    rtStart += rtDuration;
}

HRESULT VideoGenerator::InitializeSinkWriter(Windows::Storage::Streams::IRandomAccessStream^ stream)
{    
    ComPtr<IMFAttributes> spAttr;
    ComPtr<IMFMediaType>  mediaTypeOut;   
    ComPtr<IMFMediaType>  mediaTypeIn;           
    ComPtr<IMFByteStream> spByteStream;
    HRESULT hr = MFCreateMFByteStreamOnStreamEx((IUnknown*)stream, &spByteStream);

    if (SUCCEEDED(hr))
    {        
        MFCreateAttributes(&spAttr, 10);
        spAttr->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, true);

        hr = MFCreateSinkWriterFromURL(L".wmv", spByteStream.Get(), spAttr.Get(), &sinkWriter);
    }

    // Set the output media type.
    if (SUCCEEDED(hr))
    {
        hr = MFCreateMediaType(&mediaTypeOut);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);     
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetGUID(MF_MT_SUBTYPE, encodingFormat);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetUINT32(MF_MT_AVG_BITRATE, bitRate);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeOut->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeSize(mediaTypeOut.Get(), MF_MT_FRAME_SIZE, videoWidth, videoHeight);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeOut.Get(), MF_MT_FRAME_RATE, fps, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeOut.Get(), MF_MT_PIXEL_ASPECT_RATIO, 1, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->AddStream(mediaTypeOut.Get(), &streamIndex);   
    }

    // Set the input media type.
    if (SUCCEEDED(hr))
    {
        hr = MFCreateMediaType(&mediaTypeIn);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeIn->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);   
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeIn->SetGUID(MF_MT_SUBTYPE, inputFormat);     
    }
    if (SUCCEEDED(hr))
    {
        hr = mediaTypeIn->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeSize(mediaTypeIn.Get(), MF_MT_FRAME_SIZE, videoWidth, videoHeight);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeIn.Get(), MF_MT_FRAME_RATE, fps, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = MFSetAttributeRatio(mediaTypeIn.Get(), MF_MT_PIXEL_ASPECT_RATIO, 1, 1);   
    }
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->SetInputMediaType(streamIndex, mediaTypeIn.Get(), NULL);   
    }

    // Tell the sink writer to start accepting data.
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->BeginWriting();
    }

    return hr;
}

HRESULT VideoGenerator::WriteFrame(
    DWORD *videoFrameBuffer,
    const LONGLONG& rtStart,        // Time stamp.
    const LONGLONG& rtDuration      // Frame duration.
    )
{
    ComPtr<IMFSample> sample;
    ComPtr<IMFMediaBuffer> buffer;

    const LONG cbWidth = 4 * videoWidth;
    const DWORD cbBuffer = cbWidth * videoHeight;

    BYTE *pData = NULL;

    // Create a new memory buffer.
    HRESULT hr = MFCreateMemoryBuffer(cbBuffer, &buffer);

    // Lock the buffer and copy the video frame to the buffer.
    if (SUCCEEDED(hr))
    {
        hr = buffer->Lock(&pData, NULL, NULL);
    }
    if (SUCCEEDED(hr))
    {
        hr = MFCopyImage(
            pData,                      // Destination buffer.
            cbWidth,                    // Destination stride.
            (BYTE*)videoFrameBuffer,    // First row in source image.
            cbWidth,                    // Source stride.
            cbWidth,                    // Image width in bytes.
            videoHeight                // Image height in pixels.
            );
    }
    if (buffer.Get())
    {
        buffer->Unlock();
    }

    // Set the data length of the buffer.
    if (SUCCEEDED(hr))
    {
        hr = buffer->SetCurrentLength(cbBuffer);
    }

    // Create a media sample and add the buffer to the sample.
    if (SUCCEEDED(hr))
    {
        hr = MFCreateSample(&sample);
    }
    if (SUCCEEDED(hr))
    {
        hr = sample->AddBuffer(buffer.Get());
    }

    // Set the time stamp and the duration.
    if (SUCCEEDED(hr))
    {
        hr = sample->SetSampleTime(rtStart);
    }
    if (SUCCEEDED(hr))
    {
        hr = sample->SetSampleDuration(rtDuration);
    }

    // Send the sample to the Sink Writer.
    if (SUCCEEDED(hr))
    {
        hr = sinkWriter->WriteSample(streamIndex, sample.Get());
    }

    return hr;
}

Going further

Some useful links to go further:

Media foundation doc: http://msdn.microsoft.com/en-us/library/windows/apps/ms694197.aspx
About Media foundation: http://msdn.microsoft.com/en-us/library/windows/apps/ms696274.aspx
Creating a WinRT component – part 1:http://blogs.msdn.com/b/eternalcoding/archive/2012/08/13/creating-a-winrt-component-using-c-cx-deform-a-direct2d-effect-toolkit.aspx
Creating a WinRT component – part 2: http://blogs.msdn.com/b/eternalcoding/archive/2012/08/28/creating-a-winrt-component-using-c-cx-part-2-adding-a-custom-direct2d-effect-to-deform.aspx

How to create an animated GIF with WinRT

2013-02-28T15:05:20Z

Why using GIF files?

For one of my project (Flipflop: http://apps.microsoft.com/windows/app/flipflop/99c01512-fe4f-4d1a-872e-eb9fd6638ff4), I needed to create an animated GIF based on a group of HTML5 canvas.

A GIF produced by Flipflop

The main goal of Flipflop is to allow users to create an animated flipbook. And to allow them to share it, I finally found that GIF files were a great deal.

Creating a GIF using WIC components

So I propose you to share the small class that I used to achieve this goal: GifMaker.

To work this class requires you to specify frames size and obviously all frames:

using System;
using System.Collections.Generic;
using System.Runtime.InteropServices.WindowsRuntime;
using Windows.Foundation;
using Windows.Graphics.Imaging;
using Windows.Storage;

namespace Utilities
{
    public sealed class GifMaker
    {
        readonly List<byte[]> frames = new List<byte[]>();
        private readonly uint frameWidth;
        private readonly uint frameHeight;

        public GifMaker(uint width, uint height)
        {
            frameWidth = width;
            frameHeight = height;
        }

        public void AppendNewFrame([ReadOnlyArray]byte[] frame)
        {
            frames.Add(frame);
        }

This class mainly uses WIC or actually Windows.Graphics namespace. This namespace contains a class called BitmapEncoder that will allow us to generate files using specific encoders such as GifEncoder.

The point is that GIF files are composed of a list of frames. You just have to browse through them and send each of them to the encoder using GoToNextFrameAsync to register a new frame:

public IAsyncInfo GenerateAsync(StorageFile file)
{
    return AsyncInfo.Run(async ctx =>
        {
            var outStream = await file.OpenAsync(FileAccessMode.ReadWrite);

            var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.GifEncoderId, outStream);

            for (int i = 0; i < frames.Count; i++)
            {
                var pixels = frames[i];
                encoder.SetPixelData(BitmapPixelFormat.Rgba8, BitmapAlphaMode.Ignore,
                                     frameWidth, frameHeight,
                                     92.0, 92.0,
                                     pixels);

                if (i < frames.Count - 1)
                    await encoder.GoToNextFrameAsync();
            }

            await encoder.FlushAsync();
            outStream.Dispose();
        });
}

Easy, isn’t it ?

Adding delay between frames

To be complete, you also have to handle the delay between frames and for this point, I must admit it took me a long time to figure out how to do that.

The solution is handled by frame properties (encoder.BitmapProperties) and especially via the property named “/grctlext/Delay” (!!).

To the final code is like the following:

public IAsyncInfo GenerateAsync(StorageFile file, int delay)
{
    return AsyncInfo.Run(async ctx =>
        {
            var outStream = await file.OpenAsync(FileAccessMode.ReadWrite);

            var encoder = await BitmapEncoder.CreateAsync(BitmapEncoder.GifEncoderId, outStream);

            for (int i = 0; i < frames.Count; i++)
            {
                var pixels = frames[i];
                encoder.SetPixelData(BitmapPixelFormat.Rgba8, BitmapAlphaMode.Ignore,
                                     frameWidth, frameHeight,
                                     92.0, 92.0,
                                     pixels);

                if (i == 0)
                {
                    var properties = new BitmapPropertySet
                        {
                            {
                                "/grctlext/Delay",
                                new BitmapTypedValue(delay / 10, PropertyType.UInt16)
                            }
                        };

                    await encoder.BitmapProperties.SetPropertiesAsync(properties);
                }

                if (i < frames.Count - 1)
                    await encoder.GoToNextFrameAsync();
            }

            await encoder.FlushAsync();
            outStream.Dispose();
        });
}

Using it from other languages

For instance if you want to use it from JavaScript, you just have to do something like that:

var picker = new Windows.Storage.Pickers.FileSavePicker();
picker.fileTypeChoices.insert("Gif files", [".gif"]);

picker.pickSaveFileAsync().then(function(file) {
    if (!file) {
        return;
    }
    var gifMaker = new Utilities.GifMaker(800, 600);

    for (var commandsIndex = 0; commandsIndex < currentFlipflop.pages.length; commandsIndex++) {
        // This code is used to retrieve canvases bytes
        var bytes = Flipflop.Tools.GetBytesfromFlipflop(currentFlipflop.pages[commandsIndex],
            800, 600);

        gifMaker.appendNewFrame(bytes);
    }

    gifMaker.generateAsync(file, 200).done();
});

Obviously it works the same way from .NET!

Introduction to CSS3 Transitions

2013-02-28T05:54:00Z

A good-looking application must provide user with visual feedback. User must always know that an order (a click, a tap or whatever) is well received and understood by the application and animations are a great tool to do so.

The new HTML 5 specification (to be honest, I should say “the new CSS 3 specification”) introduces a great tool to handle simple animations: the transitions.

According to “CSS Transitions Module Level 3” specification on W3C site, CSS3 Transitions allows property changes in CSS values to occur smoothly over a specified duration.

The aim of this article will be to first describe the concept of transitions and then to see how CSS3 Transitions works and how we can handle browsers that don’t support the feature:

CSS3 Transitions
Putting it all together
Transitions without CSS3 Transitions
Conclusion
Going further

In addition, I suggest you to read the “Introduction to CSS3 Animations” (by David Rousset) which is an excellent companion for this article.

To see how CSS3 Transitions can be used, I developed is a sample of a game which uses CSS3 Transitions to animate cells of a puzzle (and which will fallback to JavaScript if your browser doesn’t support CSS3 Transitions):

To launch a game, just click on “Mix it!” and try to resolve the puzzle by clicking the cells!

<p>Your browser does not support iframes.</p> Click here to show the demo : <a href="http://www.catuhe.com/msdn/transitions/index.htm">http://www.catuhe.com/msdn/transitions/index.htm</a>

The code of this game is available here.

CSS3 Transitions

Introduction

At the beginning, the W3C CSS workgroup resisted adding transitions to CSS arguing that transitions are not really style properties. But eventually designers and developers managed to convince them that transitions is about dynamic styles and can take place in a CSS file.

According to the W3C site, CSS3 Transitions are able to animate the following types of properties: (click here to show them)

color: interpolated via red, green, blue and alpha components (treating each as a number, see below)
length: interpolated as real numbers.
percentage: interpolated as real numbers.
integer: interpolated via discrete steps (whole numbers). The interpolation happens in real number space and is converted to an integer using floor().
number: interpolated as real (floating point) numbers.
transform list: see CSS Transforms specification: http://www.w3.org/TR/css3-2d-transforms/
rectangle: interpolated via the x, y, width and height components (treating each as a number).
visibility: interpolated via a discrete step. The interpolation happens in real number space between 0 and 1, where 1 is "visible" and all other values are "hidden".
shadow: interpolated via the color, x, y and blur components (treating them as color and numbers where appropriate). In the case where there are lists of shadows, the shorter list is padded at the end with shadows whose color is transparent and all lengths (x, y, blur) are 0.
gradient: interpolated via the positions and colors of each stop. They must have the same type (radial or linear) and same number of stops in order to be animated.
paint server (SVG): interpolation is only supported between: gradient to gradient and color to color. They then work as above.
space-separated list of above: If the lists have the same number of items, each item in the list is interpolated using the rules above. Otherwise, no interpolation.
a shorthand property: If all the parts of a shorthand can be animated, then interpolation is performed as if each property was individually specified.

And the following properties must be supported for transitions:(click here to show them)

background-color (color)
background-image (only gradients)
background-position (percentage and length)
border-bottom-color (color)
border-bottom-width (length)
border-color (color)
border-left-color (color)
border-left-width (length)
border-right-color (color)
border-right-width (length)
border-spacing (length)
border-top-color (color)
border-top-width (length)
border-width (length)
bottom (length and percentage)
color (color)
crop (rectangle)
font-size (length and percentage)
font-weight (number)
grid-* (various)
height (length and percentage)
left (length and percentage)
letter-spacing (length)
line-height (number, length and percentage)
margin-bottom (length)
margin-left (length)
margin-right (length)
margin-top (length)
max-height (length and percentage)
max-width (length and percentage)
min-height (length and percentage)
min-width (length and percentage)
opacity (number)
outline-color (color)
outline-offset (integer)
outline-width (length)
padding-bottom (length)
padding-left (length)
padding-right (length)
padding-top (length)
right (length and percentage)
text-indent (length and percentage)
text-shadow (shadow)
top (length and percentage)
vertical-align (keywords, length and percentage)
visibility (visibility)
width (length and percentage)
word-spacing (length and percentage)
z-index (integer)
zoom (number)

SVG

The properties of SVG objects are animatable when they are defined as animatable:true in the SVG specification: http://www.w3.org/TR/SVG/struct.html.

Declarations

To declare a transition in a CSS file, you just have to write the following code:

transition-property: all;
transition-duration: 0.5s;
transition-timing-function: ease;
transition-delay: 0s;

This declaration defines that any update on any property will be done in 0.5s (and not immediately so).

You can also define your translations on a per property basis:

transition-property: opacity left top;
transition-duration: 0.5s 0.8s 0.1s;
transition-timing-function: ease linear ease;
transition-delay: 0s 0s 1s;

And finally you can use the shorthand property “transition” to define all you need in a single line:

transition: all 0.5s ease 0s;

In this shorthand version you can precise as many properties as you want separated by a comma:

transition: opacity 0.5s ease 0s, left 0.8s linear 0s;

The transitions will be triggered when a property of the target object is updated. The update can be done with JavaScript or using CSS3 by assign new class to a tag.

For example, using IE10 if you have the following CSS3 declaration:

-ms-transition-property: opacity left top;
-ms-transition-duration: 0.5s 0.8s 0.5s;
-ms-transition-timing-function: ease linear ease;

When you update the opacity of your tag, the current value will be animated to the new value over 0.5s with a ease timing function (which give a smooth animation).

Non linear transitions

The “transition-timing-function” line defines that the transition will not be linear but will use a timing function to produce a non linear animation.

Basically, CSS3 transitions will use cubic bezier curve to smooth the transition by computing different speed over its duration.

The following functions are supported:

linear: Constant speed
cubic-bezier: Speed will be computed according to a cubic bezier curve define by two control points: P0 et P1 (so you will have to define 4 values here: P0x,P0y and P1x, P1y.
ease: Speed will be computed with cubic-bezier(0.25, 0.1, 0.25, 1)
ease-in: Speed will be computed with cubic-bezier(0.42, 0, 1, 1)
ease-inout: Speed will be computed with cubic-bezier(0.42, 0, 0.58, 1)
ease-out: Speed will be computed with cubic-bezier(0, 0, 0.58, 1)

Here is a simulation tool (using SVG of course) to show the impact of each timing function:

<p>Your browser does not support iframes.</p> Click here to show the demo : <a href="http://www.catuhe.com/msdn/transitions/easingfunctions.htm">http://www.catuhe.com/msdn/transitions/easingfunctions.htm</a>

This simulator is written with pure JavaScript code to facilitate the understanding of the function:

TRANSITIONSHELPER.computeCubicBezierCurveInterpolation = function (t, x1, y1, x2, y2) {
    // Extract X (which is equal to time here)
    var f0 = 1 - 3 * x2 + 3 * x1;
    var f1 = 3 * x2 - 6 * x1;
    var f2 = 3 * x1;
 
    var refinedT = t;
    for (var i = 0; i < 5; i++) {
        var refinedT2 = refinedT * refinedT;
        var refinedT3 = refinedT2 * refinedT;
 
        var x = f0 * refinedT3 + f1 * refinedT2 + f2 * refinedT;
        var slope = 1.0 / (3.0 * f0 * refinedT2 + 2.0 * f1 * refinedT + f2);
        refinedT -= (x - t) * slope;
        refinedT = Math.min(1, Math.max(0, refinedT));
    }
 
    // Resolve cubic bezier for the given x
    return 3 * Math.pow(1 - refinedT, 2) * refinedT * y1 +
            3 * (1 - refinedT) * Math.pow(refinedT, 2) * y2 +
            Math.pow(refinedT, 3);
};

This code is the implementation of the cubic bezier based on this definition and you can find the source of the simulator here.

Delay

The “transition-delay” line defines the delay between an update of a property and the start of the transition

Events

An event is raised at the end of a transition: “TransitionEnd”. According to your browser the correct name will be:

Chrome & Safari: webkitTransitionEnd
Firefox: mozTransitionEnd
Opera: oTransitionEnd
Internet Explorer: MSTransitionEnd

The event will give you the following information:

propertyName: Name of the animated property
elapsedTime: The amount of time the transition has been running, in seconds

Here is an usage sample for IE10:

block.addEventListener("MSTransitionEnd", onTransitionEvent);

More about CSS3 transitions

I can mainly propose two reasons why CSS3 transitions are really useful:

Hardware acceleration: CSS3 Transitions are directly handled on the GPU (where available) and produce smoother results. And it is really important on mobile devices where computing power is really limited
Better separation between code and design: For me, the developer must not be aware of animations or anything related to design. In the same way the designer/artist must not be aware of JavaScript. That’s why CSS3 Transitions are really interesting as designers can describe all the transitions in the CSS without needing developers

Support and fallback

Since PP3, IE10 (which you can download with Windows “8” Developer Preview here) supports CSS3 Transitions:

This report was produced by http://caniuse.com/#search=CSS3 transitions.

Of course, as the specification is not finished (working draft), you must use vendor’s prefixes such as –ms-, –moz-, –webkit-, –o-.

We can obviously see that we need to provide a transparent solution in order to address all kind of browsers. The best way will be to develop an API that can detect the support of CSS3 transitions. If the browser doesn’t support the feature, we will fallback to some JavaScript code.

It is important to support a fallback method if you rely on transitions for websites functionalities. If you don’t want to do that, you should consider using transitions only for design enhancements. In this case, the site will still work but only supported browsers will deliver the full experience. We speak here of “progressive enhancements” as the more powerfull the browser is, the more features he gets.

Transitions without CSS3 Transitions

So to be able to support a fallback to CSS3 Transitions, we will develop a small toolkit to provide transitions by code.

First of all, we will create a container object for our namespace:

var TRANSITIONSHELPER = TRANSITIONSHELPER || {};
 
TRANSITIONSHELPER.tickIntervalID = 0;
 
TRANSITIONSHELPER.easingFunctions = {
    linear:0,
    ease:1,
    easein:2,
    easeout:3,
    easeinout:4,
    custom:5
};
 
TRANSITIONSHELPER.currentTransitions = [];

To support the same level of easing functions, we must declare an “enum” with all required fields.

The toolkit is based on a function which is called every 17ms (to achieve animations at 60 fps). The function will enumerate through a collection of active transitions. For each transition the code will evaluate the next value given the current value and the target value.

We will need some handy functions to extract value of properties and units used:

TRANSITIONSHELPER.extractValue = function (string) {
    try {
        var result = parseFloat(string);
 
        if (isNaN(result)) {
            return 0;
        }
 
        return result;
    } catch (e) {
        return 0;
    }
};
 
TRANSITIONSHELPER.extractUnit = function (string) {
 
    // if value is empty we assume that it is px
    if (string == "") {
        return "px";
    }
 
    var value = TRANSITIONSHELPER.extractValue(string);
    var unit = string.replace(value, "");
 
    return unit;
};

The main function will process active transitions and will call the cubic bezier function to evaluate current values:

TRANSITIONSHELPER.tick = function () {
    // Processing transitions
    for (var index = 0; index < TRANSITIONSHELPER.currentTransitions.length; index++) {
        var transition = TRANSITIONSHELPER.currentTransitions[index];
 
        // compute new value
        var currentDate = (new Date).getTime();
        var diff = currentDate - transition.startDate;
 
        var step = diff / transition.duration;
        var offset = 1;
 
        // Timing function
        switch (transition.ease) {
            case TRANSITIONSHELPER.easingFunctions.linear:
                offset = TRANSITIONSHELPER.computeCubicBezierCurveInterpolation(step, 0, 0, 1.0, 1.0);
                break;
            case TRANSITIONSHELPER.easingFunctions.ease:
                offset = TRANSITIONSHELPER.computeCubicBezierCurveInterpolation(step, 0.25, 0.1, 0.25, 1.0);
                break;
            case TRANSITIONSHELPER.easingFunctions.easein:
                offset = TRANSITIONSHELPER.computeCubicBezierCurveInterpolation(step, 0.42, 0, 1.0, 1.0);
                break;
            case TRANSITIONSHELPER.easingFunctions.easeout:
                offset = TRANSITIONSHELPER.computeCubicBezierCurveInterpolation(step, 0, 0, 0.58, 1.0);
                break;
            case TRANSITIONSHELPER.easingFunctions.easeinout:
                offset = TRANSITIONSHELPER.computeCubicBezierCurveInterpolation(step, 0.42, 0, 0.58, 1.0);
                break;
            case TRANSITIONSHELPER.easingFunctions.custom:
                offset = TRANSITIONSHELPER.computeCubicBezierCurveInterpolation(step, transition.customEaseP1X, transition.customEaseP1Y, transition.customEaseP2X, transition.customEaseP2Y);
                break;
        }
 
        offset *= (transition.finalValue - transition.originalValue);
 
        var unit = TRANSITIONSHELPER.extractUnit(transition.target.style[transition.property]);
        var currentValue = transition.originalValue + offset;
 
        transition.currentDate = currentDate;
 
        // Dead transition?
        if (currentDate >= transition.startDate + transition.duration) {
            currentValue = transition.finalValue; // Clamping
            TRANSITIONSHELPER.currentTransitions.splice(index, 1); // Removing transition
            index--;
 
            // Completion event
            if (transition.onCompletion) {
                transition.onCompletion({propertyName:transition.property, elapsedTime:transition.duration});
            }
        }
 
        // Affect it
        transition.target.style[transition.property] = currentValue + unit;
    }
};

The current version of the toolkit only supports numeric values but if you want to animate complex values (such as color) you just have to decompose them to simple values.

Registering a transition in the system will be done using the following code:

TRANSITIONSHELPER.transition = function (target, property, newValue, duration, ease, customEaseP1X, customEaseP1Y, customEaseP2X, customEaseP2Y, onCompletion) {
 
    // Create a new transition
    var transition = {
        target: target,
        property: property,
        finalValue: newValue,
        originalValue: TRANSITIONSHELPER.extractValue(target.style[property]),
        duration: duration,
        startDate: (new Date).getTime(),
        currentDate: (new Date).getTime(),
        ease:ease,
        customEaseP1X:customEaseP1X,
        customEaseP2X:customEaseP2X,
        customEaseP1Y: customEaseP1Y,
        customEaseP2Y: customEaseP2Y,
        onCompletion: onCompletion
    };
 
    // Launching the tick service if required
    if (TRANSITIONSHELPER.tickIntervalID == 0) {
        TRANSITIONSHELPER.tickIntervalID = setInterval(TRANSITIONSHELPER.tick, 17);
    }
    
    // Remove previous transitions on same property and target
    for (var index = 0; index < TRANSITIONSHELPER.currentTransitions.length; index++) {
        var temp = TRANSITIONSHELPER.currentTransitions[index];
 
        if (temp.target === transition.target && temp.property === transition.property) {
            TRANSITIONSHELPER.currentTransitions.splice(index, 1);
            index--;
        }
    }
 
    // Register
    if (transition.originalValue != transition.finalValue) {
        TRANSITIONSHELPER.currentTransitions.push(transition);
    }
};

The “tick” function is launched when the first transition is activated.

Finally you just have to use modernizr to define if CSS3 Transitions is supported by the current browser. If not, you can fallback to our toolkit.

The code for the TransitionsHelper can be downloaded here: http://www.catuhe.com/msdn/transitions/transitionshelper.js

For example, in my puzzle game, the following code is used to animate the cells:

if (!PUZZLE.isTransitionsSupported) {
    TRANSITIONSHELPER.transition(block.div, "top", block.x * totalSize + offset, 500, TRANSITIONSHELPER.easingFunctions.ease);
    TRANSITIONSHELPER.transition(block.div, "left", block.y * totalSize + offset, 500, TRANSITIONSHELPER.easingFunctions.ease);
}
else {
    block.div.style.top = (block.x * totalSize + offset) + "px";
    block.div.style.left = (block.y * totalSize + offset) + "px";
}

We can note that I could use another way to animate my cells when CSS3 transitions are supported: I could have defined a collection of CSS3 classes with predefined left and top values (one for each cell) to affect them to right cells.

Some frameworks and toolkits already exist to support software transitions:

jQuery.transition.js: http://louisremi.github.com/jquery.transition.js/test/index.html
jQUery-Animate-Enhanced: https://github.com/benbarnett/jQuery-Animate-Enhanced

By the way, you can also use the old good animate() method of jQuery.

Conclusion

As we saw, CSS3 Transitions is a really easy way to add animations to your project. You can produce a more reactive application just by using some transitions when you want to change values.

By the way, there are two solutions if you want to implement a JavaScript fallback:

You can do all in the JavaScript side and if you detect the support of CSS3 transitions, you will inject CSS3 declarations in the page.
Or you can use standard way (using true CSS3 declarations in the CSS files) and just detect the need of fallback in JavaScript. For me, it is the better option as the fallback must be an option and not the main subject. In a near future, all browsers will support CSS3 Transitions and in this case you will just have to remove your fallback code. Furthermore, it is a better way to let all the CSS under the control of the creative team and not in the code part.

Going further

Blog about CSS3 Animations by David Rousset: http://blogs.msdn.com/b/davrous/archive/2011/12/06/introduction-to-css3-animations.aspx
CSS3 Transitions specifications: http://www.w3.org/TR/css3-transitions/
IE Test Drive on CSS3 transitions: http://ie.microsoft.com/testdrive/Graphics/hands-on-css3/hands-on_transitions.htm
Others useful posts: