The MossyBlog Times Archives 2007 - 2009 : Skinning

POC #1 - Project Harmony (pt1) (Silverlight & Flash)

scbarnes — Thu, 15 Nov 2007 11:18:00 GMT

I've been enjoying the overall experience of what Silverlight has to offer since I started with Microsoft back in January 07. That being said, I've often been an internal "goto" person at times for some Flash advice on behalf of Microsoft customers around the world.

In light of this, recently I also encountered a situation where a customer was looking to use both Silverlight and Flash together, something which I kind of gave a confused look at? (ie .. not because I wanted them to pick ours etc, but more to the point what was the root of the problem).

The problem was simply that they had deadline to meet, they wanted to retain the video in Windows Media format but were looking to use the GUI inside Flash as a base. Fair enough, this isn't a Zero Sum Game and we play well with others.

This isn't the first I've seen similar stories around this, but mostly its from Design shops around the world whom are keen to embrace Silverlight and Flash (fingers in both barrels) but are kind of skittish to start with (i.e. the assumption is really you have to pick a team, red vs blue).

Thus, I decided to put together this proof of concept, titled "Harmony". The project is quite extensive and I'll spend the next month dissecting it and talking about different cross-sections of how you as a designer & developer are able to produce compelling enriching experiences with Silverlight and that you can execute on your creative vision. The rest is simply semantics and bits/bolts.

In this first iteration of my Project "Harmony" I wanted to test to see how well the performance would be in the event one was to use 90% Silverlight and 10% flash. In that overlay Flash on top of Silverlight and see what happens.

The results speak for themselves, suffice to say that I really put in a lot of gratuitous animations in Silverlight to really try and break the performance overall.

You can view for yourself here:

http://www.demo.members.winisp.net/poc1/

(Note: Link Requires Silverlight 1.1 and Flash 9.x)

My Objectives for this first iteration was:

Objectives (v1.0)

Determine to see if Flash (Transparent windowless) degrades in performance in the event it sits on top of Silverlight.
Determine if PNG overlays impact the performance of the video whilst mixed with Flash.

Conclusion (v1.0)

I did see some performance degrading on both technologies, but that is mostly due to both making use of alpha transparency (I'm yet to meet a runtime of any brand that can withstand alpha transparency)
Working inside Flash & then crossing over to Visual Studio / Expression Blend is not for the feint hearted (I kept tripping up in C# vs AS3.0 scripting syntax hooks hehe - nothing against both technologies but its more of a mind map thing).

Highlights (v1.0)

Using PNG's a designer is capable of doing some interesting things to a graphical interface. In this case used a bullet that appears to not only be embedded into the TV Screen but also shatters it.

I also added the "bullets" in the top right corner as PNG overlay to also test how 2xOverlays inside the Video would perform.
Using Video Reflection within Silverlight, one is able to really bring home a gloss finish by giving it a "glass" effect (as we all know there isn't enough glass effects online
Using an infinite loop within Silverlight, I was able to create a "ambiance" feel to the UI by making two sets of images fade in/out underneath the Flash area and the Silverlight Video. This again, was to really see how the performance would go with a Silverlight animation underneath Flash.

Using Adobe Photoshop for the bitmap artwork, I stumbled upon a technique whereby if you take a photo, duplicate its layer and then apply "gussian blur" with 50% transparency it kind of gives this glow/ambiance effect commonly seen in most games today.

Silverlight (Video aside of course) handles its progressive loading of assets in a unique way, I'm yet to put my finger on it but I must look into this more as I expected the GUI to take much longer to load (as there aren't any preloaders in this design).
The Flash Slider (ie works like iTunes) was done reasonably quickly, and the code has already been ported to Silverlight suffice to say, it looks seamlessly integrated within the GUI and unless you don't have Silverlight or Flash installed, it would argueably be forgiven as being part of one technology not 2.

It's a small experiment at first, mainly to test the waters and see what holds together initially.

The walk away message here is simple, this isn't a Zero Sum Game firstly and I want to stress that the most. The other impact is that if you're keen to try Silverlight but aren't quite ready for that big leap, it's ok you can use both without penalties implied.

The last but most important of all, is prove your concepts initially. In that if you're looking to switch from using Flash (for whatever reason) and aren't bold enough to do the lock stock and barrel or aren't allowed to, that's ok. Establish some objectives initially, build a Proof of Concept (POC) and chip away at the idea, as you will no doubt learn both technologies limitations faster than any book can really teach you.

The secret of a truly successful RIA in my opinion is knowing the technologies limitations.

The next POC (pt2) I'll talk more about the JavaScript Bridge and how I was able to make the Thumbnail Slide load up the Video Playlist etc. I'm also putting together some Video Casts on not only how I made this but also will upload the code once I get it locked into Codeplex.com (open source). All code & design is hand-made by myself only.

(NOTE: This is a crude, raw POC and progress bars etc for both technologies aren't important for this initial purpose. They are coming, and I'll talk more about that later. Again, this is a POC so don't lock this down as some end to end final production grade solution).

(NOTE: I used Teamfortress 2 Artwork, because it's my favorite game at the moment, I have personal history with it and well, the Soldier video is by far the funniest game trailer I've seen in quite some time. Valve Software are legends in my book - My Steam ID is "[te] Skittlez")

Next: The "HOW" (Code Dump / VideoCast).

Related Post: Writing Proof of Concept RIA's

RIA, building your control framework.

scbarnes — Mon, 13 Aug 2007 09:38:40 GMT

Silverlight 1.0 is around the corner, and it won't have controls assigned to it upon official release. Yet, this is something very important to the success of Silverlight going forward, as without a control framework there is really no point right?

This is where Silverlight 1.1 will come through strong, and win over some folks with its upcoming new pieces to the first iteration puzzle. Yet, already 3rd Party vendors are producing controls today and automatically folks whom have been waiting impatiently for controls in Silverlight are likely to flock to these vendors.

Is this bad? not really, people want to build today, not tomorrow and so it's understandable. Yet, you can also build your own framework today should you wish to. The primitives in Silverlight have been in place for quite some time, and they will be available for 1.0 (with some extra benefits thrown in 1.1 of course).

The key ingredients to a framework are simply the following:

Text Input
Mouse Events
Support for External Media / Graphic elements
Animation / Threaded timed looping.

Armed with these, you could produce an entire framework ranging from a button all the way up to a tabbed layout. In fact, I've seen some folks produce Accordion Panels already. Yet, their is a lot to take on and namely it's most difficult piece to overcoming complexity is how do you not only make the framework work, but also for developers & designers at the same time?

Here are some considerations you should undertake if you decide to build a control / application framework tomorrow in Silverlight or Windows Presentation Foundation (WPF).

Part 1 - Skinning

Skinning and styling are somewhat confused with one another at times. In that changing a Font Color for a button label is not skinning. Adding your own custom texture to the surface of the button is skinning. It's can be both an easy and complex task to achieve, as when you think about a Button for example, it initially has 4 states associated to it.

Each state represents how the button will be displayed. Typically the default state shown when a user first see's a button is that of either "Normal" or "Disabled" depending on the context of use. This is basic skinning 101 style stuff, and the approach of a RIA Producer is to look at ways of injecting their own style of button without having to bust open lines of code to do so. Ideally the best option would be to see the button state in a tree format, then when you click on the "Normal" node, you are asked a bunch of questions around how you would like to apply skinning techniques to produce an expected result. This can either be done as part of an XML Editor of your own (which in turn spits out say XAML) or it can be done via a design tool (Expression Design / Blend ).

This is basic skinning, yet the more creative souls you work with, the more control they wish to rule over the skins. An example would be a design in which I want a Modal window to display.

In the Diagram to the Right, there are two pieces that are to be displayed inside a RIA (Rich Interactive Application) application (wrote many moons ago in Adobe Flex).

The first piece, represents the outer shell, in that this was to be the "overall" window in which I would dock my graphical interface into.

The second piece (smaller window) represented how my Modal/Non-Modal windows would be displayed.

Each of these pieces had a variety of display states, and the objective for me was to change the colors depending on how the user interacted with the application in question. In that they were not only context aware but also stretched according to screen real-estate.

As you can imagine the complexity of skinning just went up a few points, and doing this inside Adobe Flex at the time was quite hard (in fact I had to re-write a lot of the framework to adhere to specific skinning principals of my own). Yet, all I was doing was inserting an array of skin states to the equation?

It comes back to Skinning is something which is quite fluid and needs to be looked at from all sorts of angles. You're dealing with both a designer - and - a developer in one hit, so you need to provide turn-keys if you will to empower them in choosing multiple states, with multiple stretching techniques that can be bolted on via an array of modes - which in turn are flipped on/off through different event contexts.

In looking at the above graphic, you'll also note that there is a lot of static gray in the design? Should this change per state change or should this stay static. We could easily argue that the colored gradients are realistically the changing pieces and therefore can be adjusted by swapping in respective PNG files (alpha transparency remaining per color context). This the smart approach, and probably the least expensive (given there will need to be an entire update to the overall UI should one chose to repaint the gray bits every time).

Overall, the point is there are many variables at play and to approach skinning through a "4-state" update sequence for example, is quite immature and not realistically helpful to the RIA Producer of tomorrow.

The way forward is simple, there needs to be a GUIStateManager Class, in which has nodes. Think of these nodes as you would a tree control. Each node has a cluster of elements associated to it, yet each node can be shared or duplicated (duplication is much easier but can be expensive for memory). A Designer needs to assign these nodes to a control / layout of their respective choosing (in my case, a Window). The trick is you need to think of ways in which you can empower your developers & designers of tomorrow in making this happen, as it's not good enough to build a prescribed skinned control framework like many others have done in the past.

As this defeats the whole purpose of why Silverlight and WPF exist, and using Expression Design + Expression Blend there are strong hints to a more streamlined approach to building a complex hierarchy of skin states, whilst providing flexibility in choosing how each state should be displayed per control.

You can view a more structured example of the Window via:

http://www.mossyblog.com/archives/614.cfm
(My Originating blog post back in my Adobe Flex days)

http://www.mossyblog.com/SynergyFLEX/v2_0_1/SynergyFLEX.html
(See the above GUI inside Adobe FLEX, scaling according to your screen resolution)

Next... Part 2 - Stretching...

The MossyBlog Times Archives 2007 - 2009 : Skinning

POC #1 - Project Harmony (pt1) (Silverlight & Flash)

My Objectives for this first iteration was:

Objectives (v1.0)

Conclusion (v1.0)

Highlights (v1.0)

RIA, building your control framework.

Part 1 - Skinning