Jaime Rodriguez On Windows Phone, Windows Presentation Foundation, Silverlight and Windows 7
I had to learn DeepZoom recently and along the way I put together some handy notes .. below are the notes organized in a near step-by-step explanation format. This is a very long post, but I hope it has useful insights for any one wanting to do deepzoom so I recommend you read it all. If you must skip, then the outline will help you. imo, part 3 and 5 are the good stuff.
Part 1 – The history and brief explanation on how DeepZoom works.
Part 2 – Constructing a DeepZoom image using Image Composer
Part 3 – Introduction to the DeepZoom object model – goes way beyond the docs I hope
Part 4 – Coding a deepZoom ‘host’ user control with Pan & Zoom
Part 4.1 – Adding a few extra features to our User Control
Part 5 – Lessons learned on the code, documenting the gotchas **must read even if you know Deep Zoom already
Part 6 – Give me the code, just a zip file w/ the goodies
Part 7 – Show me the outcome; what did we build?
A lot of people equate DeepZoom to SeaDragon – they assume SeaDragon was the code name and DeepZoom is the marketing name-. This assumption is not quite right (unless you equate your engine to your car model). Seadragon is an incubation project resulting from the acquisition of Seadragon Software; the Seadragon team is part of the Live organization and are working on several projects (like Photosynth). DeepZoom is an implementation that exposes some of the SeaDragon technology to Silverlight.
DeepZoom provides the ability to smoothly present and navigate large amounts of visual information (images) regardless of the size of the size of the data, and optimizing the bandwidth available to download it.
How does DeepZoom work? DeepZoom accomplishes its goal by partitioning an image (or a composition of images) into tiles. While tiling the image, the composer also creates a pyramid of lower resolution tiles for the original composition.
The image to the right shows you what a pyramid; the original image is lowest in the pyramid, notice how it is tiled into smaller images, also notice the pyramid creates lower resolution images (also tiled). A few of the docs I read said the tiles are 256x256, but from peeking through the files generated by the composer I am not convinced; I do know from reading through the internal stuff that there is some heavy math involved here, so I trust they tile for right size :).
All of this tiling is performed at design-time and gets accomplished using the DeepZoom composer.
At run-time a MultiScaleImage downloads a lower resolution tile of the image first and downloads the other images on demand (as you pan or zoom); DeepZoom make sure the transitions from lower to higher res images are smooth and seamless.
Given all this, how is Deepzoom different than say a ScaleTransform (for zoom) on a high resolution image?
With a ScaleTransform, usually you would download the whole high res image at once; this delays how quickly the end user gets to see the image when the page or application loads. Some times people apply a trick where you use different resolutions images, since you are not tiled you will likely end up downloading several big images (consuming more network bandwidth) or the download time will continue to be high if the initial downloaded image is not small enough, also the transitions from low to higher res are going to be more noticeable unless your write the transitions yourself.
DeepZoom and its tiling make it possible to see bits quicker and can optimize for bandwidth. In the worst case scenario where some one looked at every single one of the tiles at the highest resolution, DeepZoom would have an extra overhead of 33% compared to downloading the single highest resolution image at once, but this ‘worst case’ scenario is almost never hit, most of the time DeepZoom can save you from downloading too much.
Another feature in DeepZoom is its ability to create ‘collections’ from the composite image. This provides you the ability to compose a scene ( group of images ), optimize them for speed & download, but still maintain the ‘autonomy’ and identity of the image, you can programmatically manipulate (or position) these images from within the DeepZoom collection (more on collections in part 4).
Once you have a DeepZoom image, you will need an instance of the MultiScaleImage class in your silverlight application to load that image. Instantiating the MultiScaleImage class can be done from XAML
<MultiScaleImage x:Name="DeepZoom" Source="easter/info.bin" />
or from code:
MultiScaleImage DeepZoom = new MultiScaleImage () ;
DeepZoom.Source = new Uri ( “easter/info.bin”) ;
Before going through the DeepZoom API it makes sense to understand the terminology used:
Now, we navigate through the interesting properties and methods in MultiScaleImage
The interesting methods are:
In my opinion, surprisingly missing from the API were:
The goal here is to code a sample reusable control just to illustrate the points; along the way we will of course implement enough features for our Easter Egg Hunt. [Update: Sorry about belatedness, I started this on 3/22 but had a trip that prevented me from playing around, so I am late from easter]
<UserControl x:Class="DeepZoomSample.Page" xmlns="http://schemas.microsoft.com/client/2007" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" > <Grid x:Name="LayoutRoot" Background="White"> <MultiScaleImage x:Name="DeepZoom" Source="easter/info.bin" /> </Grid> </UserControl>
using System; using System.Windows; using System.Windows.Controls; using System.Windows.Documents; using System.Windows.Ink; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Animation; using System.Windows.Shapes; using System.Windows.Browser;
namespace DeepZoomSample { // this code came from Peter Blois, http://www.blois.us/blog // Code ported by Pete blois from Javascript version at http://adomas.org/javascript-mouse-wheel/ public class MouseWheelEventArgs : EventArgs { private double delta; private bool handled = false;
public MouseWheelEventArgs(double delta) { this.delta = delta; }
public double Delta { get { return this.delta; } }
// Use handled to prevent the default browser behavior! public bool Handled { get { return this.handled; } set { this.handled = value; } } }
public class MouseWheelHelper {
public event EventHandler<MouseWheelEventArgs> Moved; private static Worker worker; private bool isMouseOver = false;
public MouseWheelHelper(FrameworkElement element) {
if (MouseWheelHelper.worker == null) MouseWheelHelper.worker = new Worker();
MouseWheelHelper.worker.Moved += this.HandleMouseWheel;
element.MouseEnter += this.HandleMouseEnter; element.MouseLeave += this.HandleMouseLeave; element.MouseMove += this.HandleMouseMove; }
private void HandleMouseWheel(object sender, MouseWheelEventArgs args) { if (this.isMouseOver) this.Moved(this, args); }
private void HandleMouseEnter(object sender, EventArgs e) { this.isMouseOver = true; }
private void HandleMouseLeave(object sender, EventArgs e) { this.isMouseOver = false; }
private void HandleMouseMove(object sender, EventArgs e) { this.isMouseOver = true; }
private class Worker {
public event EventHandler<MouseWheelEventArgs> Moved;
public Worker() {
if (HtmlPage.IsEnabled) { HtmlPage.Window.AttachEvent("DOMMouseScroll", this.HandleMouseWheel); HtmlPage.Window.AttachEvent("onmousewheel", this.HandleMouseWheel); HtmlPage.Document.AttachEvent("onmousewheel", this.HandleMouseWheel); }
}
private void HandleMouseWheel(object sender, HtmlEventArgs args) { double delta = 0;
ScriptObject eventObj = args.EventObject;
if (eventObj.GetProperty("wheelDelta") != null) { delta = ((double)eventObj.GetProperty("wheelDelta")) / 120;
if (HtmlPage.Window.GetProperty("opera") != null) delta = -delta; } else if (eventObj.GetProperty("detail") != null) { delta = -((double)eventObj.GetProperty("detail")) / 3;
if (HtmlPage.BrowserInformation.UserAgent.IndexOf("Macintosh") != -1) delta = delta * 3; }
if (delta != 0 && this.Moved != null) { MouseWheelEventArgs wheelArgs = new MouseWheelEventArgs(delta); this.Moved(this, wheelArgs);
if (wheelArgs.Handled) args.PreventDefault(); } } } } }
protected double _defaultZoom = 1.3; public double DefaultZoomFactor { get { return _defaultZoom; } set { _defaultZoom = value; } }
private double _currentTotalZoom = 1.0;
public double CurrentTotalZoom { get { return _currentTotalZoom; } set { _currentTotalZoom = value; } }
private double _maxZoomIn = 5000; protected double MaxZoomIn { get { return _maxZoomIn; } set { _maxZoomIn = value; } } private double _maxZoomOut = 0.001;
protected double MaxZoomOut { get { return _maxZoomOut; } set { _maxZoomOut = value; } }
/// <summary> /// Performs a Zoom operation relative to where Image is at. /// Example, call DoZoom twice with a Zoom of 1.25 will lead to an image that is zoomed at /// 1.25 after first time and ( 1.25 * 1.25 for second time, which is a 1.56 /// </summary> /// <param name="relativeZoom"> new zoom level; this is a RELATIVE value not absolute.</param> /// <param name="elementPoint"></param> void DoZoom(double relativeZoom , Point elementPoint) { if ( _currentTotalZoom * relativeZoom < MaxZoomOut || _currentTotalZoom * relativeZoom > MaxZoomIn) return; Point p = DeepZoom.ElementToLogicalPoint(elementPoint); DeepZoom.ZoomAboutLogicalPoint(relativeZoom, p.X, p.Y); this.Zoom = relativeZoom; _currentTotalZoom *= relativeZoom; }
void OnMouseWheelMoved(object sender, MouseWheelEventArgs e) { // e.Delta > 0 == wheel moved away, zoom in if (e.Delta > 0) { DoZoom( DefaultZoomFactor, _lastMousePosition); } else { // Zoom out DoZoom( 1/ DefaultZoomFactor, _lastMousePosition); } }
protected bool _isDragging = false; protected Point _lastDragViewportOrigin; void DeepZoom_MouseLeftButtonDown(object sender, MouseButtonEventArgs e) { this._lastDragViewportOrigin = DeepZoom.ViewportOrigin; this._lastMousePosition = e.GetPosition(DeepZoom); this._isDragging = true; }
void DeepZoom_MouseMove(object sender, MouseEventArgs e) { if (_isDragging) { Point newViewport = _lastDragViewportOrigin; Point currentMousePosition = e.GetPosition(DeepZoom); newViewport.X += (_lastMousePosition.X - currentMousePosition.X) / this.DeepZoom.ActualWidth * this.DeepZoom.ViewportWidth; newViewport.Y += (_lastMousePosition.Y - currentMousePosition.Y) / this.DeepZoom.ActualWidth * this.DeepZoom.ViewportWidth; this.DeepZoom.ViewportOrigin = newViewport; _lastDragViewportOrigin = newViewport; } // NOTE: it is important this be after the isDragging check … // since this updates last position, which is used to compare for dragging. _lastMousePosition = e.GetPosition(DeepZoom); }
In the last sections I took it slow and walked through the code to explain what we were working on. Going forward below will pick up the pace a bit, and the original code will be tweaked to get into a host control with a bit more navigation and troubleshooting advise.
this.PanRight.Click += new RoutedEventHandler(PanRight_Click); this.PanLeft.Click += new RoutedEventHandler(PanLeft_Click); this.Home.Click += new RoutedEventHandler(Home_Click); this.PanBottom.Click += new RoutedEventHandler(PanBottom_Click); this.PanTop.Click += new RoutedEventHandler(PanTop_Click);
void Pan(PanDirection direction) { double percent = PanPercent; if ( UseViewportScaleOnPan ) percent *= this.DeepZoom.ViewportWidth; switch (direction) { case PanDirection.East: this.DeepZoom.ViewportOrigin = new Point(this.DeepZoom.ViewportOrigin.X - Math.Min(percent, this.DeepZoom.ViewportOrigin.X), this.DeepZoom.ViewportOrigin.Y); break; case PanDirection.West: this.DeepZoom.ViewportOrigin = new Point(this.DeepZoom.ViewportOrigin.X + Math.Min(percent, (1.0 - this.DeepZoom.ViewportOrigin.X)), this.DeepZoom.ViewportOrigin.Y); break; case PanDirection.South : this.DeepZoom.ViewportOrigin = new Point(this.DeepZoom.ViewportOrigin.X , this.DeepZoom.ViewportOrigin.Y + Math.Min( percent, 1.0 - this.DeepZoom.ViewportOrigin.Y)); break; case PanDirection.North : this.DeepZoom.ViewportOrigin = new Point(this.DeepZoom.ViewportOrigin.X, this.DeepZoom.ViewportOrigin.Y - Math.Min( percent, this.DeepZoom.ViewportOrigin.Y)); break; } }
/// <summary> /// Zooms in around an ELEMENT Coordinate.. /// I technically did not need this function to abstract and could have called DoZoom directly /// </summary> /// <param name="p"></param> void ZoomIn( Point p ) { if (_useRelatives) DoRelativeZoom(Zoom / DefaultZoomFactor, p, ZoomDirection.In); else DoZoom(DefaultZoomFactor, p ); }
this.LayoutRoot.KeyUp += new KeyEventHandler(DeepZoom_KeyUp);
void DeepZoom_KeyUp(object sender, KeyEventArgs e) { if (e.Handled == true) return; if ( ( RunningOnWindows && ((Keyboard.Modifiers & ModifierKeys.Control) == ModifierKeys.Control) && (e.Key == Key.Add || (e.Key == Key.Unknown && e.PlatformKeyCode == 0xBB))) || ( RunningOnMac && ((Keyboard.Modifiers & ModifierKeys.Control) == ModifierKeys.Control) && (e.Key == Key.Add || (e.Key == Key.Unknown && e.PlatformKeyCode == 0x18 ))) ) { ZoomIn( _lastMousePosition); } else if ( (RunningOnWindows && ((Keyboard.Modifiers & ModifierKeys.Control) == ModifierKeys.Control) && (e.Key == Key.Add || (e.Key == Key.Unknown && e.PlatformKeyCode == 0xBD)))|| ( RunningOnMac && ((Keyboard.Modifiers & ModifierKeys.Control) == ModifierKeys.Control) && (e.Key == Key.Add || (e.Key == Key.Unknown && e.PlatformKeyCode == 0x1B ))) ) { ZoomOut( _lastMousePosition ); } e.Handled = true; }
void PullData(ref MultiScaleImageDummyDataWrapper data, ref MultiScaleImage msi) { data.ViewportWidth = msi.ViewportWidth; data.ViewportOrigin = msi.ViewportOrigin; data.AspectRatio = msi.AspectRatio; data.UseSprings = msi.UseSprings; }
Overall I was quite impressed with DeepZoom. it is pretty cool stuff; I wish I had some cool pictures for a better application, but I did not try since I knew I could not top memorabilia.
My personal advise: do not databind to DeepZoom for now. Pull to it on Motion Finished.
No keyboard input goes into DeepZoom (since it is a FrameworkElement). In order to have keyboard input you must have a Control that has focus; since keyboard events bubble you can handle Keyboard input at a higher level (e.gl LayoutRoot, just check to see if it has not been handled previously).
On my real app –which can’t be shared as it was a customer’s app –. I ran into an issue when using Collections. My images were showing up in the wrong place. I reported it already and they are investigating –during the shower, where I do my best thinking- I came with the theory that is the resolution independence in WPF ( 96 to 72 DPI conversion). I have not confirmed.
I did not discuss collections in the post, so will try it here. Collections are cool because it gives you access to your Subimages so you can manipulate them. Move them around, scale them, animate position and Opacity. For now, beta1 has only one Collection; I think it would be cool to have multiple collections so you can aggregate. This can kind of be simulated via logic, but would be nice if it was in the control. If you simulate it, the advise I was given is do not simulate it by overlaying two MultiScaleImage controls one on top of the other, there are a few known issues with interactions on overlays (though to be honest I tried it and did not run into issues).
UseSprings= true is pretty cool, but pending how quick you want to do your panning/zooming, turning it off can make your app appear more responsive. I would not turn UseSprings off for a consumer facing app, but I would consider doing it for an internal app.. For example, I am doing a Heatmap with lots of data in it, for analytical purposes. Since it is drill through I am considering it.
When panning, make sure you handle MouseLeave on your control.
Handling mouse wheel is not available out of the box is trivial but Peter Blois has a great solution. Do not write the code to handle wheel. Peter’s code works great so far. Check his blog for updates too, he has a nice abstraction now to the same API.
If you skipped section 3, check it out. Understanding the object model is critical and takes 5 mins.
If you are writing a DeepZoom application, I recommend you use the old instantiation via silverlight.js … Click To Activate will eventually go away in IE, but in the mean time it is pretty annoying for an app that is so visual and so focused on mouse navigation.
If subscribing to MultiScaleImage.OpenImageSucceded make sure you do it from your constructor right after initializeComponent. I tried to do it of UserControl.Loaded and when doing a load on a page with image cached that is too late.
If possible try to ‘hold’ any operations until OpenImageSucceded has fired ( no pans, zooms before that). I saw weird results if I try to access properties on MultiScaleImage before this event; in particular if you access the SubImages collection before ImageOpenSucceded, then I would get an empty collection and when ImageOpenSucceeded was fired, the collection would not be overridden; so advise for collections is don’t touch SubImages before the OpenImageSucceeded fires.
Is at Skydrive
You can see it here; it is not visually impressive but I think it shows a bit of what you can do with DeepZoom and most important it is functional code you can quickly refactor and reuse. If I missed a common deepZoom task let me know. I added two extra “easter eggs” beyond the bunny and the eggs above in the walk through.
Thanks to Tim Aidlin who chose the colors and gave me cooler icons for the map; I butchered them a bit when I turned them into controls so don’t hold it against him, he is a gifted designer –you can see his real work on the MIX website and any thing else MIX08 branded.
PingBack from http://kylenews.101freehost.com/30aproperties.html
Jamie,
Super job!! I'm going to use this tutorial and get DeepZoom on my web site after the MVP Summit. This is wild!
Nice advertising your availability in the evenings while your in Europe!
Best to you!
Karl
On my Deep zoom Post , I recommended that for Deep Zoom applications you instantiate the control using
Deep Zoom which is a technology that combines Silverlight with SeaDragon is a powerful and easy way to
Build your own Silverlight 2.0 Deepzoom application and host it with "Silverlight Streaming"
Ever since we released Deep Zoom Composer during MIX, there has been a ton of great feedback you have
Quite a few people provided me feedback on my Deep Zoom post last month . With the help from Jamie Rodriguez's
Quite a few people provided me feedback on my Deep Zoom post last month . With the help from Jamie Rodriguez's
Today we covered Deep Zoom, based on Seadragon technology, which allows the users of our Silverlight
DeepZoom is a cool feature of Silverlight 2.0 allowing the user to zoom and pan around an image, while
This has been talked about since the ESRI Developer Summit 2008 earlier this year and earlier this week Richie Carmichael wrote up about a new Microsoft Silverlight map viewer that he has developed for ArcGIS Server 9.3. The Silverlight map viewer is
Since we released Deep Zoom Composer a while ago at MIX, we relied extensively on all of you to both
para quem quiser se aprofundar no Deep Zoom (ops, escolha estranha de verbo), aqui vão algumas referências