Microsoft Live Labs just released a new free service at seadragon.com. It lets you transform any web image into a Seadragon-powered zoomable photo experience, then embed the results anywhere you want. With Seadragon.com, you can share full resolution photos that are fast, easy and fun to explore, regardless of the web browser or platform.
Take a look at this photo I shot at Yosemite National Park last summer. Click on this image to zoom in, or use your mouse wheel or the + and – buttons. Pan by simply dragging the image with your mouse. Click the home button to restore the view or click the right-most button to expand the image to the full browser window. Can you find the two climbers approaching the summit of El Capitan?
It’s really easy to create your own zoomable photos. It’s free and it doesn’t require you to sign up for anything. All you need is the URL to your web image. (The photo above comes from my Flickr stream.) Paste the URL at seadragon.com and click one button. Depending on the resolution of your image, it may take a moment or two to create the zoomable version. You can then copy the link to the zoomable viewer page for your image on seadragon.com and share it on Twitter, your Facebook status, or anywhere else you choose. Or, you can do what I did for the photo above and copy a one-line script that lets you embed the zoomable viewer on your own web page, blog posting, eBay advertisement, or other web destination.
The Seadragon viewer isn’t your average zoomable viewer; it includes some cool new technology that delivers stunning performance but still makes it accessible to virtually everyone. If you have Silverlight installed (and if you don’t, you should), the viewer uses Silverlight Deep Zoom to deliver a buttery-smooth full screen zooming experience. But even if you don’t have Silverlight, the viewer will still operate using our Seadragon Ajax technology. The appearance and features are identical, but the performance will be a bit slower than with Silverlight Deep Zoom. This new dual-mode viewer makes sharing high quality, full resolution images easier than ever, without turning away any of your potential audience.
Now you can easily share your full resolution digital photos and provide a seamless, easy-to-use and flexible viewing experience for everyone. Rather than just showing a static low resolution preview, that same preview can magically transform into the full resolution image. And all of this happens without the pain of waiting for long downloads and slow Internet response time. You can dramatically improve the way you share your individual photos on the web.
For more information, including some behind the scenes details, read the blog post at www.livelabs.com as well as this blog post from inside the Seadragon team. Or just visit www.seadragon.com and try it out for yourself!
I’m very excited to share the news that JPEG XR is now an approved ISO/IEC International standard. (It’s official designation is ISO/IEC 29199-2.) In addition, JPEG XR has also been approved as ITU Recommendation T.832. You can find more information in the press release from the JPEG Committee.
As I’ve discussed in previous blog posts here, JPEG XR is the standardized incarnation of HD Photo, an innovative new format for digital photos developed at Microsoft, and first shipped in Windows Vista. There are a few minor differences between HD Photo and JPEG XR; these changes were the result of some excellent work by the JPEG Committee that helped insure the highest possible quality for the approved standard.
JPEG XR offers some dramatic benefits when compared to the original JPEG file format that we all know and love:
- Better compression – JPEG XR offers improved efficiency compared to JPEG, and the type of compression artifacts are often less objectionable than the typical JPEG compression artifacts. JPEG XR offers a very wide range of compression levels, including perceptively lossless or mathematically lossless compression. Regardless of your requirements, JPEG XR probably offers a compression option that’s ideal for that scenario.
- More image formats – JPEG XR supports 8bpc (bits per channel), 16bpc and 32bpc, as well as several special bit depth formats. Pixel values can be stored as either integers, scaled fixed point numbers or full floating point values; this provides full support for numerous high dynamic range (HDR) imaging scenarios, as well as support for wide gamut color spaces. In addition to 3-channel RGB, JPEG XR supports monochrome, CMYK and n-channel formats up to 16 independent channels. many of these formats also support an alpha channel. This wide range of image formats allows for dramatically better image quality and allows this single new file format to effectively replace many previous formats that were required for specific scenarios.
- Advanced decoding features – JPEG XR provides progressive decoding, allowing lower resolution previews or specific cropped areas to be displayed without the need to decode the entire image. Additionally, JPEG XR images can be cropped, rotated, flipped and resized (within certain constraints) without ever needing to decode and then re-encode the image. That means these operations are much, much faster and no additional image quality is lost due to the additional encoding steps.
The many innovations delivered in JPEG XR originated from over ten years of research and development at Microsoft. And through our participation as a member of the JPEG Committee, we’ve been able to deliver additional improvements and create an International standard available to everyone.
So the next question everyone asks is “when will it be available in a camera?” Since I don’t work for a camera company, I can’t answer that question. But I’m pleased and proud that because JPEG XR is now a free International standard, there’s very little standing in the way of making that happen!
(Updated 7/30/09 to correct the IEC logo.)
The JPEG XR Technical Spec Work is Done
The headline for this blog post is the headline of the press release from the most recent meeting of the Joint Photographic Expert’s Group (JPEG) Committee. The JPEG XR standard has advanced to Draft International standard balloting. Basically, that means the specification is frozen for implementers; there will not be any more technical changes. The press release goes on to say that International Standardization is expected to be completed this year. Woohoo!
Here’s the excerpt from the press release discussing JPEG XR:
JPEG XR is a new compression format supporting high dynamic range and promises to bring a new level of successful user experience to users of digital cameras. JPEG XR advanced to draft international standard balloting at the San Francisco meeting, entering the final phases of standardization and freezing the specification for implementers. The committee expects the JPEG XR International Standard (IS) to be published later this year. The JPEG committee is hosting a meeting to provide a special preview of the standard and discuss possible approaches to raw file compression. JPEG XR offers the potential to give cost and compression benefits to users producing high-quality, high dynamic-range images.
The complete press release from the meeting is here: http://jpeg.org/newsrel24.html
This is really exciting news! As most followers of this blog know, JPEG XR is based directly on HD Photo, the new photography file format that Microsoft introduced with the launch of Windows Vista. JPEG XR offers some great benefits for photographers compared to JPEG, or other file formats. It’s great to see this technology is now being embraced as an International standard. I expect this will dramatically accelerate its adoption in cameras and digital photography software and services.
On a Personal Note… The Return of Bill’s Blog
| Yes, this is indeed my first blog post in nearly 14 months. That’s sad, and certainly not the way to manage a blog that you expect anyone to spend the time to read. It’s time to fix that. A lot has been happening in the last year. First, I changed jobs. As the development work on HD Photo reached an end and the efforts transitioned to standardization, I moved on to a different challenge. Since last November I’ve been a Group Manager at Microsoft Live Labs, responsible for the Seadragon Technology. I’ve continued to stay involved in JPEG XR standardization, but HD Photo and JPEG XR is no longer my primary job responsibility. While several folks from Microsoft been actively involved, JPEG XR standardization belongs to the JPEG Committee, not to Microsoft. It is the committee's responsibility to provide any all public information about the overall standards process and status of the JPEG XR working group. So, up until now (with the conclusion of the technical standards development work), there really wasn’t a whole lot of HD Photo information for me to blog about. The work we’re doing in the Seadragon team builds directly on some of the key objectives we first defined for HD Photo. Our team is extremely excited by JPEG XR and will be working hard to take advantage of it’s advanced capabilities. My goal is to provide more frequent updates here on the blog. At the same time, I’m going to expand the focus to a broader discussion of digital photography and imaging technology, with a strong focus on JPEG XR, Seadragon (including Silverlight Deep Zoom, Photosynth, Seadragon Ajax and Seadragon Mobile), and related technology and industry activities. Your ideas and suggestions are always welcome! Bill Crow
Seadragon Group Manager
Microsoft Live Labs Follow me on Twitter: http://www.twitter.com/billcrow | 
XBox Live Gamertag:
|
This blog is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
HD Photo plug-ins for Adobe® Photoshop® CS2 and CS3 have been officially released for both Windows and OSX. They're available now from the Microsoft Download Center.
Here are the separate downloads for Windows and OSX.
The Windows version is supported on Windows XP and Windows Vista, and works with Photoshop CS2 and CS3. While not officially supported, it will also work with limited features with older versions of Photoshop and with Photoshop Elements. The included README file has details.
The OSX version works with both 10.4 (Tiger) and 10.5 (Leopard), and supports Photoshop CS2 and CS3.
The features supported on Windows and OSX are identical. The files created with either version are fully compatible with each other and with Windows Vista, Windows Live Photo Gallery, the HD Photo Device Porting Kit, and several other applications that support HD Photo.
I previously posted information about the features provided in the plug-in and an overview of the encoder parameters and what they do. Take a look at previous blog entries here and here. I also gave a presentation at WinHEC 2007 about HD Photo Best Practices. While targeted primarily for hardware developers, the presentation contains a lot of information that may be useful for Photoshop users that want to best optimize their encoder parameters. You can find a copy of the presentation here: WinHEC 2007: HD Photo Implementation Guidelines.
If you've been using the beta version released last summer, you won't find anything new here (other than the removal of the expiration date and the absence of BETA in the name.) All the changes we've made since the last beta are under the covers, fixing a couple minor bugs, addressing several theoretical security vulnerabilities and generally bringing the code up to current Microsoft standards for released software. But on the surface, the released plug-ins look and act identical to the most recent betas. Don't forget that the BETA versions will expire on December 31st, so you should definitely download and install these new released versions.
Once again, I'd like to acknowledge and thank Pegasus Imaging Systems for their participation in the development of these plug-ins.
This blog is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
HD Photo supports high dynamic range wide gamut color (HDR/WG) image content using the scRGB color profile with multiple fixed point or floating point numerical formats in 16 or 32 bits per channel (bpc). HDR/WG introduces a fundamentally different way to manage image information compared to the 8bpc unsigned integer RGB pixel format that is common to JPEG and many other formats. This article will take a closer look at color management in general and the specific issues associated with scRGB and HDR/WG images.
What is Red?
This sounds like a simple question, but it's actually very hard to answer accurately within today's digital photography ecosystem. In most cases, we use a standardized definition (typically the sRGB color profile) and rely on all the components of the ecosystem to understand this standard and act accordingly. By doing so, we accept some significant limitations, because sRGB was designed to be a "least common denominator, specifying a subset of the total visible color spectrum that could reasonably be managed by all components in an ecosystem. The diagram shows the sRGB color space in reference to the entire visible color spectrum. More specifically, sRGB was defined to specify colors based on how CRT monitors work, maximizing the probability that images distributed across multiple systems would still appear with the correct colors without a lot of special software or circuitry to adjust colors for proper appearance on standard computer displays.
sRGB is an "output referred" color space, designed to match the color performance and gamma of a CRT monitor under typical home or office viewing conditions. Since it's impractical to expect the vast majority of users to measure the actual color profile of their computer monitor, sRGB provided a normalized reference standard. Monitor makers build monitors that will ideally match the sRGB standard, and image devices and applications encode images based on the sRGB color profile to help insure that colors will appear correctly.
So, for the vast majority of computer users, the sRGB color profile defines "what is red?" along with the other primary colors (blue and green) and the color temperature of white (6500K). This is the color space that is the standard in all JPEG files, and the default color space for most other image file formats. Because the sRGB color space is based on the capabilities of an output device (a computer screen), it's an output referred color space.
(sRGB gamut diagram courtesy of Wikipedia.)
SIDE NOTE: What's a little ironic is that today, most computer displays are LCD's, not CRT monitors. LCD's have a very different color characteristic and gamma than CRT's. However, every LCD display includes circuitry to accept an input signal (either digital or analog) as if it was a CRT monitor, and convert that sRGB content to the monitors own specific output referred color space. It's a huge disadvantage to use sRGB as the intermediate color space, but since this is still the universal standard, LCD display manufacturers have little choice.
A Camera Sensor's View of Red
When a typical digital camera captures an image, some of the light that falls on the sensor goes through a red filter and hits a photo sensor cell designed to measure the luminance of this red wavelength light. (There are separate color filters for blue and green.) So, every camera sensor has it's own definition of red based on the specific characteristics of the sensor and color filter.
(Bayer pattern sensor diagram courtesy of Wikipedia.)
SIDE NOTE: The X3 sensor technology from Foveon handles this in a very different way, but we'll save that discussion for another day. Though the capture method is different, the fundamental color space principals are the same.
The camera's definition of red is based on it's own specific color profile, and is referred to as an "input referred" (or sometimes "camera referred") color space. The numerical value that represents red in this color space is based on how the camera sensor works, not on how we ultimately want to view the color red. In fact, the camera's "input referred" color space is independent of specific lighting conditions; these conditions must be taken into account as part of the processing of the input referred image information to accurately reproduce colors.
Imagine that you take multiple pictures of the same shiny red apple under different lighting conditions -- florescent, incandescent, etc...
Ok, let's not image it; let's try it out. I rummaged around in my refrigerator and found an aging apple. So, while it's not the most photogenic piece of fruit, it will serve to demonstrate what we're talking about. Here's an example of two different versions of "red" based on photos taken under different lighting conditions:
Because of these different light sources, the camera will measure different values for the red color of the apple. However, as far as we're concerned, the apple is always the same red color, regardless of the lighting conditions. So, the input referred color information from the camera sensor must be converted based on the lighting conditions to the output referred color space that will be used to view the different photos of the apple on a computer display.
Fortunately, cameras do this for us; we typically refer to this process as white balance. In most cases, it's automatic; the camera finds something in the scene it believes to be white and adjusts the various color channels accordingly. Or the photographer may choose to manually select the white balance setting, either based on a pre-defined reference value or by manually measuring a white reference. Regardless of the process, this is one of the essential steps to convert images from the input referred color space of the camera sensor to the output referred sRGB color space that is the standard for JPEG files.
Many digital cameras (including virtually all DSLR's and many advanced point-and-shoot cameras) provide an option to save images in a RAW file format. This RAW format contains the actual measured sensor data in the format as captured by the sensor, and is therefore an input referred representation of the image content. Since the definition of colors in a camera's input referred color space is based on the specific characteristics of the camera's sensor system, RAW files are always unique to a specific make and model of camera. By capturing and preserving the RAW sensor data, a photographer can prevent the camera from discarding information from the camera's input referred color space when it is converted to the output referred sRGB color space of a JPEG file. The photographer is reserving the right to make their own decisions on what information to discard by manually processing the RAW image file to produce the desired output referred image to view or print.
Something Between Input and Output
With everything we've discussed so far, we still haven't done a very good job of answering the (presumably) simple question we started with: what is red?
The typical digital camera shooting JPEG photos relies on two different definitions. The camera's input referred color space provides a definition of red based on the characteristics of the camera's sensor. The camera uses white balance information to convert this to sRGB, an output referred color space based on the typical characteristics of a CRT monitor.
Using an input referred color space will result in different color values to represent the red apple based on the device and the lighting. Using an output referred color space will result in different color values to represent the red apple based on the specific output device (for example, a display vs. a printer.) But what we really want is a numerical value that is specific to the color of the red apple, regardless of the input or output conditions!
The solution lies in the use of an intermediate color space that is not defined based on specific input or output devices. A "scene referred" color space defines colors in a meaningful way to us, based on standard reference of the actual colors contained in an image. A scene referred color space typically has a very wide color gamut, since it's purpose is to accurately describe the full palette of colors that could appear within the scene, regardless of what device was used to capture the image, or how the image will ultimately be rendered. A scene referred color space provides the best and most accurate definition of "what is red?".
In most cases (unless we take explicit steps to do otherwise with tools that enable this capability) we simply use the output referred sRGB color space as this intermediate color space for manipulating the image. While this makes things very simple (avoiding the need to then convert to an output referred color space for display to the screen or sharing the photo), it forces us to accept the limited color gamut of sRGB.
Adobe® Photoshop® refers to this intermediate scene-referred color space as the "working space". By default, most Photoshop users have the working space set to sRGB, again for simplicity. However, Photoshop offers several different color profiles specifically designed to be a better working space than sRGB.
"Adobe RGB" is a working space that provides a slightly wider gamut than sRGB, with more color spectrum that is appropriate for printers. (Remember, sRGB is defined based on the characteristics of display monitors.) But Adobe RGB still limits the color gamut so it can be effectively used with 8bpc images. If the gamut is defined too large, then with 8bpc values, the steps between each of the 256 unique color values become apparent, creating unwanted image artifacts.
By using 16bpc unsigned integers, a larger color gamut for the scene referred color space is possible. Adobe Photoshop provides the "Adobe Pro Photo RGB" color profile specifically for this purpose. It provides a much better working space for editing digital photos, but requires the user to manually convert to sRGB to create a final JPEG output file.
The latest trend for serious digital photographers are RAW workflow image processing applications, including Adobe® Lightroom® or Apple® Aperture™. With these applications, the photographer works directly with the RAW file until the final image is rendered for display or printing. In actuality, it's impossible to work directly with the RAW file for image processing because its not even an image yet. It must first be de-mosaiced from the source Bayer pattern format and transformed from the unique input referred color space for the particular camera. Otherwise, the same color editing operation would produce different results based on the camera used. These applications have their own internal wide gamut scene referred intermediate working color space where all image processing operations are performed. The process of converting from the RAW image to this intermediate scene referred color space is performed automatically every time a RAW file is accessed, with no need for the user to be involved, or even know that it's happening.
Go Ahead, Let Yourself Float
With Windows Vista (and HD Photo), Microsoft has introduced comprehensive support for the scRGB scene referred color profile. Unlike other color profiles that rely on unsigned integers to represent the color values, scRGB uses floating point numerical values. (scRGB can also used fixed point representation, which is simply a method of using signed integer numbers to represent fractional values.) The idea behind scRGB is not to redefine the color space, but to simply remove it's boundary limitations.
A color space is defined by the specific color values of it's primary colors (red, green and blue for the color spaces we're discussing) and the color temperature that defines white. As we've discussed, the vast majority of digital images use the sRGB color space as a universal reference that maximizes color compatibility across applications, platforms and devices, but in doing so, defines rather harsh limits.
scRGB uses the same color primaries and white point as sRGB, but uses fixed point or floating point numbers to allow color values that are beyond the gamut limits defined by these primaries. This provides a color space that is fully compatible with the well-established sRGB standard, but removes restrictions imposed by using unsigned integer numerical values. Conversion between sRGB and scRGB is simply a matter of changing numerical formats; no complex color value transformations are required. scRGB offers all the ease and simplicity of sRGB, but removes it's limitations. This makes it the ideal intermediate scene referred working space for digital photography. A (virtually) unlimited color gamut and dynamic range is available to preserve all image content, but the same file can be treated as if it's a standard sRGB image for display and sharing using a simple numerical conversion to unsigned integer values.
Don't Forget the Gamma
Another aspect of the definition of a color profile is the gamma. This specifies the linearity (or more specifically, the non-linearity) of the transform from input luminance levels to output numerical values. it's an important component when using only 8bpc to represent color values, but it also represents another compromise based on technical limitations.
Let's suppose that you take a photo of a white object and carefully set you exposure so the resulting 8bpc sRGB image file has a value of 255 in all channels to represent this white color. Then you changed your lighting (or your camera exposure controls) to take another photo that was 1/2 the brightness of the original white reference photo. You might expect that the resulting 8bpc sRGB image file would show a value of around 128 in all channels for this 1/2 brightness gray image. You would be wrong.
Most color profiles employ a non-linear gamma curve to the luminance levels of all channels to redistribute the 256 unique values to produce better visual results. In the case of sRGB, this also mimics how a CRT monitor works.
For sRGB, the 1/2 brightness gray image will result in a numerical value of around 186, not the midpoint value of 128. This is based on an sRGB gamma of approximately 2.2. (The actual gamma calculation for the sRGB color profile is a little more complex, but we can use a value of 2.2 for an approximation for our purposes.) This means that the sRGB profile uses 186 steps to represent the lower half of the luminance spectrum, and only 69 steps (255-186) to represent the upper half of the luminance spectrum. sRGB defines a non-linear luminance curve to provide more detailed information in the darker or shadow areas at the expense of the brighter or highlight areas. Since we're far more likely to see visual differences between each of the 255 total luminance steps, this non-linear representations significantly reduces the chance of seeing those artifacts.
While this is great for efficiently using only 8bpc to represent color values, it distorts the numerical representation of color and impacts the proper calculation of color values for image processing. For example, if an application tries to combine two 1/2 luminance images to create a full luminance image, adding the two sRGB values together will result in a total that is much greater than the correct full luminance value. Likewise, simply dividing an sRGB luminance value by two will produce a result that is much darker than the correct 1/2 luminance image level.
Many imaging applications simply ignore this problem and provide the appropriate user interface controls to allow the user to visually manipulate the image independent of the numerical values. (Move a slider until you see the brightness level you want.) In other cases, applications "do the right thing" by first converting non-linear color profile content to linear values (using higher precision numerical formats) to perform image processing mathematics, then convert the result back to the desired non-linear color profile.
The scRGB color profile is designed to always be used with higher precision (typically 16bpc or 32bpc) fixed or floating point numerical formats, and therefore it's not necessary to use a gamma correction to compensate for the limitations of 8bpc formats. The scRGB color profile has a gamma of 1.0, meaning that there is a linear relationship between scene illumination the encoded numerical value. Going back to our example, The numerical value for the gray image that was 1/2 the exposure level of the reference white image would result in a numerical scRGB color space value that was 1/2 the value for the reference white level.
Camera sensors are linear response devices. LCD displays are linear response devices. The correct method for performing image processing mathematics is to use a linear representation. It makes far more sense to store image information using a linear luminance values rather than distorting them with a gamma curve designed to compensate for the limitations of a particular numerical format or device. By not restricting ourselves to the limitations of an 8bpc format, we can dramatically simplify and improve end-to-end photo manipulation and processing.
Big Changes Takes Time
The scRGB color space and HDR/WG image formats will dramatically change the way most digital imaging is performed. The vast majority of imaging software, devices and services are currently based on the use of unsigned integer, gamma adjusted, sRGB image formats. It's going to take some time for all these components in the digital photography ecosystem to migrate to this improved methodology.
Adobe Photoshop CS3 (and CS2) are the current leaders in this space. These newer versions of Photoshop support 32bpc mode, using floating point values and what Adobe calls the "Linear RGB" color profile. This is the equivalent of scRGB, and using Photoshop CS3 (or CS2), it's possible to create, edit and save HDR/WG scRGB compatible images. With the HD Photo plug-in for Adobe Photoshop, you can use HD Photo to efficiently store these HDR/WG images in a variety of fixed or floating point numerical formats.
But we still have a long way to go. There are still lots of issues and limitations in using HDR/WG image formats.
-
Photoshop CS3 only offers a subset of image adjustments and filters when using 32-bit mode. Many Photoshop functions have not been enhanced to handle HDR/WG content.
-
Photoshop CS3 doesn't allow color profile conversion in 32-bit mode, making it very problematic to move between other scene referred color profiles (such as Pro Photo RGB) and scRGB.
-
There are no RAW conversion applications or utilities (including the Adobe Camera RAW module in Photoshop CS3) that allow input referred RAW files to be converted directly to a scene referred scRGB (or Linear RGB) color space.
-
Adobe Photoshop allows 32bpc floating point content to be stored in a TIFF file, but few other applications can understand floating point TIFF files. The Macintosh OS X operating system has support for HDR/WG color spaces as part of Core Image, and even provides support for 32bpc floating point TIFF files. However Core Image interprets these files differently; it assumes the use of the sRGB color profile and that the floating point HDR/WG image data is gamma adjusted. Photoshop stores these TIFF files in an scRGB compatible linear gamma format. The result is that the image looks dramatically different when opened in Photoshop vs. the Mac OS X viewer.
-
Most other photo and imaging applications have no support at all for HDR/WG pixel formats and expect all images are stored using unsigned integer values. Many applications further assume that the precision is always limited to 8bpc and that the color space is always sRGB. While there are applications adding support for HD Photo, in many cases, they only support limited pixel formats and have no support for HDR/WG content or image processing capabilities.
That said, we have to start somewhere and I'm very pleased to see the progress that's been made so far. There are enough pieces in place that you can work with these formats today, albeit with some caveats and limitations. I'm really excited about the progress being made to create the new JPEG XR standard, based on HD Photo. When this standard is finalized it will go a long way to "legitimize" HDR/WG image formats and the associated application support, and will provide the needed reference to motivate the development of compatible software, devices, services and platforms that take full advantage of the powerful capabilities provided by HDR/WG image formats and the scRGB color space.
HD Photo and the new imaging infrastructure in Windows Vista represent the leading edge of this next generation of digital photography. In the near future, it will be a whole lot easier to more accurately answer the simple question "what is red?". It's going to be very cool to watch all of this develop in the months ahead. Get ready for some exciting changes!
We've just posted updated versions of the HD Photo Plug-ins for Adobe Photoshop Software to the Microsoft Download Center. There are now versions for both Windows and Macintosh OS X.
There is an updated beta for Windows Vista and Windows XP, plus a new beta for both Intel and PPC Macs running OS X.
In addition to the direct links above, you can always find the various HD Photo downloads by going to the Microsoft Download Center at http://www.microsoft.com/downloads and searching for "HD Photo".
What's New?
Based on a lot of really good customer feedback, we've redesigned the encoder options dialog box, separating the basic and advanced controls in two separate modes. This is more consistent with similar dialogs in Photoshop, allows the dialog to be smaller, and makes it a less confusing. We also did some other re-arranging and general clean-up of the dialog user interface, with better consistency among related controls. The pixel format control will now only offer the pixel formats that are compatible with the current image mode in Photoshop. You can save to any of the supported HD Photo pixel formats, but you may first have to convert to the correct image mode from within Photoshop. The encoder options dialog also remembers it's state, so it will re-open in the same mode and with the same settings as the last time it was opened within the session.
The (default) basic mode of the encoder dialog is simple and straightforward. The only controls are the quality setting and an explicit option for lossless mode. Setting the quality setting to 1.0 is the same selecting the Lossless option.
Clicking the "Advanced Mode" button switches the dialog to a different set of encoder controls. Rather than the more simplistic Quality setting in Basic mode, the Advanced mode provides full control over all the detailed encoder settings. These settings are fully documented in the included ReadMe file, and we've talked about them in a previous post on this blog.
We also implemented a major architectural overhaul of the underlying codec and memory management. These changes significantly reduce the memory requirement to encode and decode an image, improving performance and allowing support for much larger images without encountering memory limits. This was a fairly major change, and the main reason why it's taken us longer than we planned to ship this beta.
The tiling options in the previous beta did not really operate correctly and that's been fixed. We fixed a bug when saving to an HD Photo file from a RAW file that was opened in Photoshop from Lightroom. There were a handful of other more obscure bugs that have also been fixed.
The Setup program has also been completely overhauled. It will now automatically locate the installation of Photoshop CS2 or CS3 and install the plug-in directly to the appropriate directory. If you have both versions, it will be installed in both places. It also creates an entry under the Programs menu to provide easier access to the ReadMe documentation file and the End User License Agreement (EULA.)
Did You say Mac?
Yes! As promised, we're now releasing a version of the HD Photo plug-in for Photoshop running on OS X. This supports both Photoshop CS2 and CS3 running on OS X 10.4 (Tiger.) It supports either version of Photoshop on both Intel and PPC Mac systems.
The Mac version is functionally identical to the Windows version. It supports all the same pixel formats, encoder options and file format features as the Windows version.
The encoder options dialog is functionally identical to the Windows versions, with all the same options and controls.
One very cool feature with the Mac version is support for thumbnails in the OS X finder. OS X doesn't currently HD Photo as a native format. However, whenever Photoshop writes and HD Photo file, it also creates a thumbnail on the resource fork in a compatible format for OS X. The result is great looking thumbnails for HD Photo files in Finder. Very cool!
The Mac installation package is a disk image containing the documentation and separate installers for both Photoshop CS2 and CS3. Run the appropriate installer for your version. It will only install if it finds the compatible version of Adobe Photoshop.
Still a Beta?
Yes, these updated releases are still officially in BETA mode. We believe that these are the final versions, but because we made some significant changes and this is the first public release of the Macintosh version, we elected to keep them in BETA mode for this release. We're continuing to do additional testing, and as always, we welcome any comments, issues, bug reports or other feedback. We're also very interested to hear if you run into any problems installing the plug-in on either Windows or Mac OS X. You can post your feedback via this blog, or send it to the "hdphoto" email name at microsoft.com.
We're very excited to be able to release these updated plug-ins, especially with the addition of support for Mac OS X. Have fun and let us hear your comments.
This is a pretty exciting day for all of us working on HD Photo. Today, the Joint Photographic Expert's Group (JPEG) announced a new work item for the standardization of a HD Photo as a new file format called JPEG XR (XR is short for "extended range".) You can read the full details in the Microsoft press release here and the JPEG press release here. (Go ahead and check it out; we'll wait here.)
Wow! Our group at Microsoft's Core Media Processing Team has been working on HD Photo for over five years. The underlying compression technology is based on work from Microsoft Research that goes back even farther than that. Since our first public disclosures about "Windows Media Photo" (the original name of HD Photo), our goal has been to develop the ultimate successor to JPEG as the format of choice for all digital photography. We also announced earlier this year that we were committed to standardizing HD Photo as an open format. With today's announcements, we're moving a lot closer to both of those goals.
An Illustrious History
For as long as digital photography has existed, digital photos have been defined by the technology created by the JPEG Committee, and it has become the most widely used standard in the world. Simply put, digital photography would have never been possible without the work of Joint Photographic Experts Group and their JPEG image compression technology.
Over the years, the JPEG format has become ubiquitous. Every application, device or service that is in any way related to digital photography recognizes and/or produces this format. JPEG is digital photography.
But for all the value the original JPEG format delivers, it's starting to show its age. Limitations of the JPEG file format are hindering the ability to innovate new features and quality improvements in digital photography. There have been multiple approaches to work around these limitations, but we need significant improvements to the industry standard photo format to enable new opportunities for innovation and market growth.
JPEG 2000 introduced several excellent innovations, but it largely addresses requirements beyond the role of the original JPEG format. So, while JPEG 2000 is making some valuable contributions in other areas of digital imaging, this advanced format doesn't directly address the key requirements for consumer and professional digital photography.
A New Format for Digital Photography
From the beginning, Microsoft designed HD Photo to meet the specific needs of end-to-end consumer and professional digital photography, and address the current limitations of the original JPEG format. HD Photo offers new pixel formats to retain higher image quality, great new progressive decoding features for more powerful image access and manipulation, and significant compression improvements to improve quality, reduce file sizes and enable better performance.
The use of the original JPEG format for digital photography has been limited to 24-bit images; each pixel is made up of three colors (red, green and blue), each stored as an 8-bit value. This allows 256 different values for each color, or 16.8 million possible colors. While that sounds like a lot, it significantly limits the ability to create the highest quality images. Camera sensors typically capture up to 68.7 billion colors, so when saving as a JPEG file, a lot of information has to go. Additionally, during any image editing operation, a much greater range of colors is required to preserve information that would otherwise exceed the finite limits of the color range. Once data is lost beyond these limits, it can't be recovered in the future.
HD Photo offers a wide range of pixel formats at 8, 16 or 32 bits per channel including high dynamic range wide gamut formats using fixed point or floating point representations. HD Photo offers options that eliminate the quality limitations of the original JPEG format's lone 24-bit pixel values. This previous blog post shows just one example of the capabilities enabled by HD Photo's new pixel formats.
HD Photo's progressive decoding architecture makes it possible to efficiently decode a the image at a lower resolution, reduced quality, or a selected region by processing only a portion of the entire compressed file. Progressive decoding enables a whole bunch of very cool features. The Photosynth Technology Preview from Microsoft Live Labs is an excellent example of the new class of applications powered by progressive decoding. Photosynth is discussed in some detail in a previous blog post here. HD Photo also helps power Virtual Earth 3D and Microsoft Lab's HD View Giga-pixel Image Viewer.
HD Photo delivers state of the art image compression, offering up to twice the compression efficiency of original JPEG format (providing the same quality in half the file size) as well as higher quality lossy compression that scales all the way to mathematically lossless compression. This makes it possible to use higher bit depth pixel formats for better image quality while retaining similar file sizes. Better compression also allows for much smaller files using traditional 8-bit pixel formats. Smaller files can be saved and transferred faster, providing longer device battery life and an improved user experience. HD Photo delivers this improved compression using a high performance algorithm that is equally at home on low cost, low power devices or high end multi-core PC's.
An Open Standard
The path towards standardizing HD Photo began earlier this year, and now becomes public with today's announcement from the JPEG Committee. This is one more step in an ongoing process towards our stated objectives for standardization. Our goal is that HD Photo becomes JPEG XR, an open standard under the complete control of the JPEG Committee, with royalty-free rights to all required patents granted by Microsoft. Microsoft is actively participating with the JPEG Committee to produce a complete and detailed technical specification enabling anyone to create implementations in any programming language, on any platform and under a wide variety of business models. We're excited to be contributing to the next great standard for digital photography, enabling a whole new level of feature and technology innovation, improving photo quality and creating exciting new applications and services. It's our sincere hope that that today's announcement will help accelerate the process of making HD Photo (JPEG XR) as ubiquitous as JPEG is today.
Bill Crow
HD Photo Program Manager
HD Photo supports a new paradigm for image editing. , Higher fidelity images can be stored in a high dynamic range, wide gamut format using either fixed or floating point numerical encoding. HD Photo retains image content that would otherwise fall outside the visible range and be clipped using the more typical unsigned integer numerical representation (TIFF, JPEG, PNG, and most other formats.) This may happen when the camera converts from RAW, or during any other editing or conversion operation. Take a look at a previous post on this blog for more information about high dynamic range wide gamut pixel formats.
This is a Big Change!
Most imaging applications aren't designed to understand high dynamic range, wide gamut pixel formats. There are some specialized tools, used primarily by the visual effects world, that work with high dynamic range formats. Adobe Photoshop CS3 (and CS2) supports high dynamic range wide gamut editing using the 32-bit editing mode. You probably never realized that Windows Vista Photo Gallery - the simple, easy-to-use photo tools built into Windows Vista - also supports high dynamic range wide gamut editing. That's right, Photo Gallery.
Let's walk through an example, using the Beta of the new Windows Live Photo Gallery. This new update for Windows Vista Photo Gallery is coming soon, with some really cool new features (Big Hint: It now has "Live" in it's name), and it will also be available (with HD Photo support!) on Windows XP. You can check out this post on the Microsoft PhotoBlog for more details on all the cool new features.
This example works the same way using Windows Vista Photo Gallery, available today in all versions of Windows Vista. (I've included links below to the source photos so you can try this out for yourself.) I chose the new Windows Live Photo Gallery because they've added a really cool exposure histogram control, which will help us better illustrate what's going on.
Doing it Old School
Here's an 8-bit sRGB JPEG file, opened in Windows Live Photo Gallery. This photo definitely has some problems.
It appears that either the camera or some automatic image adjustment program made an overall exposure adjustment decision based on an average scene illumination level. The painted wall on the left, the dark background on the right and Anthony's dark clothes all biased this average exposure setting, causing his face and the balloons to be over exposed. Let's zoom in on that area and select the Exposure adjustment controls under Photo Gallery's "Fix" tools.
The histogram clearly shows the over exposure problem. There is a large percentage of image content that is clipped at the upper limit of the exposure range.
We can try to fix the photo using the Brightness control to reduce the effect of the over exposure.
While this reduces the blinding brilliance of the image, we weren't able to recover any of the color or detail in Anthony's face, or in the balloons. That topmost yellow balloon is still clipped to the limit in the red and green channels, resulting in a flat, featureless yellow surface for the entire balloon. Again, the histogram clearly shows what's going on here. Any image data that may have existed in this highlight area was clipped to the upper limit of the numerical range when it was encoded in an 8-bit JPEG file. The Brightness adjustment reduced the levels, but all channels are still clipped at a fixed limit. All this adjustment did was reduce the level of that clipped limit. We haven't done that much to rescue the details and color in this over exposed photo.
This problem isn't unique to JPEG. We'd have the exact same results with any image format that uses unsigned integers (0-255 for 8-bit or 0-65535 for 16-bit) to encode the data, including TIFF, PNG, or even HD Photo if using an unsigned integer mode.
Once More, with HD Photo
Now let's load the same photo as stored in a 16-bit fixed point scRGB HD Photo file. As you can see below, the displayed visible range of the image is the same as the JPEG image, so the initial appearance looks identical. We can also see that even though this is a 16-bit image, the file size is about the same as the equivalent 8-bit JPEG image.
We're starting with the same apparent over exposure problem. Let's go through the same editing process and see how things differ when using a high dynamic range, wide gamut pixel format. First, we'll zoom in on the same over exposed area and open the Exposure tools under the "Fix" menu.
Note that the histogram for this version of the photo looks a bit different. There appears to be a lower percentage of image content near the upper limit of the displayed exposure range, and there doesn't appear to be that same "pile" of image content pressed up against that upper histogram limit. that's because there's a bunch of content in this image that's not currently displayed in this histogram. The histogram only shows the visual range, but the HD Photo file includes image content that is outside of the current visual range. There's more data that was captured by the sensor, but when the image exposure was adjusted based on the overall average level, this additional data wasn't clipped within the 16-bit fixed point scRGB HD Photo file, even though it appears to be clipped in the current display view. Let's perform the same Brightness adjustment and see how things are very different when editing high dynamic range, wide gamut images. We'll actually make a larger adjustment change because we're working with a larger range of exposure values.
Wow! That's a big difference! The histogram clearly shows that there was a bunch of image content that we weren't seeing in the original version, but in a high dynamic range wide gamut format, it wasn't clipped to the upper limit of the visible exposure range. When we adjusted the Brightness, this content came back into the visual range; there is no hard clipping visible at the upper end of the exposure range. The visible difference in the image is dramatic. We were able to recover a great deal of color and detail in Anthony's face and the over exposed balloons. Now we can see that the upper yellow balloon is actually translucent; all that detail was completely lost when the initial exposure decisions caused the highlights to clip at the upper end of the visual exposure range. Now that we've undone most of the over-exposure damage, we could continue on and tweak the contrast, shadows and highlights as desired.
Comparing the Results
Here's a comparison of the two different results based on only the exposure adjustments from above. You can really see the differences in Anthony's face and shirt, as well as the yellow balloons. We can retain most of the desirable scene brightness while still recovering the lost highlight color and details.
This is just one example of the improved editing capabilities that are possible using high dynamic range wide gamut image formats (and editing applications that support these formats.) This capability is here today for HD Photo using Windows Vista Photo Gallery, with expanded features and support for Windows XP coming later this summer in Windows Live Photo Gallery.
High Dynamic Range Wide Gamut editing is also supported in Adobe Photoshop CS3 (and CS2.) You can download the free plug-in to add HD Photo support to Adobe Photoshop on Windows Vista and Windows XP, and version for OS X (both PPC and Intel) is coming very soon. Hopefully we'll see more imaging applications add support for high dynamic range wide gamut formats in the near future. The new Windows graphics infrastructure, combined with HD Photo, can enable a whole new world of image processing innovation.
Your Turn
Here are links to the actual JPEG and HD Photo files I used for this example. (They're low resolution images so they can download faster.) Simply right-click and "Save As..."
AnthonyS.jpg AnthonyS.wdp
Open these in the Windows Vista Photo Gallery editor and try this for yourself. Better yet, ask you buddy to try to fix the JPEG version, then you show how it's done using the HD Photo version. Have fun!
I'll be attending WinHEC again this year, with multiple sessions on HD Photo. WinHEC 2007 takes place at the Los Angeles Convention Center May 15th-17th. I'll be presenting a session entitled "HD Photo Implemention Guidelines" on Wednesday at 11A. I'm also hosing a "chalk talk" on "HD Photo Wide Gamut Imaging" Wednesday afternoon. The chalk talks are a new feature of WinHEC this year, designed to be much more of an interactive discussion rather than a typical Powerpoint presentation. It should be a lot of fun. The agenda describes both of these sessions for device driver developers and hardware engineers, but if you're involved in any aspect of digital imaging software or hardware, I'm sure you'll find these sessions interesting and informative.
In addition to these two sessions, we'll also be showing some HD Photo demonstrations as part of the Microsoft Pavilion in the conference exhibit area. I'll be hosting those demos so be sure to stop by and say hello.
I'll also be presenting sessions on HD Photo and hosting demonstrations at WinHEC 2007 Taiwan June 13th-14th, and then continuing on to WinHEC 2007 Japan June 18th-19th. I'm looking forward to meeting you there and sharing in an interactive discussion about HD Photo.
My presentations at WinHEC last years was the first signficant public discussion of HD Photo (then known as Windows Media Photo) and we generated a tremendous amount of attention and interest. Since then we've shipped the 1.0 version of the HD Photo Device Porting Kit, Windows Vista, .NET Frameworks 3.0, Windows Imaging Component, and the beta of the Windows HD Photo Plug-in for Adobe® Photoshop® software, all with support for HD Photo. I'm looking forward to some equally interesting discussions this year.
See you there!
The Beta 2 version of Microsoft® Expression® Design was recently released with full support for opening and saving HD Photo files. It's available for either Windows Vista or Windows XP.
So what exactly is Expression Design? In the words of the product team:
Expression Design is a professional illustration and graphic design tool that lets you build compelling elements for both Web and desktop application user interfaces.
Unleash your creativity with innovative vector drawing tools and non-destructive effects that you can apply to either vector or imported bitmapped images. Apply both hard-edged and organic strokes while maintaining the flexibility of vector paths. Expression Design has been built from the ground up for designers by designers.
Take a look at the Expression web site for lots more details on Expression Design and the other applications in the new Expression Suite family.
Expression Design offers HD Photo (actually called Windows Media Photo, as it is referred to throughout Windows) via the Open..., Import..., Save, Save As..., Export... and Export as Image File... commands. Here's the encoder options dialog that Expression Design displays when exporting or saving an HD Photo file:
Expression Design works exclusively with 8bpc RGB images, with our without alpha channels. HD Photo files of any pixel format may be opened, and if required they will be converted to Design's 8bpc RGB working format. On save or export, files may be saved in either 24bpp RGB or 32bpp RGBA modes.
Expression Design is a powerful new application that can handle a wide range of image creation and editing tasks, and integrates with the other applications in Microsoft Expression Suite. It's another great tool for working with HD Photo files. Check out the free Beta 2 download and 180-day evaluation license.
Have fun!
Bill Crow
A Mode by Any other Name
So, as reported previously, we now have released a beta of the Windows HD Photo Plug-in for Adobe® Photoshop® software. This plug-in allows you to open and save HD Photo files from Photoshop. Photoshop is a powerful application with an incredible array of features. Among those are the ability to support a wide range of image modes, or as we refer to them in HD Photo - pixel formats. The HD Photo plug-in handles the majority of Photoshop's modes and HD Photo's pixel formats with ease, automatically creating the appropriate mapping, and allowing you to full control to choose the appropriate mode/format for your specific needs.
Mapping Pixel Formats to Modes
When you open an HD Photo file in Photoshop, the choice of the Photoshop mode associated with the HD Photo file is completely automatic. The plug-in selects the appropriate mode to retain the full fidelity of the HD Photo file's pixel format.
Here's how HD Photo pixel formats translate to Photoshop modes on Open.
Most of these mappings should be pretty straight forward. 8-bit to 8-bit, 16-bit unsigned integer to 16-bit. HD Photo supports multiple high dynamic range, wide gamut formats in different bit depths and numerical interpretations. All of these map to Photoshop's 32-bit floating point mode.
There are a few HD Photo pixel formats that the plug-in doesn't support. HD Photo offers pre-multiplied alpha channels, but there is no such equivalent in Photoshop. This format is used for very specific rendering and compositing scenarios, so its not very likely you would ever need to open a file in this pixel format in Photoshop. HD Photo's various n-Channel modes are also not supported via the plug-in. Even though Photoshop can support multiple channels, the default interpretation is different enough that it simply doesn't make sense to try to force-fit files in these pixel formats. Again, the use of n-Channel pixel formats is fairly specialized and not useful for images that are typically edited manually.
But everything else is supported - RGB, CMYK, Grayscale, black/white, alpha channels - the HD Photo plug-in handles all these various pixel formats, mapping them to the appropriate Photoshop modes.
Mapping Modes to Pixel Formats
When you save an image from Photoshop to and HD Photo file, you're typically provided a choice of different pixel formats. The pixel formats you're offered depends on the current Photoshop mode. It's basically the reverse mapping of the same table above.
HD Photo offers numerous pixel formats and it's important to understand when its appropriate to use the various choices. For example, there are several different ways to store high dynamic range, wide gamut content, each with its own specific advantages and disadvantages. Photoshop uses 32-bit floating point for HDR images, so you can always store in this pixel format to guarantee you're preserving absolutely all the content. However, there are alternate pixel formats that preserve sufficient dynamic range and precision for most photographic applications, but use a more compact numerical format. For example, unless storage space is of absolutely no concern, you can save HDR photos in 16-bit fixed point, producing a much smaller file. The results will typically be much better than using a 32-bit format with a higher compression ratio. This is a topic we'll dive into in more detail in a future post.
Basic vs. Advanced Encoder Options
When saving an image in HD Photo format, there are a whole bunch of encoder options at your disposal. Lot's of time, you most likely don't want to bother with all those choices. You can select the "Basic" mode and set a single slider to control the balance between image quality and file size. the default setting (on a scale of 0.0 to 1.0) is 0.9 and will result in a high quality image with reasonable compression. The exact amount of compression is completely depends on the content of the image. This control effects the quality of the compressed image, but the actual file size can't really be predicted. You'll also discover different visual results among the different pixel format bit depths. That's primarily because you can apply much more compression to the higher bit depths before you ever see the impact of that compression on you're 8-bit monitor. In general, you should be careful of using lower quality settings with higher bit depths; you're defeating a lot of the reason you chose a higher bit depth to begin with.
For more control of the encoder, you can select the Advanced Mode. the ReadMe file that comes with the plug-in describes these options in some detail. We also covered them here in a previous post about the DPK encoder utility.
Back to the Basic Mode. That scale from 0.0 to 1.0 doesn't map directly to the Quantization setting in the Advanced Options. It's actually controls a couple different encoder options according to the following rules:
- If Quality=1.0, this is the same as lossless mode.
- if Quality >0.4, Overlap mode is set to One Level; otherwise it's set to Two Level.
- If the bit depth is <=8 then Quantization = 105 - (Quality * 100)
- If the bit depth is >8 and <=16 then Quantization = 185 - (Quality * 180)
- If the bit depth is >16 then Quantization = 245 - (Quality * 240)
These rules were chosen to provide a reasonable approximation of expected results based on the Basic Quality settings, but with experimentation you'll discover that different settings are appropriate for different types of images.
If you're really starving for even more details, you can always check out the complete HD Photo Feature Specification.
Color Profiles
Saving files in Photoshop provides the option to embed an ICC color profile in the file and this is fully supported by HD Photo. However, HD Photo has explicityly-defined default color profiles so in many cases you can save space by not saving he color profile. For Unsigned Integer pixel formats (in either 8-bit or 16-bit modes), HD Photo assumes sRGB if no color profile is present. So, whenever you're using sRGB, there's no need to embed the profile. Likewise, whenever using Photoshop's 32-bit mode, HD Photo will always assume the color profile is scRGB. (This is identical to Adobe's Linear RGB color profile.) So again, there is no need to embed a color profile for this mode; it will simply be ignored when the file is decoded.
Options, Options, Options
There are a variety of additional options in Advanced Mode, which all address specific requirements or usage modes. If you're not sure what settings are right for you, then the best choice is to simply stick with the defaults for these options.
That should be more than enough to get you started. Up next time (and I'll do my best not to make you wait too long) we'll talk about the recommended workflow to convert from RAW camera formats, preserving the maximum image quality.
Bill Crow
Didn't we already do that?
Regular readers of my blog (which is a tad oxymoronic, since it appears I have a hard time being a regular poster) already know all about HD Photo. We've been talking about at various conferences, via this blog and on various websites for quite a while. However, during all that time, we never "officially" announced this new still image file format. There have been press reports, quotes in the media, blog posting, independent analysis and speculation, and seemingly endless comments, but never a press release.
So, all that changed this week at the Photo Marketing Association (PMA) conference in Las Vegas, NV. We announced a bunch of stuff that you probably know all about, plus some signficant new information. The full press release is here. We also have a new HD Photo information page at Microsoft's Pro Photo website.
We announced the availability of the HD Photo file format, with support in Windows Vista. We also announced support for Windows XP and Server 2003 via a Windows Imaging Component (WIC) redistributable or via .NET Framework 3.0. But you already know all that.
We announced the Device Porting Kit (DPK), allowing developers to implement HD Photo encoders and decoders for any device or platform. We also announced that HD Photo is licensed 100% royalty free. But again, you already know all that.
So what's new?
I'm very excited that we announced a free set of HD Photo plug-ins for Adobe® Photoshop® software, developed in conjunction with Adobe Systems, Inc. The set of plug-ins includes support for CS2 and CS3 on Windows XP, Windows Vista and Macintosh OS/X. The beta release for Windows is available now; you can download it here. Our goal is to have the beta version for OS/X available in a few weeks, and final versions released by May.
The plug-ins provide support for a wide range of HD Photo features including extensive pixel format support, alpha channels, multiple color formats, and high dynamic range, wide gamut imaging. The full set of encoder parameters is also available including quantization, color sub-sampling, overlap processing, tiling, planar alpha channels and option pixel format conversion. When released shortly, the OS/X version will provide the first support for HD Photo on the Macintosh.
With these plug-ins, it's now possible to easily create a wide range of HD Photo files. Using Photoshop actions, you can easily batch convert to/from the HD Photo format. Over the next several weeks I'll be posting a set of tutorials on using the plug-ins to take full advantage of the powerful capabilities offered by HD Photo. We'll also talk about the great features in Photoshop (especially in the new CS3 Beta release) that are ideal when working with HD Photo.
In addition to working with Adobe, we're also working together with Pegasus Imaging Systems to develop this set of plug-ins. Check out their website for a very extensive set of healthcare, document and photo imaging solutions for developers and end users. Cool stuff!
Did someone stay standards?
I'm also very pleased that we have announced our commitment to submit HD Photo for standardization. There's really no details to talk about yet. We're in the process of evaluating the appropriate standards organization and the process. But we're committed to pursue standardization for HD Photo.
We've worked very hard over the past several months to address the comments and concerns we've heard about achieving broader deployment for HD Photo. We changed the licensing terms to eliminate the previous very small royalties. We changed the name to a generic, un-trademarked term. We've made the DPK available through a simple click-through End User License Agreement.
Now we're taking the next big step. Our goal is to turn the format over to an appropriate standards organization. Ideally, this will include the publishing of an open specification, making possible to implement compatible encoders and decoders that are completely independent of Microsoft's reference source code. This should fully address any concerns that have been raised about the option for open source implementations.
Anyone who has been involved in standardization, especially related to advanced technology, can tell you it is often a complicated and sometimes lengthly process. We're working hard to move forward in a timely manner; we'll share more details as we have them.
Give it a Whirl
By now you may have already heard of and played with Photosynth, a new Technology Preview from Microsoft Live Labs. If you haven't had a chance to try out Photosynth for yourself, trust me, you'll be glad you spent the time to check it out. So, rather than me give a long-winded explanation of what it is and what it does, go experience the Photosynth Technology Preview yourself right now.
Spend some time exploring. Review the "Getting Started" and other useful information, including all the cool keyboard shortcuts. Check out different photo collections. When you're in Gary Fagan's studio, be sure to explore the pictures on the wall. Try out all the buttons in the upper-right corner of the screen. And for goodness sake, be sure to exercise that wheel on your mouse and zoom, zoom, zoom.
When you can finally tear yourself away from all that fun, come on back here and we'll talk a little bit about what's under the covers and how it relates to HD Photo. See you soon! You can find the Photosynth Technology Preview at http://labs.live.com/photosynth.
I Recognize that Pixel!
Welcome back. Pretty cool, right? The Photosynth Technology Preview is made up of multiple technologies, all working together to create a unique experience. While this combination is pretty astounding, each of the individual technologies has the potential to be applied in many different applications.
One of the amazing capabilities in the Photosynth Technology Preview has actually happened before you even start viewing photos. Photosynth accepts a collection of individual photos, all shot in the same spatial location, from arbitrary angles, at arbitrary lens settings, with arbitrary lighting, and quite possibly at very different times. The only requirement is that each photo overlaps one or more of the other photos so that all photos in the collection are connected through some combination of overlaps.
Photosynth analyzes all the photos and finds various points that are common among one or more photos. It's a little beyond the scope of this blog to explain exactly how all that happens. (Translation: I don't have a clue.) But the task of figuring out exactly where each photo is projected in a 3D environment is (for the most part) a completely automatic process. Once a collection is stated, new photos can be added later. The "cloud of points" you see in the background of all the photos is the set of individual points that were identified to exist in one or more photos.
In addition to calculating the placement of the photo, Photosynth also calculates the position of the camera when the photo was taken. The camera button in the upper right turns on and off the orange pyramids that represent these camera locations. In addition to clicking on a white rectangle to display a specific photo, you can also click on these orange pyramids to view the photo taken from that particular vantage point.
Browsing GigaPixels of Photos
So if you spent some time playing with Photosynth, you've certainly discovered that you can smoothly and seamlessly zoom and pan from one photo to the next. If you select the 2D view, you can instantly view every photo in the collection, and quickly zoom in to finest pixel detail on any individual photo in the collection. If you didn't try zooming and panning in the 2D view, go back and try it now. Then stop and think about exactly how many gigapixels worth of digital photos that you're free to interactively explore, all over an Internet connection, and all with virtually no delays or pauses.
One thing you may have noticed is that when you quickly zoom in to a photo, or quickly pan to a neighboring photo when zoomed in, the image first appears soft and out of focus, then quickly comes into sharp focus. As you continue to zoom into the finest detail, this process of adding the sharp detail to a softer out-of-focus preview continues. If you look very closely, you may even notice that this happens in segments of the photo, starting near the center of the screen, then moving to the out edges. If you have a less-than ideal Internet connection, this process will most likely be slowed down enough so you can clearly see the various stages. (But even with a slow, clunky Internet connection, the user interface is always smooth and responsive.)
So as you experiment with zooming and panning around a photo collection in Photosynth (especially in 2D mode), you can begin to see the clues as to what is happening to provide this near-magical experience.
Uncovering the SeaDragon
The technology component in Photosynth that provides this amazingly smooth and continuous exploration of arbitrarily large high resolution photo collections is an incubation project at Microsoft Live Labs, code-name SeaDragon. The name comes from the name of a start-up company that joined the Microsoft family earlier this year, becoming part of Microsoft Live Labs.
The SeaDragon technology makes it possible to smoothly and quickly view any number of web images with a responsive, intuitive user experience. It doesn't matter how large the images are - each image could be gigapixels in size. Nor does it really matter how many images are in the collection. The SeaDragon technology handles it all.
The key to this magic is to only transmit the minimum subset of information that is required for any particular view. As the view changes (either zooming or panning), the minimum information for the new view is transmitted and the SeaDragon user interface (powered by Microsoft DirectX) smoothly crossfades from one view to the next. Because your computer screen is a finite size, there is always a finite limit on the amount of information that is needed at any point in time to fill that screen. It doesn't matter how large the individual images are or how many of them are in the collection; the amount of information required to fill the screen is always the same. (Read that last sentence again; when that sinks in and make sense, you'll understand the key to how this all works.)
But there's a catch. (Isn't there always?) For this to work, not only do you need to figure out exactly what the minimum required information is, you also need to be able to put your hands on just that information as quickly as possible. That's exactly what the SeaDragon technology can do, with a little help from HD Photo. (I told you this would come back home eventually.)
Mipmaps and Regions
Every photo you see in the Photosynth Technology Preview is an HD Photo image. The photos that make up each collection are stored in their full resolution in HD Photo format on the server. The tiling feature of HD Photo has been used to encode the photos so the data is internally organized within the compressed HD Photo file into bite-sized tiles.
The HD Photo codec provides built-in features to decode images at reduced resolutions (often called mipmaps). So when Photosynth only needs a small thumbnail of a high resolution image, The HD Photo codec running on the server can extract the subset of data needed for this low resolution display (a small fraction of the entire file) and using a compressed domain transformation operation, create a new HD Photo file that only contains that low resolution information.
Because this is a compressed domain operation, the server never had to decode or re-encode the compressed data to create this low resolution "thumbnail" of the larger, high resolution image. The only work involved was to copy a portion of the compressed data and wrap it up in a container to make a new HD Photo file. This very small HD Photo file is sent across the network connection, and then decoded by the HD Photo codec on the client to provide the low resolution view required for the particular display.
When zooming in to the fine details of a high resolution image, the HD Photo codec is able to very quickly extract an arbitrary rectangular region by accessing only the image tiles that overlap that region. Like the mipmaps described above, this is accomplished by simply extracting a small portion of the compressed data and building a new (and very small) HD Photo file to be sent across the network. The client receives and decodes this small file, combining it with the other segments required to display the required view.
This combination of mipmaps and regions allows the server to quickly extract the subset of compressed data required for any particular view and transmit it to the client with no need to ever uncompress the information until the client displays it on the screen.
HD Photo provides the key features built into the standard encoder and decoder to enable this powerful client/server solution. The high performance decoder, enhanced mipmap and region decoding capabilities combined with its excellent compression efficiency provides a key component of the technology magic that powers the Photosynth Technology Preview.
Full disclosure: The preceding section describes the design of the SeaDragon technology. The current Photosynth Technology Preview uses HD Photo for all images, but is temporarily making use of some pre-cached mipmaps and regions to stream-line the development process. Moving forward, Photosynth will take full advantage of the features of the HD Photo codec to dynamically extract this information on demand, eliminating the extra overhead of pre-caching.
Beyond Piazza San Marco
The Photosynth Technology Preview is just a glimpse of what is to come. The Photosynth team is hard at work to make it possible for you to create your own collections from your own photos. Then imagine being able to not only explore your own photos, but to combine your photos with a large online community. Photosynth will make it possible to merge photo collections from any number of people, and dramatically expand the opportunities and the excitement of the online photo sharing experience.
Photosynth is just one example of how the SeaDragon technology can revolutionize the way we view information online. Instead of photos, imagine if all those images that you can smoothly and freely explore were pages of a book, magazine, or newspaper, or a collection of web pages, or a series of x-rays or MRI scans, or hundreds of high resolution blue prints. The SeaDragon technology and HD Photo will empower entirely new ways to explore online information, changing the paradigm of how we explore the Internet.
Stay tuned, it's just starting to get fun!
A New Name and a New License
No, it's not another file format. We have re-christened Windows Media Photo as HD Photo. We've listened closely to the feedback from our partners and customers, and in response, we have made some important changes, including this new name.
We have also made a substantial change to the licensing. The previous Windows Media Photo Device Porting Kit (DPK) evaluation and distribution license agreements are gone. We have replaced them with a straight-forward click-to-install End User License Agreement (EULA) for the new HD Photo 1.0 Device Porting Kit. It's now easy for anyone to receive the complete HD Photo DPK, including the compressed bitstream specification and the ANSI-C reference source code. The DPK is free, and we have eliminated all distribution royalties. HD Photo is 100% free for all developers.
We're really excited about these major changes! We've heard the requests and we've responded.
The Latest Downloads
The new HD Photo 1.0 Device Porting Kit (DPK) is now available for download:
http://www.microsoft.com/downloads/details.aspx?FamilyID=285eeffd-d86c-48c3-ab93-3abd5ee7f1ce&displaylang=en
Windows developers don't need the DPK; HD Photo is included in Windows Vista and available for Windows XP or Windows Server 2003 with either .NET Framework 3.0 (for managed code applications) or Windows Imaging Component (WIC) for unmanaged code applications.
The HD Photo Feature Specification 1.0 (also included as part of the DPK) is available separately here:
http://www.microsoft.com/downloads/details.aspx?FamilyID=6fe1ebac-c7b3-4768-90bc-13d330d5ec02&DisplayLang=en
The released version of .NET Framework 3.0 is also available:
http://www.microsoft.com/downloads/details.aspx?FamilyID=10cc340b-f857-4a14-83f5-25634c3bf043&DisplayLang=en
The SDK for .NET Framework 3.0 can be downloaded from here:
http://www.microsoft.com/downloads/details.aspx?FamilyId=C2B1E300-F358-4523-B479-F53D234CDCCF&displaylang=en
The released versions of Windows Imaging Component (WIC) are also ready to go:
x86: http://www.microsoft.com/downloads/details.aspx?FamilyId=8E011506-6307-445B-B950-215DEF45DDD8&displaylang=en
x64: http://www.microsoft.com/downloads/details.aspx?FamilyId=F64654AC-6E26-41D9-A90A-0E7783B864EE&displaylang=en
A File Format By Any Other Name
For the immediate future, we will still use the Windows Media Photo name to refer to the implementation of HD Photo in Microsoft products that are already shipping. This includes the initial release of Windows Vista, the XML Paper Specification (XPS), .NET Framework 3.0, and Windows Imaging Component (WIC).
With the new name, the spec now adds support for a new file extension - .hdp. However, the various Windows implementations won't recognize this new extension until some updates have been released at some point in the future. (No, I don't know exactly when that will be.) So we're recommending implementers stick with the current .wdp extension. You can "future-proof" your application by also adding support for the new .hdp extension.
Share: Digg it | del.icio.us | Reddit
The fine folks at the Graphics and Media Lab at Moscow State University (MSU) have completed a comparison of Windows Media Photo and nine different implementations of JPEG-2000. Their results are right in line with our own analysis; it confirms our message that Windows Media Photo delivers image quality comparable to JPEG-2000 and significantly better than JPEG.
The complete report is available here:
http://www.compression.ru/video/codec_comparison/wmp_codecs_comparison_en.html
Reviewing the Results
While I'm pleased to see this report and I agree with all the objective results, I do have a few comments related to the assertions, conclusions and the test methodology.
First, the report implies we've claimed Windows Media Photo delivers better compression than JPEG-2000. In fact, we've claimed on many occasions that our compression efficiency is comparable to JPEG-2000, and overall the format delivers a number of significant advantages. Let's take a look at all these points in more detail.
The MSU report sites our presentation earlier this year at WinHEC 2006. We presented one slide (and one portion of the demo) that included references to JPEG-2000. Here's the slide:
For this particular image, the measured RGB peak signal-to-noise ratio (PSNR) of Windows Media Photo is slightly better than JPEG-2000 except at very high compression ratios (at the far left of the curves.) These results will vary significantly based on the image content, but these are pretty typical results. In this case, we're comparing with the JPEG-2000 encoder/decoder provided by Kakadu Software (http://www.kakadusoftware.com), which we've used as our baseline reference.
The Tale of Multiple JPEG-2000's
In the MSU report, their tests show Kakadu Software's JPEG-2000 implementation to deliver the poorest quality results of the nine implementations they tested. Frankly, I was surprised that there was that much difference among different implementations of JPEG-2000. One of Kakadu Software's significant claims is their computational performance: "Kakadu offers higher processing throughput, lower memory consumption and many more features than the [JPEG-2000 Verification Model.]" While the MSU report doesn't provide any details on this topic, typically different implementations of the JPEG-2000 encoder/decoder achieve better compression quality at the expense of additional processing. Since we have designed Windows Media Photo from the start for very high performance and a low memory footprint (suitable for implementation in cameras), we chose the highest performance version of JPEG-2000 we could find for comparison. We'll certainly take a look at including other JPEG-2000 implementations in future comparisons we present.
Again, I was a bit surprised with the range of quality differences among the different implementations of JPEG-2000. The design of basic Windows Media Photo compression algorithm doesn't leave room for that big a range of differences among different implementations. This is in part because our algorithm supports both lossy and lossless compression - by definition, all implementations will return identical results with lossless compression. While we believe there is a lot of room for implementers to optimize the performance for specific platforms, the design of the algorithm promises a more consistent level of quality for all implementations.
While not implemented in our current encoder, the Windows Media Photo compressed format is designed to support adaptive quantization. Rather than use a single quantization value for the entire image, a smarter (and more processor intensive) encoder can analyze the image and adjust the quantization dynamically (at macro-block resolution) based on the image content. This will enable a new class of smart encoders in the future that should deliver dramatic improvements in overall compression quality.
Looking Beyond Compression Quality
Overall, the MSU results show Windows Media Photo as "middle of the pack" compared to the nine different JPEG-2000 implementations, which fully supports the claims we have made of delivering image quality comparable to JPEG-2000.
When we look beyond compression quality, Windows Media Photo provides a number of key advantages compared to JPEG-2000:
- Windows Media Photo provides both lossless and lossy compression using the exact same algorithm. Any implementation will provide both compression options. This is not possible with JPEG-2000; two separate algorithms are required, and in most cases only the lossy version is implemented.
- Windows Media Photo is designed for lightweight processors and low memory footprints. The entire algorithm uses only integer mathematics, with no complex operations (mostly adds and shifts; limited multiplies and no divisions.) The image can be processed in segments, minimizing the amount of memory required. Overall it is ideally suited for implementation in low-end DSP's or even ASICS; this has not proved possible with the much more complex algorithm used for JPEG-2000.
- Windows Media Photo supports an extremely wide range of pixel formats (discussed in detail elsewhere in this blog.) JPEG-2000 has far less flexible support for image formats.
- The Windows Media Photo compressed bit stream design enables future implementation of adaptive quantization encoders, providing the option for significant quality improvements when processing power is available.
It's important to emphasize the benefits of a significantly more efficient, lightweight algorithm. Windows Media Photo can (and has been) implemented in low-end digital signal processors, typical of those used in consumer cameras. It could also be implemented in very low cost ASIC's. The JPEG-2000 algorithm is considerably more complex and is not appropriate for consumer device implementation. Even if sufficiently capable processor becomes cost effective, JPEG-2000's more complex algorithm will invariably require more power, significantly reducing battery life in portable consumer devices.
Not All Compression Errors are Equal
Getting back to image quality, while the measured PSNR between Windows Media Photo and JPEG-2000 are comparable, the nature of the artifacts from lossy compression between the two formats are very different. Depending on the particular image content (and the amount of compression) the effect of these differences can favor one approach over the other.
JPEG-2000 uses wavelet technology. As the compression ratio is increased, image detail is sacrificed. While this can often deliver visually pleasing results, it can damage an image in a way that cannot easily be recovered. (Once the detail is gone, it's hard to ever get it back again.)
Windows Media Photo delivers a more random distribution of error from lossy compression, independent of the frequency of the image content. The end result is artifacts that are closer to random noise, and in many cases, this is significantly less damaging to an image, especially when it will be encoded/decoded multiple times.
Here's an example (from my demo at WinHEC):
In this example, the uncompressed image in upper left has been compressed to 12.5% of the original size using JPEG, JPEG-2000 (from Kakadu Software) and Windows Media Photo. What is displayed here are the differences - the compressed image is subtracted from the uncompressed image and the absolute value is amplified so we can compare the nature of the errors.
From this we can clearly see that JPEG-2000 errors are concentrated in the portions of the image with high frequency content. (That pink jacket has a visible weave compared to the smooth fabric of the blue shirt.) Windows Media Photo has a more random error distribution, resulting in less error in the high frequency areas, but more error in the out-of-focus low frequency areas. Overall, our experience shows this is less damaging to the integrity of the original image.
It will certainly be interesting to prepare a similar comparison using one of the very high quality JPEG-2000 implementations identified in the MSU analysis. I will definitely pick up a copy of ACDSee and share the results of this comparison based on that implementation of JPEG-2000.
A Closer Look at PSNR
I's also like to make a few comments on the test methodology used in the MSU analysis. Again, overall I applaud their efforts and I don't disagree with any of their measured results. While PSNR is a very useful objective measurement of the total errors from compression, it is only one metric of image quality. The MSU report does present visual comparisons, but this requires a high compression ratio to generate visual differences. These high compression ratios are not typical of normal usage.
It's generally accepted when evaluating image quality using PSNR measurements, an image with a PSNR value above 45db is visually indistinguishable from the original. At the other extreme, when the PSNR value of an image is below 35db, the quality is sub-standard; it may be acceptable for web or reduced resolution display, but not for subsequent editing or re-encoding. That is the quality range we have focused during the development, testing and tuning of Windows Media Photo.
I am a little surprised that the MSU researchers only measured Y-PSNR. If I understand their methodology, they are only measuring noise in the luminance channel. This is measuring the sum of a portion of each of the three color channels. This type of measurement doesn't account for chroma errors, which are very common in many compression technologies that favor luminance quality over chrominance quality. The measurements I presented above are RGB-PSNR; we measure the total PSNR across all three channels in an RGB color space. This is the error that matters for image display and editing, since this is the color space that is used. For comparison, here's the complete table of my measurements for the image of the boy shown above:
psnrRGB includes measurements of all the image content, not just luminance, and therefore for all codecs and compression ratios, the psnrRGB values are lower than the psnrY (luminance-only) measurements. The JPEG-2000 measurements in the table above are using the Kakadu Software implementation. It will be very interesting to look at the same measurements with one or more of the higher quality JPEG-2000 implementations to see how much of their improvements are in the luminance channel only, or if there is equal improvements in the total RGB PSNR measurement.
Lenna is Not as Young as She Used to Be
My bigger concern is the choice of images used for the MSU evaluation. In all fairness, these are standard images for quality comparison; they are from an informal library of the universally used set of images that have been the baseline standard for compression evaluation for over 30 years.
But that's the problem. All of these images are very far removed from the typical images (digital photos) that we all use today. The "House" and "Lighthouse" images are from a set of test images produced by Kodak many years ago. "Lenna" was scanned from the printed centerfold of the November 1972 issue of Playboy magazine. (Oh, those crazy compression scientists!) I'm not sure of the provenance of "Barbara", but I believe it dates from the same time.
All these photos were originally captured on film, underwent an unspecified chain of optical and/or digital processing, and were then scanned either from film or print. The files used for the MSU tests are 512x512 pixels (1/4 Mbyte.) This down-sampling removes significant detail and virtually erases any film grain or other noise-like content that would be typical of digital photos. These images are very different from the typical digital camera photos that are the most common content we deal with today (and what we've used to optimize and tune Windows Media Photo.) These low-detail, down-sampled images (especially the very soft scanned-from-print image of Lenna) are also very friendly to JPEG-2000's wavelet compression.
The MSU results are valid (and typical of the way image compression quality comparisons have been done for years.) However, they're not as meaningful for today's use of image compression as they could be if full resolution original digital photos were used for comparison instead.
The two examples I showed above use images that originated from digital cameras. They were shot in RAW mode, converted to an uncompressed 16bpc TIFF file, then reduced to an uncompressed 8bpc BMP file that was used as the source for compression into the various formats. These are two photos from a reference set of 30 different photos I regularly use for Windows Media Photo testing. This library of photos includes examples from a wide range of digital cameras, representing virtually every type of sensor technology in use today (CMOS, CCD, Foveon, SuperCCD-SR, RGBE.) They also include a wide variety of "typical" digital photos, including snapshots, landscapes, night shots, flash photography and more.
Here's a preview:
I believe this image library better represents the images we use today, and are more appropriate for comparing different compression solutions. I also believe it's essential to perform compression tests using the full resolution images as originally captured, not a down-sampled smaller image.
So, my homework assignment is to package up this image library and make it freely available for general use. Additionally, I'll prepare and publish updated test results comparing Windows Media Photo and at least a couple implementations of JPEG-2000 using these test images.
Summary
My thanks to the researchers at the Graphics and Media Lab at Moscow State University for adding Windows Media Photo to the many compression technologies they have analyzed. We welcome and appreciate any and all independent reviews of this new file format. Their report is comprehensive and informative, and while (as I've described above) I don't think it tells the whole story of Windows Media Photo, it's certainly an accurate summary of PSNR-measured quality of this standard set of test images compared to JPEG-2000. It confirms our own results and gives me some additional insights on future test and analysis procedures we can use moving forward.
