Bill Crow's HD Photo Blog

HDR     This file format is only used for encoding to the Windows Media Photo 24bppRGBE pixel format.  This special floating point pixel format uses a shared exponent and three independent mantissas to encode the entire pixel.  Unfortunately, while PhotoShop CS 2 supports saving in .HDR file mode, it only saves using the compressed option.  WMPEncApp.exe only supports uncompressed .HDR files.  So, we’ve provided a simple command line utility called HDR2HDR that only does one thing:  It reads and decompresses a compressed .HDR file and saves it as an uncompressed .HDR file.  HDR2HDR is included in the DPK.  If you don’t have the DPK, it’s also available here.

Fixed Point Pixel Formats

The one significant omission from the pixel formats listed in the tables above is the set of fixed point formats.  As discussed in our previous blog on high dynamic range, wide gamut pixel formats, the fixed point pixel formats provide an excellent solution for retaining the full content of an image source while still providing efficient storage and processing.

However, these fixed point formats are one of the innovations introduced with Windows Media Photo and the new graphics infrastructure in Windows Vista and .NET Frameworks 3.0.  No other file format supports fixed point encoding.  Because WMPEncApp.exe has no built-in capability for pixel format conversion, there is no way we can encode fixed point Windows Media Photo files from standard TIFF or BMP files.

In reality, if we write a little software, we could convert floating point image data to fixed point and still store it in a TIFF file.  While the resulting TIFF file containing fixed point image data would not display correctly, it could be converted to a fixed point Windows Media Photo file using WMPEncApp.exe.  That’s why there are –c option values defined for the various fixed point pixel formats.  But for this to work, you’re on your own to first convert the uncompressed image data to fixed point.  I do have a utility that does this, but trust me, it’s not ready for any public use!

If you do choose to write your own software to generate uncompressed fixed point data, you might as well simply write a WIC or Windows Presentation Foundation (WPF) application and directly call the Windows Media Photo converter.  This makes a lot more sense than writing out fixed point data in a non-standard TIFF file and then using WMPEncApp.exe.

If you can hang in there for Part 2, we will discuss how you can use the WIC Tools to create fixed point Windows Media Photo files.  Since the WIC Tools have full access to all the capabilities of WIC, the encoder can call the appropriate pixel format converter and create a Windows Media Photo file in any pixel format, independent of the uncompressed source image format.

Unsupported Pixel Formats

There are several other pixel formats that can’t easily be encoded using WMPEncApp.exe

It is possible to encode images in any of the pre-multiplied alpha pixel formats, but the uncompressed image will have to first be pre-processed to multiply the RGB channels by the alpha channel value.  This can be done using the appropriate blending operations in Photoshop.  If there is any interest, I’ll describe that process in another installment (and provide an action to offer some automation.)  Like fixed point pixel formats, a TIFF file in a pre-multiplied pixel format will not display correctly, but can be converted to an equivalent Windows Media Photo file using WMPEncApp.exe.  But also like fixed point pixel formats, if you wait for Part 2, we’ll discuss how this can be done much more easily using the WIC Tools.

While WMPEncApp.exe includes a –c option value for 32bpcRGB101010 pixel format, there is no way to create this pixel format in an uncompressed file using PhotoShop.  This is another pixel format that we’ll handle using the WIC Tools, which we will discuss in Part 2.

WMPEncApp.exe does not support encoding any of the n-Channel pixel formats.  Creating uncompressed n-Channel data will be a whole topic in itself, which we will address in a future installment of this blog series.  Once we understand how to create this n-Channel data, we’ll be able to use the WIC Tools to encode it in Windows Media Photo file format.

Compression Choices: -q, -d & -l Options

There are three parameters that control the tradeoffs between image quality and compressed file size.  In addition to their individual effect, it’s also important to understand how these three parameters interact with each other.

-q   Quantization (1-255)

One of the principal ways lossy compression is achieved is to “quantize” a set of continuous values into a smaller set of representative values.  In this way, “loss” is achieved by mapping values that are close together to the same value.  Only the remaining set of values needs to be coded and saved, reducing the amount of storage required.   The greater the degree of quantization, the more the content can be compressed, but in doing so, more of the small differences among similar values are lost.

The quantization level, specified by the –q option basically defines the amount of similarity that can be discarded.  It is an arbitrary range from 1-255; the quantization value does not correspond directly to any specific difference amount.  Quantization determines the desired image quality rather than the desired compression ratio.  The actual compression ratio is a function of both the quantization and the image content; images with less complex content will have fewer differences among values and will achieve better compression without the need for greater quantization.  Additionally, the quantization is also highly dependent on the specific pixel format, most importantly the bit depth.  Higher values will be required to greater bit depths to achieve a comparable compression ratio, since larger bit depths provide a greater range of possible values and therefore will need more quantization.

Windows Media Photo provides the unique capability of preserving all data values during quantization, effectively providing mathematically lossless compression.  When the quantization is set to 1, no values are discarded and all encoded pixel values will be returned with absolutely no loss.  This is the default setting if no value for the –q option is specified.

-d   Chroma Sub-sampling (0, 1, 2 or 3)

We can choose to reduce the resolution of the chrominance of an image prior to the quantization process.  Reducing the chrominance resolution, or chroma sub-sampling, has long been understood as an effective way to reduce image content with very little perceptible degradation.  In fact, virtually all television or video you watch, whether analog or digital, takes advantage of chroma sub-sampling to reduce the required bandwidth.  The JPEG compression format always uses chroma sub-sampling as well.  In fact, the unique capability of Windows Media Photo is not that we provide chroma sub-sampling, but that we provide a mechanism for you to reduce or eliminate this technique to improve image quality.  Of course, this only applies to RGB color images.

An image is first reorganized from RGB into a channel for luminance and two channels to describe the color information (or chrominance.)  If all chrominance is discarded, what’s left is a monochrome image.  Typically, we don’t want to go that far!

Many video systems, as well as the JPEG compression format (or at least the most common variant of it that we all use) discards 75% of the chrominance information.  The resolution of the color information is reduced by a factor of two in both dimensions.  So every four pixels in an image are represented by four luminance values but only two (one for each chroma channel) chrominance values.  What started out as 12 values (four pixels with three channels each) has been cut in half; only 6 values (four luminance values and two chrominance values) have to be saved.

In the world of digital imaging, this is referred to as 4:2:0 chroma sub-sampling, or more simply as 4:2:0.   When all chrominance information is retained (no values are discarded), this is referred to as 4:4:4.  Another popular approach, particularly for professional video applications, is to only discard 50% of the chroma values; two values for each chroma channel, or four values in total are retained.  This is referred to as 4:2:2.  Finally, if we discard all color information, retaining only the luminance, this is described as 4:0:0.  Windows Media Photo supports all these modes.

-c 3 (4:4:4)  All color information is retained, assuring full resolution of the chrominance information.  This is the default and is the recommended setting to achieve the best overall image quality.  Whenever an image is stored as an intermediate format and further editing is anticipated, it is highly recommended to use 4:4:4.

-c 2 (4:2:2)  The color information is encoded at ½ the resolution of the luminance information.  Four each set of four pixels, four luminance values are used and the eight chrominance values are reduced down to four (two for each chroma channel.) This provides perceptively lossless color encoding for the final delivery of an image.  However, if further editing of the image is anticipated, it’s recommended than any chroma sub-sampling be avoided.

-c 1 (4:2:0)  The color information is encoded at ¼ the resolution of the luminance information.  Four each set of four pixels, four luminance values are used and the eight chrominance values are reduced down to two (one for each chroma channel.) This is the same sub-sampling used by JPEG.  When converting a JPEG file to Windows Media Photo, there is no need to specify a higher chroma sub-sampling mode than 4:2:0.

-c 0 (4:0:0)  All color information is discarded and only the luminance information is retained, effectively creating a monochrome image.  For performance reasons, Windows Media Photo uses a non-traditional method to calculate luminance.  Therefore, the resulting monochrome image will not appear identical to a monochrome version of the image created using other tools.  Additionally, although all color information is discarded, the pixel format is not changed, so the image is still stored using an RGB pixel format.  It is strongly recommended that if you want to create a monochrome image, the image should first be converted to monochrome using an appropriate image editing application to achieve the desired result, and then this monochrome image should be encoded using the appropriate Gray pixel format.

-l   Overlap Processing (0, 1, 2)

Windows Media Photo uses an advanced version of a macro-block based compression scheme.  To achieve the best performance and minimize the amount of memory required to encode or decode an image, the overall image is subdivided into a set of 16x16 pixel macro blocks.  Each macro block is are further divided into four 4x4 pixel blocks.  All image encoding and decoding operations are peformed on these blocks and macro-blocks.  As a result, for high quantization values (when we are discarding a higher amount of similar pixel values), the steps between blocks and macro blocks may become visible as artifacts in the compressed image.  This is very common with JPEG (which also uses macro blocks) and significantly reduces the amount of compression that can be used without creating these visible artifacts.

Windows Media Photo addresses this problem through a combination of better quantization and an additional step of overlap processing.  This overlap processing takes into account the values of pixels in neighboring blocks and macro blocks when choosing the quantization values that represent similar adjacent pixels.  By doing so, the visible differences among adjacent blocks and macro blocks are dramatically reduced.

Two levels of optional overlap processing can be specified via the –l parameter.   Single level overlap processing (-I 1) is performed at the 4x4 block level.  For all pixels in the block, bordering pixels in adjacent blocks are also evaluated when choosing the quantization values for that block.   Double level overlap processing (-l 2) also analyzes neighboring adjacent pixels when choosing quantization values at the 16x16 macro-block level.