Packaging Team Blog: Open Packaging Conventions

November MSDN Magazine feature on OPC Digital Signature UX

jack davis — Mon, 09 Nov 2009 20:49:00 GMT

This month’s issue of MSDN Magazine (November 2009) includes our article on designing effective digital signature user experiences – check it out at:

Digital Signatures: Application Guidelines on Digital Signature Practices for Common Criteria Security
http://msdn.microsoft.com/en-us/magazine/dd890995.aspx

The article focuses on user experience (UX) considerations to make electronic signatures as easy to use and understand as hand-written signatures. Digital signatures, however, add particular UI caveats that developers and UX designers need to consider.

Introduction to Windows 7 Packaging Digital Signature APIs - Pointers to learning resources

Ruke Huang — Fri, 04 Sep 2009 20:46:00 GMT

Chapter 12 of the Open Packaging Conventions (OPC) standard (ECMA 376-2 1st edition, 2006) defines the functionality and operation of “digital signatures” for signing content stored in OPC-based file formats. Windows 7 incorporates built-in native-code support for accessing OPC-based files along with support to both signing and validating stored content using digital signatures. This blog provides links to information about digital signatures and how to use the OPC digital signature APIs.

OPC digital signatures are based on the W3C XML Signature Syntax and Processing specification (aka, “XML DigSig”). As background to understand OPC digital signatures, it’s helpful to first read the XML DigSig specification. The MSDN Packaging API Overview provides a starting point for learning about the Windows 7 Packaging APIs and associated digital signature APIs.

After reading the Packaging API Overview then you can continue with the MSDN Digital Signatures Overview to familiarize yourself with Packaging digital signature APIs.

Of course, seeing actual working sample code is a great way to understand a new set of APIs. The Music Bundle Signature Sample is very useful in showing how the APIs actually operate. This sample shows how you can use OPC digital signature APIs to sign content stored in a package and then later validate the signature along with the associated signed content.

Comparing the OPC managed-code and native-code Part URI helper APIs

opc — Tue, 18 Aug 2009 22:26:00 GMT

Windows 7 includes new native-code APIs that provide functionality to compose and parse Part URIs. These APIs are provided through the IOpcUri and IOpcPartUri interfaces. Managed-code provides similar functionality with the System.IO.Packaging PackUriHelper class.

A difference between the two API sets is that while the managed-code PackUriHelper APIs support both Pack URIs and Part URIs (also called "part names"), the native-code IOpcFactory::CreatePartUri and IOpcPartUri APIs only provide functionality to create and manage Part URIs. The native-code Packaging APIs do not provide any functionality to access packages or package content as external resources, and therefore only supports Part URIs to address part content within the immediate package.

The managed-code PackUriHelper and native-code CreatePartUri/IOpcPartUri APIs are both designed to be ECMA 376-2 (2006) compliant.

The table below shows a comparison between the managed-code and native-code APIs, and which APIs provide similar functionality:

Managed API	Equivalent Native API	Parity
PackUriHelper.CreatePartUri	IOpcFactory::CreatePartUri	See 1.1
No equivalent API	IOpcFactory::CreatePackageRootUri
PackUriHelper.GetRelationshipPartUri	IOpcUri::GetRelationshipsPartUri	Identical
PackUriHelper.GetRelativeUri	IOpcUri::GetRelativeUri	Identical
PackUriHelper.ResolvePartUri	IOpcUri::CombinePartUri	See 1.2
PackUriHelper.ComparePartUri	IOpcPartUri::ComparePartUri	Identical
PackUriHelper.GetSourcePartUriFromRelationshipPartUri	IOpcPartUri::GetSourceUri	Identical
PackUriHelper.IsRelationshipPartUri	IOpcPartUri::IsRelationshipsPartUri	Identical
PackUriHelper.Create	No equivalent API
PackUriHelper.ComparePackUri	No equivalent API
PackUriHelper.GetNormalizedPartUri	No equivalent API
PackUriHelper.GetPackageUri	No equivalent API
PackUriHelper.GetPartUri	No equivalent API

1. API differences

1.1 IOpcFactory::CreatePartUri differences from PackUriHelper.CreatePartUri

The functionality of the native-code IOpcFactory::CreatePartUri API, follows the rules provided in ECMA 376-2 OPC standard, Appendix A.3 for resolving a relative reference to a valid Part URI (part name). The IOpcFactory::CreatePartUri and IOpcUri::CombinePartUri native-code APIs both produce the same output ("Native-code Behavior" below).

Input	Native-code CreatePartUri Behavior	Managed-code CreatePartUri Behavior
Contains bad UTF-8 encoding e.g. '%FC':	Leaves %FC unchanged in output.	Changes to valid UTF-8 output %C3%BC. (See * below)
Contains empty segments i.e. multiple forward slashes	Collapses multiple forward slashes to a single forward slash.	Does not collapse. (This is OPC rule violation).
Begins with 2 or more forward slashes	Collapses multiple forward slashes to a single forward slash.	Fails, throws an "ArgumentException".
Contains a single trailing forward slash	Removes trailing slash.	Fails, throws an "ArgumentException".
Contains a percent-encoded NULL (%00)	Returns an "E_UNEXPECTED" HRESULT error.	Percent-encoded NULL is kept unchanged, and returns a valid part name.

* - Characters above 0x7F are required to be escaped using UTF-8. For example, if the input URI contains %FC, which is not a legal UTF-8 sequence, the rules provided in RFC 3987 section 3.1, require leaving the input unchanged. The PackUriHelper.CreatePartUri method however, decodes %FC as UTF-16 to its octet <FC> (character 'ü') and then re-encodes it to the valid UTF-8, '%C3%BC'. Unfortunately, this approach is incompliant with RFC rules. [Note: If the relative reference contains a %FC, both IOpcUri::CombinePartUri and PackUriHelper.ResolvePartUri leave it unchanged.]

1.2 IOpcUri::CombinePartUri differences from PackUriHelper.ResolvePartUri

The managed-code PackUriHelper.ResolvePartUri API differs from the native-code IOpcUri::CombinePartUri API in that it resolves the input relative reference against a base URI, but it does not ensure that the result is a valid part name. In contrast, the native-code IOpcUri::CombinePartUri API both resolves the input and ensures that the result is a valid part name.

Input	Native-code CombinePartUri Behavior	Managed-code ResolvePartUri Behavior
Contains percent-encoded unreserved characters	Un-encodes unreserved characters, resulting in a valid part name.	Leaves characters in encoded form resulting in an invalid part name.
Contains empty segments i.e. multiple forward slashes	Collapses multiple slashes to a single forward slash, resulting in a valid part name.	Does not collapse empty segments, resulting in an invalid part name.
Begins with 2 or more forward slashes	Collapses multiple slashes to a single forward slash, resulting in a valid part name	Fails, throws an "ArgumentException".
Contains a single trailing forward slash	Removes trailing slash.	Does not remove trailing slash resulting in an invalid part name.
Contains a percent-encoded NULL (%00)	Returns an " E_UNEXPECTED " HRESULT error.	Percent-encoded NULL is kept unchanged, and returns a valid part name.

- Ali Naqvi

Comparing the OPC Managed-Code and Native-Code APIs

opc — Thu, 06 Aug 2009 00:57:00 GMT

Windows 7 incorporates a new set of native-code APIs that provide integrated support for the ECMA 376-2 (2006) Open Packaging Conventions (OPC) standard. These new native-code Packaging APIs provide similar functionality to what had previously been available in .NET managed-code through the System.IO.Packaging APIs.

This article highlights programming differences between the native-code and managed-code APIs, and will be useful to developers who are looking to transition applications from one API environment to the other. Both OPC APIs are compliant with the ECMA 376-2 (2006) standard, and files created by either API set can be compatibly accessed by the other.

Differences Between the OPC Managed-code and Native-code APIs

Zip Encoding:
The managed-code System.IO.Packaging APIs create packages using the ZIP32 encoding by default. You cannot specify which encoding to use when creating the package. If the package size exceeds the limits imposed by ZIP32 encoding, the package is automatically saved using ZIP64 encoding. The native-code Packaging APIs differ in that they create packages using ZIP64 encoding by default. You can specify in the IOpcFactory::WritePackageToStream API flags to use ZIP32 encoding, however, if the package size exceeds the limits imposed by ZIP32 encoding, the API will fail and the package will not be saved.

In-place Editing:
The managed-code System.IO.Packaging APIs allow you to open a package, make changes to the content, and then save the changes back to the package. The native-code Packaging APIs differ in that they do not allow you to edit a package in place. They can read a package using the IOpcFactory::ReadPackageFromStream API, make edits, and then save to another package file via the IOpcFactory::WritePackageToStream API. Using the APIs to simultaneously read and write to the same stream, or to open two streams to the same file at the same time is not supported.

Core properties API:
The native-code API currently does not provide any equivalent of the managed-code PackageProperties class, which provides ability to read and edit the Core Properties part. When using the native-code APIs you will have to manually get the Core Properties part from the package, parse it, and then edit its contents. The MSDN Set Author Sample shows how to access and update the contents of the Core Properties part.

Pack URI support:
The managed-code PackUriHelper class supports creating URIs based on the Pack URI scheme* (pack://) that address both parts in a package and an entire package. The native-code Packaging APIs only make use of part URIs, and as such, only support creating and operating with the path component of Pack URIs that represent part names (IOpcFactory::CreatePartUri and IOpcPartUri).
*ECMA 376-2 (2006), Annex B

PackWebRequest/PackWebResponse:
The managed-code API includes PackWebRequest and PackWebResponse classes that allow you to access content within a URI-addressable package stored on a website. PackWebRequest and PackWebResponse are based on the managed-code WebRequest and WebResponse classes that support URI requests to access resources over the web. The native-code API provides no equivalent to this feature.

Content Types Stream missing in the Package:
The ECMA 376-2 (2006) OPC standard is unclear as to what to do if the [Content_Types].xml stream is missing. If the [Content_Types].xml stream is missing the managed-code Package.Open method will open the package, as if it were empty. (If the [Content_Types].xml stream is missing, none of the parts in the package are recognized as valid parts.) The native-code IOpcFactory::ReadPackageFromStream API, however, considers a package without a [Content_Types].xml stream an error, and will fail if there is no [Content_Types].xml stream.

Compression for Relationships Parts:
In the managed-code APIs, the relationships parts use the same compression as the source part; and for the package relationships part, no compression is used. In the native-code API, all relationships parts use a default compression of OPC_COMPRESSION_NORMAL.

Accessing Relationships Parts:
The managed-code APIs allow you to use the Package.GetPart API with the name of the relationships part to get that part and its content. They also allow you to perform all operations that are permitted on parts.

The native-code Packaging APIs do not allow direct access to relationships parts. IOpcPartSet::GetPart will return an error if requesting a part with a relationships part name. Instead you can call IOpcPart::GetRelationshipSet or IOpcPackage::GetRelationshipSet to get the IOpcRelationshipSet object. The IOpcRelationshipSet object provides access to the individual relationships, and a read-only stream to the relationships markup.

Editing Relationship Stream:
The managed-code APIs allow you to get the stream for a relationship part's content, and edit it. The native-code IOpcRelationshipSet::GetRelationshipsContentStream API only provides a read-only stream to the relationship part's content. To add or delete relationships from the IOpcRelationshipSet, you need to use the CreateRelationship and DeleteRelationship APIs.

- Ali Naqvi

Adventures in Packaging - Episode 1

jack davis — Mon, 18 May 2009 07:21:00 GMT

Technorati Tags: OPC,OPC basics,Open Packaging Conventions,ISO 29500,ECMA 376

If you're developer who creates or accesses data files (and who doesn't?), then this blog might be one for you to keep an eye on.

Welcome to the first installment of Microsoft Packaging Team blog!

The Packaging team works in the Microsoft Windows Division to provide programming APIs that support the Open Packaging Conventions standard. Open Packaging Conventions ("OPC") is a new file technology documented by the ISO/IEC 29500-2 and ECMA 376-2 standards.

Several members of the team worked on the original System.IO.Packaging APIs released for managed-code in .NET 3.0. We're particularly excited now with the upcoming release of Windows 7 which will include new native-code Win32 Packaging APIs that will ship as part of the operating system! Microsoft is strongly embracing open standards and interoperability - the commitment to incorporate OPC as an integral element of the operating system is another step in that direction.

Perhaps an initial question might be, "what is OPC and what makes it so compelling"? Rather than being a specific file format, OPC is a container-file technology that's designed to create file formats based on a flexible open framework. OPC integrates elements of Zip, XML, and the Web into an open industry standard that makes it easier to organize, store, and transport application data. OPC is the core file technology for many of the new file formats supported by Microsoft products. This new generation of OPC-based files include Office 12 versions of Word (.docx), Excel (.xlsx), and PowerPoint (.pptx), along with XPS (.xps), Semblio (.semblio), plus a growing number of other new Microsoft and third-party applications such as Autodesk AutoCAD (.dwfx) and Siemens UGS (.jtx). While each of these file formats share OPC as a foundation, the data content contained in each differs depending on the specific format. Before going too much further, perhaps a couple of words on terminology.

In packaging terms...

A "package" corresponds to a "Zip archive".
A "part" corresponds to a "file" (i.e. "a data stream") stored within the Zip.

ZipPackages
In using Zip as its physical container, all OPC-based file formats are, in fact, Zip files. You can simply append ".zip" to any OPC file (package) to open and examine its contents in Windows Explorer or your favorite Zip utility. This makes packages a great choice for organizing multiple application data streams into a single file that’s portable and easy to access. It’s important to note, however, that while all OPC files are Zip files, the reverse is not necessarily true: not all Zip files are OPC files. OPC adds two requirements to a Zip file:

The names of all of the parts (files) stored in an OPC package must be URI-compliant.
The package must contain a “[Content_Types].xml” file.

URI Part Names
The first requirement*, URI-compliant part names, enables potential web-access to the parts stored in a package when the package is located on a web server. In situations where an original filename is not URI-compliant, the filename is typically "percent-encoded" to a URI-compliant form. For example, a part with the filename "my file.txt" would be percent-encoded as "my%20file.txt" (you've probably seen this in many of the URLs on your Web browser).
*Re. ISO 29500-2, Section 9.1.1 “Part Names”.

The [Content_Types].xml Part
The second requirement**, a "[Content_Types].xml" part, is used so that the content of all the parts in the package are clearly and accurately defined, not only today but into the future. Since many three or four characters filename extensions have multiple meanings, Content_Types is used to accurately define part content through the use of MIME-style media types. The markup within a Content_Types part is fairly simple and contains just two basic types of elements: "Default" and "Override" elements.

Default: associates a generic file "Extension" to a specified "ContentType".
Override: associates a specific "PartName" to a specified "ContentType"
(overrides any Default extension association).

The following is an example of a [Content_Types].xml part:

<?xml version="1.0" encoding="utf-8" ?>
<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">
<Default Extension="htm" ContentType="text/html" />
<Default Extension="css" ContentType="text/css" />
<Default Extension="png" ContentType="image/png" />
<Default Extension="jpg" ContentType="image/jpeg" />
<Default Extension="mp3" ContentType="audio/mpeg3" />
<Default Extension="xml" ContentType="application/xml" />
<Override PartName="/docProps/core.xml"
ContentType="application/vnd.openxmlformats-package.core-properties+xml" />
</Types>

When using the Win32 native-code Packaging APIs or .NET managed-code Packaging APIs the Content_Types file is created and managed automatically. If you're creating a Zip package on your own you'll also need to include a [Content_Types].xml file that contains the markup to define the content types for all of the parts contained in the package. As shown in the above example this is fairly simple to do.
**Re. ISO 29500-2, Section 9.1.2 “Content Types”.

The Adventure Continues...
These are some basics to help get you started but there are many other additional services that OPC provides. A goal of this blog will be to highlight uses and features of both the Win32 native-code Packaging and .NET managed-code System.IO.Packaging APIs for creating, organizing, and accessing information stored in OPC files. We think you'll find the Open Packaging Conventions an exceptionally flexible standard for managing application data; and in particular, the packaging APIs an indispensable tool to help you take advantage of this new file technology - more to come in following episodes...

Thanks for listening,
Jack

PS: Here are some links for more information and related topics about Packaging:

Open Packaging Conventions (OPC) MSDN Forum - New!
http://social.msdn.microsoft.com/Forums/en-us/os_opc/threads/
OPC: A New Standard for Packaging Your Data
http://msdn.microsoft.com/msdnmag/issues/07/08/OPC/default.aspx
The Addressing Model of the Open Packaging Conventions
http://msdn2.microsoft.com/en-us/library/aa480199.aspx
Office Extensibility / Open XML SDK blog
http://blogs.msdn.com/brian_jones/
Win32 Native-Code: Data Access and Storage | Packaging
http://msdn.microsoft.com/en-us/library/dd371623.aspx
.NET Managed-Code: System.IO.Packaging
http://msdn.microsoft.com/en-us/library/system.io.packaging.aspx
ISO/IEC 29500-2: Open Packaging Conventions (2008)
http://standards.iso.org/ittf/PubliclyAvailableStandards/c051459_ISOIEC%2029500-2_2008%28E%29.zip
ECMA 376-2: Open Packaging Conventions, 1st Edition (2006)
http://www.ecma-international.org/publications/files/ECMA-ST/Office%20Open%20XML%201st%20edition%20Part%202%20(PDF).zip
ECMA 376-2: Open Packaging Conventions, 2nd Edition (2008) (matches ISO)
http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-376,%20Second%20Edition,%20Part%202%20-%20Open%20Packaging%20Conventions.zip
Here's a recent article and sample by Rich Quackenbush - the title says it all...
Using OPC to Store Your Own Data
http://www.devx.com/dotnet/Article/41235
And here's a fun post by Matt Hamilton describing his use of packaging in his Comicster application.
Zip Your Streams with System.IO.Packaging
http://www.madprops.org/blog/zip-your-streams-with-system-io-packaging/