Craig Kitterman's Interoperability Community Blog

Sam Hiser says comments on his blog are posted automatically (I have no idea, I haven't posted there in a while), but yours are turned off.  I never know what reality is for Sam, but I thought I'd do a real experiment as long as I was here finding out exactly what you said.

I can't wait for the demonstration of "fidelity" in da Vinci to show up and make the ODF Translator folk appear foolish.  I have seen no evidence of how that could possibly happen, and I boggle at the notion that fidelity means roundtripping wrapped up binary stuff that they can't handle so it comes back into Office ok but never soils the lips of ODF (although OpenOffice.org may decide to work with those binary bits, completely out-of-band with ODF of course).  

Wait, isn't this the sneaky stuff that Microsoft is supposed to be doing in OOX and that is decisive evidence of evil afoot?

I guess one man's fidelity is another's contradiction, what?

"The test cases used for the project were based on real world scenarios and documents provided by the European Commission and the resulting translations are very good."

There is a reason why Microsoft did not use their own test suite to validate this "open source" project, don't you think?

You are deliberating hiding this.

Dennis,

Do yourself a favor. Password-protect an Office 2007 document. Then, much like anyone from Microsoft, paid by Microsoft, or selling himself out to Microsoft will try to convince you to do, crack-open the ZIP file and explore.

Hallucination!

@Stephane,

[Re: your comments elsewhere, I am Dennis E. Hamilton, I go by orcmid.  Some sites where I am registered use my given names and I'm never quite sure how that happens, but my own pages provide my accurate identity and contact information.  I never post anonymously anywhere.]

I don't understand the concern about Password protection.  You should finally look at the ODF specification yourself and read the rules for encryption of Zip parts in ODF packages.  But mainly, I don't get your point.  What is it you expect me to find when trying to inspect a password-protected .docx as a Zip?  

About test suites.  I don't understand how a test suite for, say, Microsoft handling of .docx files would be useful in testing .odt files.  You mean they should take their test documents and send them out the ODF translator and see what happens, like maybe round-trip them back in and compare with the original test document?  

I have no idea what the Microsoft Office test documents are like, but I would suspect that many if not most are unsuitable for that.  When I make up documents to test an application, they are designed for many things and rarely for exercising the conversion between a native format and a "peer" format.  I think testing for interconvertibility with a "peer" format will have mostly unique tests looking for edge cases and proper rendering (or accounting for) areas where there is an area where the formats are not conformable.  But that's just what comes to mind based on my own testing experience.

In any case, I thought the translator team's choice of a test suite from the EU reflecting the kinds of documents that agencies there  want to be able to interchange was a reasonable start.  Focused coverage on the least that has to work strikes me as an useful way to get to 1.0.  

The hard part is the current lack of very many fully-conformant ODF documents to play with, so I notice the translator exception list has to reference OO.o-specific behavior instead, in many places in the list of exceptions and caveats.  

Focusing where there is a demand seems like the right first thing to do.  Besides, isn't that the popular wisdom for how Microsoft enters an area.  Watch out for release 3.0!

Dennis,

If I suggest to password-protect an Office 2007 document and crack-open it, don't you think there is a reason for that. Try it.

As for Microsoft's own test suite, I have probably not worded it well enough. When I mentioned the Microsoft test suite, I was specifically thinking of their document corpus. It's obvious that, they must be gathering those documents for 15 years or so, and that those are certainly critical to make sure that a build does not introduce a regression.

My question was whether or not this corpus was used to validate the CleverAge project. And, as a corollary, I was wondering whether there was a reason why Microsoft does not talk about it, and instead let the CleverAge worry about this stuff, as if it did not matter that their work has to be validated by Microsoft.

To understand the bigger picture, a refresher : "Pretending interoperability", http://fussnotes.typepad.com/plexnex/2006/11/pretending_inte.html

@Stephane

I don't see anything that isn't by design.  The document has been encrypted (weakly) and you can't get around the encryption by trying to open the .docx as a .zip.  Do you think you should be able to open the .zip even though the author has protected the document?

I made two versions of the same file, one with a password, one without.  The one with the password is almost twice the size of the one without.  I also installed WinZip, in your honor, and set it up to work so I could Open As ... the .docx files with WinZip (they still launch Word by default).  The unencrypted one opens up nicely as a Zip.  On the other, Zip doesn't recognize it as a Zip file so it offers to add it into a Zip for me.  

I think you have an unmet expectation here.  It strikes me that the encryption facility is operating as expected.

So again, what's the problem?  

Dennis said "(...) Do you think you should be able to open the .zip even though the author has protected the document? (...) On the other, Zip doesn't recognize it as a Zip file"

Well,

1) obvious contradiction in the specs : files are not ZIP-based. Spoiler : read my article here, towards the end : http://www.codeproject.com/useritems/office2007bin.asp

2) Previous versions of MS Office would keep the document type the same, underlying stream names and types the same, and would only encrypt the actual buffers. So Content Management Servers would still be able to deal with it as an OLE file, able to read the metadata, and so on. This is a feature regression and ramifies to non-Microsoft CMSes.

If you know this, and still think we are talking about ZIP and XML only, which is what OOXML claims, that should be one counter-example that blows it.

OK, I get your complaint.  However,

1. I believe the previous behavior of Office was a bug.  A passworded file should disclose nothing and it should not be subject to alteration in any way.  That's from a better understanding of security practices than when passwording was first introduced.

2. This is application behavior and not part of the OOX specification.  There are perhaps better security provisions that are carried out inside the OOX structure and these may be far more appropriate in a variety of conditions where there are sophisticated requirements and practices in place.

3. I could have done the same thing by encrypting the file myself, even using something like OpenPGP (an "open standard").  But the convenience of the Word 2007 password and of having Word automatically recognized one of its encrypted documents is certainly far simpler in terms of usability.

4. Now, do yourself a favor.  Make a simple document in OpenOffice.org Writer.  Save it in the ODF-based format that OO.o implements.  Then Save As to a new name and specify password protection.  Notice the differences in the files.  You can open the password-protected Zip structure, but there are pieces in there (with XML extensions) that aren't XML and not readable in an XML processor.  The only readable parts are the manifest and the meta.xml part - oh, and the cute thumbnail that is substituted in a protected document.  (I claim that even having meta.xml not be encrypted is a security flaw, because it discloses information; the manifest is less revealing but still iffy.)   This is all provided for in the ODF specification.  Does it help you in your cited use case?  I don't think so.  

"A passworded file should disclose nothing and it should not be subject to alteration in any way."

You are taking my point out of context. I say fine, go there but I won't bother respond to that.

"The only readable parts are the manifest and the meta.xml part "

Well, that's exactly what it should do. And that's consistent what older versions of MS Office do as well.

This is what Content Management Servers expect to be able to do. It's a feature, you know.

I won't teach a lesson on why metadata is important. Let's leave it at that.