In the Trenches

What is Open Metadata?

marcmill — Thu, 03 Nov 2005 00:01:00 GMT

In Windows Vista, you'll be able to register property handlers on your own file types. This is in fact what my PDC Talk was all about. These property handlers are used to expose your file's contents to other parts of the system. They use a standard schema to enable stacking, grouping, filtering, and other metadata-search operations across heterogeneously typed items. The Universal Search Engine will for example call the registered property handlers to enumerate all the properties on each file during indexing. So, indeed, the most important feature that a registered property handler provides is metadata reading. In this sense, the interface for property handlers, IPropertyStore, is a simplified version of IFilter.

But property handlers provide another important feature. They allow the system to write metadata to your file types. Although this introduces extra complexity for the file type designer, it enables many cool end-user features. For example, a user doesn't have to start up the file's associated application in order to change the Author property. Instead, the user can simply select the item from within Explorer and drag it into a different Author stack. Or, just type in the name of the new Author into the preview pane.

Property handlers which support open metadata can reason over properties which aren't intrinsic to the underlying file type. These handlers can accept any value for any property, persist that value, and then return the same value later. And by "any property", I include those properties whose schema is invented after the property hander is written. Quite a trick!

Of course, the platform provides serialization APIs and other services to make it easier to support open metadata (the IPropertyStoreCache implementation returned by SHCreatePropertyStoreOnMemory supports IPersistStream), but usually adding this support to your file format carries some nonzero cost. So why do it? Supporting Open Metadata means that users can organize their files of your type along dimensions that you may not have anticipated. This is a very powerful idea! Here are some example scenarios that come to mind:

The "rating" property in Windows Vista isn't intrinsically supported by most file formats, but it's a cool feature to be able to rate your Visio documents, for example, and then sort or group them by rating.
Imagine an IT manager at a law firm who deploys the "legal case number" and "client name" schema elements onto that firm's computers. The lawyers at the firm will be able to within Explorer (or the Common Item Dialog) assign case numbers and client names to the various documents associated with that case. This could include not only word processing and spreadsheet documents, but other file types like faxes and media files which represent, say, evidence or recordings of hearings.
The IT manager could then get fancy by creating and deploying custom Virtual Folders which perform interesting queries like:
Stack all my documents by their associated legal case number, then group those stacks by client name.
That IT manager will look like a super-star!
Imagine being able to attach a "priority" property to documents, like you can for messages in Outlook. Then you could quickly filter your documents based on their priority.

Open Metadata allows users to enable custom search and organize scenarios across heterogeneous sets of items.

Customize the Windows Vista Start Menu Search Box

marcmill — Wed, 12 Oct 2005 19:19:00 GMT

Internet Explorer has this feature that lets you add search commands to the address bar. For example, with some registry hacks you can make it so that typing "imdb Lord of the Rings" will do an IMDB search. The registry schema is described in more detail here and here and here and probably many other places...

Well, it turns out the search box in the Start menu for Windows Vista Beta 1 has a similar feature. So now you can just hit the Windows key, and then type (for example) "msdn IShellFolder" + <Enter> to do an MSDN search for IShellFolder. In Beta 1, the registry key for programming this is different from the one used for IE, but look for these two be unified in Beta 2. Care of my colleague Bret who owns the start menu search box -- one of my favorite Windows Vista features -- here are some that we've been using:

//
// Note, this schema will not be supported in Beta 2, instead
//  it will be unified with the IE schema for doing this.
//
[HKEY_CURRENT_USER\Software\Microsoft\Windows\
                 CurrentVersion\Explorer\StartPage\Alias]
"msdn"="http://search.microsoft.com/search/results.aspx?
                            qu=%s&View=msdn"
"msn"="http://search.msn.com/results.aspx?q=%s"
"stock"="http://moneycentral.msn.com/scripts/webquote.dll?Symbol=%s"
"urban"="http://www.urbandictionary.com/define.php?term=%s"
"imdb"="http://us.imdb.com/find?q=%s"
"wiki"="http://en.wikipedia.org/wiki/Special:Search?search=%s&go=Go"

Read Kurt Vonnegut's Blog

marcmill — Mon, 10 Oct 2005 06:20:00 GMT

I just finished reading Vonnegut's latest book, A Man Without a Country, a collection of short essays by this national treasure. On the back it says, "Thank God, Kurt Vonnegut has broken his promise that he will never write another book." Indeed!

It struck me that if you squint real hard, then this book is Kurt Vonnegut's blog. Or at least the closest thing he will ever get to having a blog. I mean, he's a hero of mine, but he really is a Luddite, in fact Vonnegut says he welcomes this description. In his words:

Electronic communities build nothing. You wind up with nothing. We are dancing animals. How beautiful it is to get up and go out and do something. We are here on Earth to fart around. Don't let anybody tell you any different.

Pretty funny since the last page of the book is an advertisment for artwork being sold at Vonnegut's web site. I can't wait until my "Goodbye Blue Monday" T-Shirt comes in the mail!

My other heros? Philip Glass, Alan Dershowitz, and Bill Gates. Of the four, Vonnegut is the only one I have yet to meet in person. Which reminds me, I still owe Raymond the story of my meeting with Glass.

PIDL Me This: The Explorer Storage Architecture

marcmill — Fri, 07 Oct 2005 01:22:00 GMT

The core Shell storage architecture has existed in some form since Windows 95. Since then, it's been evolving and will continue to as long as Windows is shipping. There are at least three notable places (that I can think of right now) where this storage platform is carried forward in Vista. First, more of the shell core functionality can now operate over non-filesystem places in the shell namespace, which is most evident in the new copy engine API (IFileOperation) and the new common file dialog API (IFileDialog and friends). Second, the addition of a rich shell namespace extension that can reason over the Windows search platform and abstractly distribute queries to multiple providers and scopes including remote queries to other machines. And finally, the addition of the shell property system which brokers property metadata for user interaction and multiplexes properties on items from their various providers.

I will almost certainly be blogging about these three and more in the future, so I'd like as a baseline to briefly discuss some of the high-level concepts of the shell storage architecture. This topic is discussed in greater detail elsewhere (here too), but I'm going use more modern terminology and paint only a conceptual and perhaps more idealistic picture than actually exists. After all, Explorer is both a platform and an application, and sometimes platform beauty has to succumb to the realities of shipping the application with the desired feature set.

The Namespace. The shell namespace is a tree of items, shell items to be precise. You can manipulate these items by using the IShellItem interface. The namespace is rooted at the root item, called the desktop folder. Shell items can have children that are other shell items. And you can enumerate the children of a shell item by binding to its IShellFolder handler and calling EnumObjects. You can ask a shell item for other services as well: you can bind to a stream on that item, you can read metadata for the item, you can get the icon or context menu on the item, you can enumerate and invoke verbs associated with the item, you can register for change notifications on the item, etc. You could build a whole shell-like application in fact by using this API set, and indeed the shell itself uses this API to deliver its user scenarios.

One of my favorite examples of a shell item is a ZIP archive file, because it is an example of an item that behaves both like a leaf node, in that it has streaming content, and a non-leaf node, in that it has child items which themselves can have children and streaming content, etc. In fact, not many people know this, but you can use the shell's namespace object model as a sort of native API for enumerating and extracting ZIP file content.

Junctions. The ZIP file is also an example of something called a junction, which is a place where one type of namespace handler delegates to another namespace handler by exposing an entrypoint to the other handler's namespace from within its own namespace. In the case of ZIP files, the file system namespace handler, which exposes a shell item namespace over the file system, junctions control to the ZIP namespace handler, which exposes a shell item namespace over the contents of a ZIP file.

Another example of a junction is the history folder. If you go into the history folder from the command prompt by saying:

> dir /a "%USERPROFILE%\Local Settings\History"
 Volume in drive C is CLAUDIA
 Volume Serial Number is 94C3-00F1

 Directory of C:\Documents and Settings\marcmill\Local Settings\History

08/10/2005  04:00 PM    <DIR>          .
08/10/2005  04:00 PM    <DIR>          ..
08/10/2005  04:00 PM               113 desktop.ini
10/06/2005  10:54 AM    <DIR>          History.IE5
               1 File(s)            113 bytes
               3 Dir(s)  25,113,985,024 bytes free

But if you view the same location in Explorer, you'll see something completely different:

That's because the History folder itself is another one of these junction points. And while the file system doesn't recognize the shell namespace junction points, Explorer does. That's what allows us to have content that appears only in the file system, in this case the history database itself, separated from the content that appears within Explorer.

Using this concept of junctions, the shell item namespace that ships with Windows provides extensibility points where third-party item namespace providers can plug in their own item namespaces. In fact, these same extensibility mechanisms are used internally to deliver features like the History folder, the control panel, and especially the new "virtual folders" feature in Windows Vista. In the case of virtual folders, the ".vfolder" files that you find in your profile directory are akin to ZIP files, in that they are junction points to a shell item namespace that operates over database query results. More on how this works in a later post.

File System Versus Item. Any shell item can be asked if it is really backed by a real file or folder on the file system, and if so, asked for that item's path. It is often tempting as a result to circumvent the shell storage abstraction by extracting the file system path from the shell item, and talking directly to the file system. After all, your code will work in the majority case where the underlying item is actually backed in some way by the file system. And in fact, sometimes this circumvention is required since the shell, for example, provides no abstraction over memory mapping the item's content from its storage.

However, when making a decision to only operate over file-backed items, you should be aware of the potential loss of generality. Do you want, for example, your thumbnail provider to be able to render thumbnails for items that appear inside ZIP files, or on storage devices that plug in to the shell namespace (but not the filesystem)? If you support the abstract storage model, then your code will have more reach.

More to come...

Fun Keyboard Shortcuts

marcmill — Mon, 03 Oct 2005 22:43:00 GMT

I like keyboard shortcuts that put focus in commonly-used edit boxes:

Ctrl+E will put focus in the "search" box in Explorer for Windows Vista Beta 1.
Alt+D will put focus in the address bar in IE.
F11 will put focus in the "find a contact" box in Outlook, this is very useful for quickly looking up someone’s phone number.
/ (forward slash) will put focus in the "search" box in gmail (too bad this doesn't work on Google's search results pages).

That last one follows a well-established UNIX convention. In fact, I think more arcane UNIX keyboarding conventions should be brought forward to modern non-terminal-based applications. That’s why the edit box in IE’s history search allows you to type CTRL+U in order to clear the input field. Some wag intern wired this one up.

MSN Search: Better Every Day

marcmill — Sat, 01 Oct 2005 21:38:00 GMT

People who know me would probably agree that I am hardly a stooge for marketing. And although I am in general very proud to work for Microsoft, there are definitely things about Microsoft that I’m ashamed of. But one thing that I’ve been especially impressed by recently has been the quality of search results that I’ve been getting back from MSN Search.

I’ve been trying to do all my searches over the past few months using both Google and MSN. Keeping in mind that search engine relevancy is highly subjective and a very personal thing, here is a summary of my impressions. A few months ago when Microsoft launched its MSN Search ad campaign just about every search that I did was better on Google. Over the months, MSN has been getting better and better, but something happened recently and now many of my searches are actually getting better results on MSN than on Google.

Some examples of searches from recent memory where MSN did better:

"Seattle Traffic" (Google) (MSN)
"Football Scores" (Google) (MSN)
"Google Headquarters" (Google) (MSN) -- this is a very funny example from the Microsoft Company Meeting.

Unfortunately, the following search yields much better results on Google than MSN:

"Marc Miller" (Google) (MSN)

COMmunism: Sharing your Apartment

marcmill — Thu, 29 Sep 2005 21:14:00 GMT

The advent of neutral COM objects introduces the interesting consequence that your thread might have to share an apartment. This means that if you assume all asynchronous calls into your thread are on the same apartment, you probably have a subtle bug. Here’s an example that I ran into recently:

My STA thread runs a WndProc which receives asynchronous posted messages WM_DOWORK. The WndProc is associated with a thread-affine object that holds pointers to other STA objects and in response to WM_DOWORK, marshals one of these objects into the stream and returns that stream to the asynchronous caller. This is a common pattern that I’ve seen many times.
That very same thread calls an API which eventually CoCreate’s a neutral object. The neutral object runs in a new apartment hosted on the same thread, a neutral apartment. While running, the neutral object decides to itself call CoCreate, which ends up pumping messages. The message pumping therefore happens in the neutral apartment.
During this message loop, my thread reenters and processes a WM_DOWORK and in response does its job of marshaling one of its objects into a stream by calling CoMarshalInterThreadInterfaceInStream. COM, when deciding how to marshal the interface, looks at the current call context and sees that the object is in the neutral apartment. As a result, the object is marshaled from the neutral apartment instead of the STA.
Later, when the object is unmarshaled from another thread, a transition happens into the neutral apartment on the same thread, which results in the object’s pointer being accidentally smuggled across threads.

If you’re lucky, then some visible problem (like a concurrency issue) will result from this sort of problem.

If you’re really lucky, it will repro somewhat consistently.

The main problem here is that, while the assumption under STA is that apartment implies thread, we cannot assert the consequent and assume that thread implies apartment. So, any asynchronous entry into the thread has to somehow ensure the correct COM context before making calls into apartment-affine objects.

Luckily, COM provides an API for doing just this. One solution to the problem described above, then, is:

When the WndProc is created, it remembers its COM context by calling CoGetCallContext for IContextCallback.
Later, when the WndProc is called, it calls IContextCallback::ContextCallback to get back into the correct COM context. Then it proceeds to dispatch messages.

Q: But Marc, this is crazy, no one calls this API today.
A: You’re right.

ML is ALSO Love

marcmill — Thu, 22 Sep 2005 05:42:00 GMT

(Don Box wrote an article in this month's MSDN Magazine called "Scheme Is Love" in which he did a brief whirlwind tour of the Scheme programming language.)

Back when gas was only $3 a gallon, if you wanted a programming language with anonymous types and type inferencing, you had to use one of them fancy languages like ML. I've actually written a few nontrivial programs in Standard ML and found the experience to be very enjoyable.

Static types without syntactic baggage. One of the major drawbacks of a dynamic language like Scheme is that the lack of strong types prevents the compiler from doing a certain class of work on your behalf - work you would instead have to do manually. But in exchange, you're saved some of the cognitive and syntactic load that most strongly typed languages have. In ML, you rarely "say" the name of a type in your code, so it reads like a dynamic language; but, because the compiler infers the types from usage, it can call you on any inconsistencies at compile time. ML has an interactive interpreter which prints out the names of the types that it infers.

- fun add3(x) = x + 3;
val add = fn : int -> int

In this case, the type inferencer is saying that "add3" is a function (fn) who maps (->) an int to an int. Now, if I try to misuse the add3 function by passing in the wrong type, the compiler throws an error:

- fun calladd3() = add3("spoon");
Error: operator and operand don't agree [tycon mismatch]
  operator domain: int
  operand:         string
  in expression:
    add3 "spoon"

Expressive type system. Like Scheme, ML has a very simple-looking type system that is surprisingly powerful. Unlike Scheme, however, the compiler can optimize the data structures based on the static type to have a very efficient runtime representation. Also, the gain from having static type checking becomes even more apparent when you start dealing with complex data structures. The result is you get very expressive types that are efficient and safe.

Tuples and Records: You can create anonymous records by just declaring them, and of course records can contain other records:

- val rec = { firstname="Marc", lastname="Miller", 
                   point = { x = 5, y = 7 } };
val rec = ... : {firstname:string, lastname:string, 
                    point:{x:int, y:int}}

You can access a value from within a record by using the #fieldname(objectname) syntax:

- val pt = #point(rec);
val pt = {x=5,y=7} : {x:int, y:int}

And tuples are simply records whose elements are unnamed:

- val tuple = ("marc", "miller", ( 5, 7 ));
val tuple = ("marc","miller",(5,7)) : string * string * (int * int)

Datatype: The "datatype" keyword can be used to create named enumerations, like so:

- datatype KnownColor = Red | Green | Blue;

- fun GetGreen() = Red;
val GetGreen = fn : unit -> KnownColor

But you can also use it to create variant types:

- datatype Color = Known  of KnownColor
                 | Custom of { red : int, green : int, blue : int };

- val blue = Known(Blue);
val blue = ... : Color

- val white = Custom({ red=0xf, green=0xf, blue=0xf });
val white = ... : Color

And datatypes, since they are named, can now contain members of their own type, which is very powerful because now you can create trees and linked lists:

datatype IntTree = IntNode of (IntTree * int * IntTree)
                 | Empty; 

- IntNode(IntNode(Empty, 1, Empty), 2, IntNode(Empty, 3, Empty));
val it = ... : IntTree

There is also support for parameterized types, like generics or templates. So instead of making an IntTree, I could have made a more generic tree:

datatype ('T) Tree = Node of ('T Tree * 'T * 'T Tree)
                   | Empty;

- Node(Node(Empty, 1, Empty), 2, Node(Empty, 3, Empty));
val it = ... : int Tree 

- Node(Node(Empty, "foo", Empty), "bar", Node(Empty, "snoo", Empty));
val it = ... : string Tree

And the type inferencer still polices everything, here I try to put a string into an int tree:

- Node(Node(Empty, "foo", Empty), 3, Empty);
Error: operator and operand don't agree [literal] ...

ML is "fun". ML's functions, like Scheme's, are first-class objects. This means they can be returned from other functions, stashed inside data structures, assigned to variables, etc. For example, I can make a function addn(n) that returns a function which will always add 'n' to the number passed into it:

- fun addn(n) = fn x => n + x;
val addn = fn : int -> (int -> int)

- val add5 = addn(5);
val add5 = fn : int -> int

- add5(4);
val it = 9 : int

Pattern matching on function arguments is another feature of ML's functions and will knock your socks off! I can write alternative versions of a functions implementation with patterns as arguments, and the pattern matcher will generate the logic to match the most specific one. For example, here's a version of factorial that uses patterns:

fun fact(0) = 1
  | fact(n) = n * fact(n - 1);

Pattern matching is a great way to pull a value out of a datatype; here I walk an "int Tree" data structure, adding the elements in a depth-first traversal:

fun addTree(Empty) = 0
  | addTree(Node(leftTree, data, rightTree)) = (data + 
                                                addTree(leftTree) +
                                                addTree(rightTree));

- addTree(Node(Node(Empty, 1, Empty), 2, Node(Empty, 3, Empty)));
val it = 6 : int

In fact, ML functions only take one parameter, but pattern matching over tuples gives the illusion of multiple parameters:

- fun add(x,y) = x + y;
val add = fn : int * int -> int

This really says: "add" is a function that takes one parameter which is the tuple of type 'int * int', where 'x' and 'y' are bound to the two ints inside that tuple.

Know your heritage. This brief tour only touches the tip of the iceberg of what's available in ML. As you can see, C# 3.0 draws on some of the syntax and concepts of this language.

Type inferencing.

var x = 3;   // C# 3.0

var x = 3;   (* ML *)

Anonymous types:

var x = new { FirstName = "Marc", LastName = "Miller" };  // C# 3.0

var x = { FirstName : "Marc", LastName : "Miller" };  (* ML *)

Lambda expressions:

var add5 = x => x + 5;     // C# 3.0

var add5 = fn x => x + 5;  (* ML *)

Custom Schema on the PDC05 Handout Build

marcmill — Mon, 19 Sep 2005 22:43:00 GMT

The API for adding custom schema isn't supported yet on the builds handed out at the PDC. So, how was I able to demonstrate custom schemas in my PRS310 talk? I've had a request to make public how I was able to go about doing this in lieu of the schema installation API.

Warning: The techniques I describe below are just temporary stopgaps and are not supported by Microsoft and will not work in future builds of Windows Vista or any other operating system. Also, I'm typing this from memory, so I apologize if there are inaccuracies:

Write a property description file describing your new properties, the one I used for recipe files is available here.
Exit Explorer; ctrl+shift+right-click on the background of the start menu, and choose "exit explorer".
Create a directory inside CSIDL_LOCAL_APPDATA called "Property Descriptions": mkdir "%LOCALAPPDATA%\Property Descriptions" should do the trick.

Put the propdesc file into that new directory, and then start up Explorer (use taskmgr). You will now be able to call the SHGetPropertyDescriptionXXX APIs to access your new properties. That means you can do what I did in PRS310 and add the canonical names for your properties to the infotip and preview pane property lists for your file types.

For example, here's the registry settings that I used for my recipe file:

[HKEY_CLASSES_ROOT\.recipe]
@="Recipe File"
"InfoTip"="prop:System.CanonicalType;System.Author;System.Rating;\
>>        MyCompany.Recipe.Difficulty;MyCompany.Recipe.Yield;\
>>        System.Comment"
"BestOf"="prop:System.DateChanged;System.Author;System.Keywords;\
>>         MyCompany.Recipe.Difficulty;MyCompany.Recipe.Yield;\
>>         MyCompany.Recipe.PreparationTime;MyCompany.Recipe.CookTime;\
>>         System.Rating;System.Comment;System.Size;System.DisplayFolder;\
>>         System.DateCreated"
"Title"="prop:System.Title;System.CanonicalType"

Note that the new schema will not be available in the column chooser. Of course, there's a hack here too: I hear that you can modify the AllFolderProperties registry value in HKCR\Folder. Again, you won't need todo this in Windows Vista Beta 2, with the schema installation API this will just happen as part of install.
Explorer will have access to your new properties, but they will not be accessible by search unless you install that new schema into the search index as well. There are a set of hacks for doing this as well, which involve changing the indexer's internal schema config files and then rebuilding the indexer. But these weren't part of my PDC talk PRS310.

There will be an API which will accomplish all these steps PLUS add the new schema to the search index available in Windows Vista Beta 2.

PDC05 Takeaways on Programming Explorer

marcmill — Sat, 17 Sep 2005 20:14:00 GMT

Flying back to Redmond, I jot down some of the key things I consistently heard from developers, in no particular order:

"We want more/better command module extensibility." The term "command module" here refers to that toolbar that goes across the top of the Explorer view and replaces the "folder tasks" area in XP.

In the current POR, the command module for Windows Vista actually will have a limited extensibility model, but some of the feedback that I received makes me think we’re not going far enough. Let me explain. In the Explorer architecture, the shell folder object being viewed tells the command module object which commands to display via an interface to be released in Beta-2. So a third-party could come along with its own shell namespace extension and then have complete control over the commands displayed when viewing that shell folder.

However, there is no way currently to extend the commands displayed by other shell folders. In particular, the most compelling shell folder you would want to party on would be the shell-folder-over-filesystem, or what shell developers call "file folder". It would be great, for example, for a third-party photo application to add commands like "remove red-eye".

Providing the correct extensibility model that we will want to live with for the next decade or more is a very challenging problem, however: How do we deal with competing and conflicting registration? How can we make it less attractive for unscrupulous ISV’s to spam this very valuable screen real-estate? So I think its understandable that we’ve been reluctant to commit to solving this extensibility problem in the most general way, but on the other hand there could be a tremendous amount of end-user value-add to enabling some better extensibility story in this space.

It’s much more likely IMHO that instead individual command implementations will provide their own extensibility points. Imagine (for example) the "Show slideshow" command allowing third-party slideshow viewers, etc.
"We want to be able to write Explorer extensions in managed code." This is complicated. The short answer is "you can", the medium-sized answer is "don’t do it!" and the long answer is "you can, if you try really hard".

The API for Windows Explorer consists of two major chunks. First is the set of reusable functionality that is provided for use in your own applications. Examples of this would be the Common File Dialog or the soon-to-be-documented ExplorerBrowser control, which indeed let you host an entire explorer "instance" inside your application. The second set of API consists of extensibility points which you can install in order to extend or customize the behavior of explorer in various ways. Examples of this include context menus, namespace extensions, property and tooltip handlers, etc.

There is no technical or philosophical reason that I know of why you can’t consume the first set of API from managed code, and indeed WinFx has several places where shell functionality is called, like SHGetFolderPath, GetSaveFileName, etc.

No, it’s that second set of API, the shell extensions, which are tricky. I wasn’t able to attend his talk, but I understand that the very articulate Jim Miller gave a talk about this very subject. Briefly, the problem is rooted in the fact that only one CLR can run in a process and, while CLRs are tested for backward compatibility, they are not of course guaranteed to be forward-compatible. So now imagine that your managed shell extension is loaded in some third-party process that is already locked-back to an older version of the CLR than what you targeted. You clearly don’t want this to happen.

My best guess is the explorer.exe process itself will avoid this issue by shipping with a startup call to the EE which tells it to always load the latest CLR version, but of course we can’t control other people’s processes and by the time they call that first set of APIs to, say host the common file dialog, then its too late to ensure a later version of the CLR than what’s already loaded.

One option that I hesitate to mention because its really quite difficult – much more so than it probably sounds – is to implement your extension point in managed code but register it such that it gets instantiated out-of-process. There are a number of tricks that you can play, depending on circumstances like the structure of your underlying data source, to mitigate the performance problems inherent in the chatty cross-process remoting that you would get with such an approach. An example of this would be a small native-side proxy which has in-process caching of your data and services the chatty calls, and then behind-the-scenes does more "chunky" calls back to the data source.
"We want a better migration path to Windows Vista Common File Dialog". Windows Vista is deprecating the old common file dialog API in favor of a long-overdue COM API complete with callback sink objects, the ability to do metadata-stamping (and therefore the ability for the user to save into a virtual folder!), the ability to return item object that are part of the extended shell namespace – not just the file system, etc. It also includes a much more formal declarative customization model and removes support for the old-style template-driven customization.

I sincerely believe that the direction this new API takes is goodness, since it enables a lot of really cool end-user features, but some feedback I’ve heard is that perhaps it asks ISVs to bite off a little more than they’re willing to chew for this first release. Under the current plan, ISVs are faced with a dilemma: either don’t support the new API and therefore deliver a more-limited-then-my-competitor’s-application experience, or support the new API through two vastly-divergent code paths: one for downlevel and the other for Windows Vista.

One thing that might be possible, for example, is to plumb parts of the new API through the old API – thus allowing some of the features like saving to virtual folders, but degrading gracefully in a downlevel experience.
"We want Microsoft to provide the forum for community schema coordination." In my PRS310 talk I spoke about two different types of properties: those provided by the system and those provided by ISVs. The system ones have the naming convention System.Groupname.PropertyName like System.Media.FNumber, and the ISV properties have the form ISVName.ApplicationName.PropertyName.

The problem is that this naming convention implies no coordination among ISV schema since each application gets its own namespace of properties. Will experiences with heterogeneous data sets therefore be suboptimal for like documents that would otherwise share common non-system properties? This question was asked in a much more understandable way by someone in the audience: "will two different CAD document types created by two different companies ever be able to share common non-system but CAD-specific properties?" Clearly, in order to pull this off you need some consensus on the schema, and the ask here is for Microsoft to set the table at which others can sit down and negotiate schema. Perhaps we can bring them drinks. I mean, Microsoft doesn’t necessarily have to be the moderator of such a community forum, but could at least provide the infrastructure for the community to make decisions on its own. Maybe even, as a coworker suggested, ratification by the community of schema could result in that schema shipping in the next update or version of Windows, or at least the universal search engine redist.
"We would be interested in being able to host the reading pane in our own applications." This one is pretty self-explanatory; I was just surprised that I heard this piece of feedback from several people independently.

Anything I missed? [:)]

Will I be able to read my new C# Code?

marcmill — Thu, 15 Sep 2005 19:02:00 GMT

As a fan of languages like ML, I'm really excited about the new features in C# 3.0 that have been announced in this PDC. But one thing that always concerned me about great swathes of ML code was the readability of that code.

If I’m working in a function and I see, for example:

    var x = f();

then what other kinds of operations can I do with x? It turns out the type-inferencer is much smarter than I am!

In ML this is mitigated partially by the REPL (read-eval-print loop) which allows me interactively ask the compiler what the type of things are. But C# has no analog to this, AFAIK.

I was discussing this issue with a colleague today and I said, "I’m scared about having to read the C# 3.0 code that my coworkers write." He said, "I’m scared about other people having to read the C# 3.0 code that I write!"

How do I do that with my file types?

marcmill — Wed, 14 Sep 2005 07:10:00 GMT

In his PDC keynote demo, Chris Capossela showed off some of Windows Vista's exciting new features for managing the ever-growing sets of data that people will have with PC's moving forward. He showed visualization features like thumbnails, search features like stacking over properties, grouping by properties, filtering with the search box, and also organizing features like property painting and editing. But all of the file types involved were types well-known to Microsoft, and all of the schema pre-baked into the box at ship time.

Come to PRS310 Wednesday at 3:15p.m. to learn how your applications can play in the property system. (Don't worry Anders's talk has a repeat scheduled for the next day [:)].)

Learn More about Programming Windows Vista Search & Organize on Channel 9

marcmill — Tue, 13 Sep 2005 20:20:00 GMT

A couple of weeks ago, a couple of my colleagues and I sat down with Robert Scoble and Catherine Heller to talk about Programming Windows Vista Search and Organize. That interview was posted this morning and you can find it here.

Indexer Internals: What is the System Schema?

marcmill — Mon, 12 Sep 2005 23:41:00 GMT

When considering whether to extend the system schema, you should consider whether your extensions are actually necessary given the hundreds of properties that already ship with Windows Vista.

You can view the schema as the indexer understands it by looking at the internal indexer schema configuration file in:

%ALLUSERSPROFILE%\appdata\roaming\microsoft\usearch\data\config\schema.txt

Note that the slashes in the property names, i.e. System/Author, will change to dots in coming releases -- so it will be System.Author instead.

The term Schema Proliferation refers to adding unnecessary schema. Schema proliferation leads to bad user experiences, for example stacking in heterogenous views. Let me elaborate: Imagine if FooCorp comes along and introduces their own idea of Author called the FooAuthor. This works okay for scenarios in which I'm only dealing with .foo files. I can stack by FooAuthor in my views, I can query for documents whose FooAuthor is Fred, etc.

But the model breaks down when I am looking at views that contain files of different types other than .foo, because all the other types use System.Author for their author fields and have never heard of FooAuthor. So if I want to assemble a view of all the .doc, .ppt, AND .foo files authored by Fred, I can't do this because the .foo files don't expose an Author property.

Schema proliferation requires a special amount of discipline to avoid. while it's obvious in my FooCorp example, it's not quite as clear in the example of, say, .pgn files (chess game files) in which you could argue that the players of the game are the defacto authors of the file. Or the "chef" of a recipe file is also kind-of the author.

PDC05 Fashion

marcmill — Mon, 12 Sep 2005 07:26:00 GMT

I once attended a talk given by Paul Allen in which he described how he would routinely visit with Microsoft's most important customers wearing a t-shirt and some shorts. I remember first coming to Microsoft as an intern and standing in line at the cafeteria behind my mentor, looking down I noticed he was barefoot! And of course there are those days when you jump out of bed and don't feel like changing out of your pajamas...

Being "fashion challenged" is a deep and significant part of the Microsoft developer culture. But it's a part of the culture you won't be exposed to at PDC05. Speakers and staff are required to wear this year official attire: the blue Microsoft event button-down shirt -- 65% polyester and 35% cotton -- and either khaki or black pants.

So when you see a Microsoft representative suffering in the California heat underneath all that crinkly suffocating polyester, stiff pants, and uncomfortable shoes, realize that this person is making for you the customer the ultimate sacrifice: the sacrifice of dressing up!

Now that's dedication!

Then, if you want to get to know the real person try to imagine someone equally passionate about the technology only they're half-shaved, wearing a beer-stained t-shirt, jean shorts, and sneakers.