Engineering Windows 7

Welcome to our blog dedicated to the engineering of Microsoft Windows 7

Federating Windows Search with Enterprise Data Sources

Federating Windows Search with Enterprise Data Sources

  • Comments 60

The Windows Explorer has evolved by enabling you to find all sorts of content by searching for it. Many of you have used the search features in Windows Vista (based on our instrumented data) from the start menu or from the search box in Explorer. It has been a long time since most of us could remember where everything is by carefully managing our folder hierarchy and finding things based on file name alone.  We often rely on domain specific search (in music players, mail clients, photo clients) but with Windows Vista and Windows 7 we make it possible to search within a namespace and across namespaces.  This post is about a new feature based on Search that allows searching across PCs and even servers in an Enterprise setting.  Alwin and Scott, program managers, and Brandon, a developer, on the “Find and Organize” feature team authored this post. --Steven

Finding your stuff

Whether you’re searching or browsing, Windows Explorer is really about finding your stuff, and once you’ve found it, doing something with it (such as copying, opening, deleting, etc). For data that lives on your PC or home network, Windows 7 has invested in HomeGroup and Libraries (subjects for a future posting from our team) to provide an easier and richer experience than ever before. However, we didn’t stop there. Over the last few years, we’ve seen enterprise customers’ important content migrate towards (or aggregated in) centralized content stores, such as SharePoint. These products typically provide great features for team collaboration, document versioning and workflow management, archiving, retention policy enforcement, and other centrally-managed functionality that IT managers appreciate.

Where Enterprise Data Lives

Important enterprise data is found on local machines, in a variety of centralized content stores and also beyond the firewall

Unfortunately, this has placed an extra burden on customers to learn each new content store’s user interface, often asking them to give up familiar desktop features like drag-and-drop. Given their collaborative focus, these sites grow organically and it can become hard to remember where a particular document was stored and then wade through long lists of them every time you want to get back to it. Enterprise customers have asked us for a solution that simplifies finding important content in these various data stores but without leaving their normal Windows work flows.

As we looked at this trend and the lack of integration with content management and content indexing web services, we used these guiding principles in developing a solution:

  • Natural for people to use. Customers want a more consistent experience for finding and working with data in these disparate content stores, and would like us to bridge the local and remote content experiences by helping them “roll over” from one to the other.
  • Easy for IT admins to deploy. IT admins don’t like to deploy code, and want low-maintenance solutions that are easy to manage. Meanwhile customers want to connect up these sources without going through long and tedious installation processes or having to get help every time they want to set up a new search location.
  • Easy for developers to adopt. Developers want to enable this functionality in their offerings quickly and easily. There are a lot of data sources which need to be supported because IT folks don’t want to be locked in to a specific server technology.

Choosing to build Federated Search

Federated Search wasn’t the only way to address these challenges. The brute force approach would have been to take our existing Windows Search indexing technology and just use it on these content stores—that index the remote content on a local PC. This isn’t a very realistic solution since it’s inefficient to have all content indexed over the network by each person’s machine, especially when the content is changing at a rapid pace and represents a large corpus. Corporate retention policies may also prevent keeping even a local index of certain sensitive data.

Fortunately, there’s a better option – Federated Search. Federated Search enables you to search a remote web service from Windows explorer and get results back that you can act on like any normal file. The largest barrier to doing Federated Search has already been taken care of too. That is, most of these content stores are already indexed on the server, or at least on some server. There are several great offerings that will accomplish this, such as Microsoft Search Server. Not only do these servers index this content, but many of them already expose search results via a standard web protocol. This is largely thanks to the prevalence of OpenSearch and RSS enabled clients (including Internet Explorer and Microsoft Search Server, among many others).

For Windows 7, we’ve added support for Federated Search using OpenSearch v1.1 and worked to make the experience a seamless one. We found this solution strikes a good balance by leveraging the strengths of content services and the strengths of local file interactions within Windows.

Natural to use

Using Windows Explorer, people are familiar with several important user interface and interaction elements. They know how to use the navigation pane to change what they’re looking at. They know how to scroll around, how to select an item (or several), and they know how to double-click to open them. Most people know how to right-click for context-sensitive options related to their selection, or how to find those options presented in the command bar. They know they can drag and drop items to move them around. They know how to change view modes. We hope that they know how to search their current location using the search box, and in Windows 7 we think we’ve made it much easier to discover and use the Preview Pane to make sure they’ve got the right result.

Searching Sharepoint from Windows Explorer in Windows 7

Searching a SharePoint site using the new Federated Search support in Windows Explorer

Much of the usefulness of building Federated Search into Explorer is our ability to take advantage of this knowledge and familiarity. This may seem obvious once you see it in action, but behind the scenes there’s quite a lot going on to make all of this happen. For example, some applications such as Microsoft Word already know how to work with web URLs. So opening a Word document from a web server is fairly straightforward. But the majority of applications you’ll encounter really only understand how to open files on the local machine or via standard network file sharing protocols. This includes everything from the built-in software like Notepad and Paint, to third-party software like Photoshop or iTunes.

To handle this case, we implemented a “just in time” download solution, which will download the file to the internet cache before opening an application or taking actions (like using the SendTo menu) which require local files. This lets us offer searches that are very “lightweight” from a server load perspective, where we display metadata and icons or thumbnails without ever requesting the actual file. Then if you take an action like previewing or opening an item, we will do some behind-the-scenes work to make a local copy of the file only if necessary.

That enables us to work with the existing application ecosystem without asking anything of developers. However, applications can also take steps to offer even better functionality in many cases. For example, Windows Photo Viewer has added support for non-file items. So if you open a picture result in the built-in photo viewer, it’s the photo viewer that downloads the item, not Explorer. This may not seem like a big deal, but it lets the photo viewer enable the forward and back buttons to jump to the next or previous result – and it will download that image on-demand. Starting at the PDC we began reaching out to third-party ISVs to encourage them to implement similar enhancements for Federated Search scenarios, and we will continue to offer guidance on how to best integrate with all of the newest Explorer features.

Finally, we support all the standard clipboard and drag-and-drop operations. So if you drag a Word document from a Federated Search query onto your desktop, it will be copied there. You’ll even see the familiar Windows Explorer copy dialog, with progress indication, cancel ability, conflict resolution, and so on.

But wait, there’s more! Windows Explorer is a great tool that many customers know and love. But some people use it without even knowing it. Countless Windows applications make use of what we call the Common File Dialog. This is a special Explorer window that lets you find and choose items to be opened or inserted into your current application, without ever leaving it. If you’ve ever clicked File and then Open or Save in an application menu, you’ve probably seen some version of this dialog. PowerPoint, for example, uses the common file dialog to insert pictures. That means from inside PowerPoint you can click Insert Picture, select the Federated Search link for your image repository, search for the picture you want, and then insert it directly into PowerPoint. This works for any existing application that supports the Common File Dialog, and there are a whole lot of them!

Inserting a picture into PowerPoint’s using Federated Search

Inserting a picture into PowerPoint’s using Federated Search

Our Federated Search solution is all about simple lightweight access with a common, familiar user interface. This has a lot of benefits as we described above, but there are also cases where a server’s web interface will offer its own benefits. This might involve advanced query building, browsing, or server work-flow tasks, for example. So Windows 7 builds a bridge to these content repositories. After doing a search against a supported location, you will see a “Search on Website” button in the command bar which allows you to seamlessly send the query up to the service’s web interface in the default web browser. You’ll also see the “Open File Location” menu item when you right-click on a search result. Selecting that option will launch the web browser to the specific location in the document repository where the file is stored.

This seamless integration of Federated Search within Windows allows customers to greatly simplify their workflow for getting at remote files while still being able to easily take advantages of the advanced functionality of content repositories.

Simple to deploy

Our next challenge was to make it easy for customers to get these new connections onto their machines. It wouldn’t be practical to ship Windows with a connection to every solution in the world, so we shifted to a way that would make it very easy for any web service to deploy a connection to their specific service.

The model we came up with is similar to the way you add favorites from the web today. A web service can place a link to an .osdx file somewhere on their web page (see Channel 9’s search page for an example). The .osdx file is a simple XML file that uses the OpenSearch description document format to describe how to connect to the web service, and gives the web service some control of how the data is presented in Windows Explorer. When a person clicks on the link, Windows performs an ultra-lightweight install process that adds a search connector to that web service and places a link to that it in the Windows Explorer favorites.

If you are an administrator in an enterprise environment, you will likely want to provide some pre-installed search connectors for your users to search the company intranet or a popular internal SharePoint site for example. You can do this by deploying the search connector (.searchconnector-ms) files to your users’ machines via typical deployment techniques such as imaging, group policy preferences or startup scripts. The beauty is that it’s just a simple XML configuration file and there’s no code that needs to get installed on their machines. It’s also possible to pin one of these as a link from the Start menu through group policy. In the group policy editor look for the policy in this area: User Configuration> Administrative Templates > Windows Components > Windows Explorer. The policy name is “Pin Libraries or Search connectors to Search again links and start menu”.

Launching a Federated Search of an enterprise Intranet from the Start Menu

Launching a Federated Search of an enterprise Intranet from the Start Menu

Easy to adopt

Of course this technology depends on having services that support it. Although there are only a few services that provide a .osdx for you today, there are many existing services that already support the basic OpenSearch requirements.

We’re already seeing positive initial reactions from enthusiasts and ISVs alike echoing that it is indeed easy to enable your service to work with our Federated Search platform. If you’re a developer and want to enable an existing web based service to support Windows 7 Federated Search, you’ll need to provide a web service that accepts an http GET request with the search terms embedded somewhere in the URL and be able to return the results as an RSS or Atom feed. These requirements are typically very easy to meet for most applications that already provide search services via a web browser.

Your web service results should include the basic RSS tags like <link>, <title>, <description>, <pubDate> to get started but there’s much more that you can include in the RSS output and customization you can do within the .osdx file to enhance the experience for the end user.

For more information, we’ve published the Windows 7 Federated Search implementer’s guide with detailed information on how to enable your data source to work with Windows Federated Search. There’s also a recorded PDC session that demonstrates how to build a Windows Federated Search compatible web service for an existing SQL database.

- Brandon Paddock, Scott Dart & Alwin Vyhmeister, Find and Organize

Leave a Comment
  • Please add 2 and 7 and type the answer here:
  • Post
  • I love search 4.0 in Windows Vista

    and  I love team find and organize.

    AWESOME WORK!!!

  • Where there is a lot of power there, I really do not like the new float trees on the left.  In Vista the fav links were locked with a "More" button to display more of them and the divider was able to be moved to whatever portion you wanted the tree to take up.

    With the new model you are always scrolling up and down to get to the other trees.  If I am down browsing the drive and want to switch to a Fav Link, I have to scroll all the way up to get to them.  Also, if I want a good number of fav links, then I have to scroll down to get to anything else.  

    With the current model it would be better to have them in a tab display in that pane to make it "one click" to get to your desired tree instead of scroll all the time to get there.  This scrolling seems to defeat the usefulness of the Fav Lins.

    Just a little feedback, not to be viewed as offensive as I really love Vista!  (Yes, I may be one of the few dozen that have a brain and do like it ;) )

    Rocky

  • "It has been a long time since most of us could remember where everything is by carefully managing our folder hierarchy and finding things based on file name alone"

    Well, you're certainly not making it easier by moving all our folders around with every Windows release... And having the jungle of "access denied" links under \Users and in your profile root folder doesn't help. Or the three different folders under AppData that applications seem to choose between at random... Some of us still try to actually *know* what's on our harddrives though. Unfortunately, that means moving everything away from the user profile folder Windows creates, and then uses as garbage dump.

    So yeah, I can see how search is useful under those circumstances.... :)

  • Cool blog, but please post full-resolution images. Clicking on an image for it only to load an image of the same size is quite disappointing.

  • <Rant>Irrelevant to this, but it would be nice if you followed some Snow Leopard innovations such as REDUCING the HDD size of each app, as well as its RAM usage! This would be supremely useful to all users, especially those on Notebooks! I realize this would probably require some radical recoding, but honestly, this is a hit you are going to have to take as flash new features (even as cool as they are) are simply not ground for a AU$200+ purchase price. The fundamental framework of your applications needs to be improved, rather than just functionally and aesthetically. Don't get me wrong, the improvements that you have made to this OS since Vista are phenomenal, but I can't help but feel that everything running faster and (for lack of a better word) better would be make the cake that much more delicious!</Rant>

    In regards to the new Search functionality, nice work, I think this will be a big help to business users, might I ask if it can be set up on a Home Lan as well? Because being able to search across my 3 PCs at home would be invaluable.

    - AeonSlayer / Simon

  • @AeonSlayer / Simon

    We have blogged quite a bit about disk footprint which is something we worked on a bunch.  Looking back at those posts you can see that the footprint is not consumed so much by the "apps" so that's not where we put a ton of our focus.  We did work on many aspects of this though and the feedback on 16GB SSD installations has been fairly consistent in this regard.  

    To be fair, the competitor you mentioned apparently gained a lot of footprint by removing the support for a whole class of processor.  It is worth doing a comparison for some of the issues raised for us as well, since one might find comparable disk footprint (for example printer drivers).  

    This is an area we pay close attention to and are always trying to balance consuming the least amount of space with recognizing that the vast majority of our customers obtain Windows as an image on a PC and don't have access to a DVD for installation so having drivers and hardware support on disk is important for recovery scenarios.  

    As far as RAM, this is of course another area we worked very hard on.  The feedback we've seen here reflects the work we've done I think.

    --Steven

  • @Steven: Fair point, and I do indeed find that Windows 7 runs much better than Vista on my Notebook, and with less install size. I simply hope that you guys will do all you can to maximize the performance of this OS.

  • MS should provide the cancelled "Live Search Center"/OneView/Casino for XP and Vista users. 7 has so many features that were postponed/cancelled for Vista.

  • Steve,

    I never tried W7, but with the previous win releases there was never a option to exclude every driver from the installation that wasn't used. Why not now (or there is)?

    I wish there was a more advanced installation option (maybe there is and i am not aware of it) to define in more detail what u want or not with a installation.

    reg,

    S.

  • ++What Jalf said!

    I play PC games like COD4 and Armed Assault, which have a lot of user made mods, and trying to work out which folder to place those mods and maps in under Vista 64 is a real PITA, because of the folder virtualisation and so on.

  • Nicely done so far,here i have a suggestion to you.Why not make a virtual system where all running programs will run in virtual environment.or make a virtual user account where all program run in virtual mode,such as when i am surfing if some malware attack my system it will only affect the virtual environment.i can reset the virtual environment and the malware gone for ever.

  • Search in the Intranet how cool is that?!!!

    If that will realy work so great like search on a local machine....then 7 has one more "WoW" Feature...love Index Searching...but with external ressources it will be extremely great...will you add function to search trought files in shared folders of a network?

  • Is it possible to create an OSDX (search federation) for Google web and image search? AFAIK, Google doesn't directly support RSS feeds for search results.

  • Excuse me OT.

    @Windows Team

    I tried to scratch a DVD video from side to side , I inserted the DVD video on the PC with Windows 7

    MAGIC!!! WORK

    I inserted the same DVD in OSX (not mine :D )

    and NOT WORK.

    possible?

  • Looks like they're hoping to win over the enterprise XP crowd that held out on Vista.  I think if it runs on XP-era hardware and has tangible productivity improvements like this one, 7 will be a winner in the enterprise.  In fact I'll bet on it "cleaning the floor" in the enterprise while maintaining a healthy majority on home computers.

Page 1 of 4 (60 items) 1234