Engineering Windows 7

Welcome to our blog dedicated to the engineering of Microsoft Windows 7

Federating Windows Search with Enterprise Data Sources

Federating Windows Search with Enterprise Data Sources

  • Comments 60

The Windows Explorer has evolved by enabling you to find all sorts of content by searching for it. Many of you have used the search features in Windows Vista (based on our instrumented data) from the start menu or from the search box in Explorer. It has been a long time since most of us could remember where everything is by carefully managing our folder hierarchy and finding things based on file name alone.  We often rely on domain specific search (in music players, mail clients, photo clients) but with Windows Vista and Windows 7 we make it possible to search within a namespace and across namespaces.  This post is about a new feature based on Search that allows searching across PCs and even servers in an Enterprise setting.  Alwin and Scott, program managers, and Brandon, a developer, on the “Find and Organize” feature team authored this post. --Steven

Finding your stuff

Whether you’re searching or browsing, Windows Explorer is really about finding your stuff, and once you’ve found it, doing something with it (such as copying, opening, deleting, etc). For data that lives on your PC or home network, Windows 7 has invested in HomeGroup and Libraries (subjects for a future posting from our team) to provide an easier and richer experience than ever before. However, we didn’t stop there. Over the last few years, we’ve seen enterprise customers’ important content migrate towards (or aggregated in) centralized content stores, such as SharePoint. These products typically provide great features for team collaboration, document versioning and workflow management, archiving, retention policy enforcement, and other centrally-managed functionality that IT managers appreciate.

Where Enterprise Data Lives

Important enterprise data is found on local machines, in a variety of centralized content stores and also beyond the firewall

Unfortunately, this has placed an extra burden on customers to learn each new content store’s user interface, often asking them to give up familiar desktop features like drag-and-drop. Given their collaborative focus, these sites grow organically and it can become hard to remember where a particular document was stored and then wade through long lists of them every time you want to get back to it. Enterprise customers have asked us for a solution that simplifies finding important content in these various data stores but without leaving their normal Windows work flows.

As we looked at this trend and the lack of integration with content management and content indexing web services, we used these guiding principles in developing a solution:

  • Natural for people to use. Customers want a more consistent experience for finding and working with data in these disparate content stores, and would like us to bridge the local and remote content experiences by helping them “roll over” from one to the other.
  • Easy for IT admins to deploy. IT admins don’t like to deploy code, and want low-maintenance solutions that are easy to manage. Meanwhile customers want to connect up these sources without going through long and tedious installation processes or having to get help every time they want to set up a new search location.
  • Easy for developers to adopt. Developers want to enable this functionality in their offerings quickly and easily. There are a lot of data sources which need to be supported because IT folks don’t want to be locked in to a specific server technology.

Choosing to build Federated Search

Federated Search wasn’t the only way to address these challenges. The brute force approach would have been to take our existing Windows Search indexing technology and just use it on these content stores—that index the remote content on a local PC. This isn’t a very realistic solution since it’s inefficient to have all content indexed over the network by each person’s machine, especially when the content is changing at a rapid pace and represents a large corpus. Corporate retention policies may also prevent keeping even a local index of certain sensitive data.

Fortunately, there’s a better option – Federated Search. Federated Search enables you to search a remote web service from Windows explorer and get results back that you can act on like any normal file. The largest barrier to doing Federated Search has already been taken care of too. That is, most of these content stores are already indexed on the server, or at least on some server. There are several great offerings that will accomplish this, such as Microsoft Search Server. Not only do these servers index this content, but many of them already expose search results via a standard web protocol. This is largely thanks to the prevalence of OpenSearch and RSS enabled clients (including Internet Explorer and Microsoft Search Server, among many others).

For Windows 7, we’ve added support for Federated Search using OpenSearch v1.1 and worked to make the experience a seamless one. We found this solution strikes a good balance by leveraging the strengths of content services and the strengths of local file interactions within Windows.

Natural to use

Using Windows Explorer, people are familiar with several important user interface and interaction elements. They know how to use the navigation pane to change what they’re looking at. They know how to scroll around, how to select an item (or several), and they know how to double-click to open them. Most people know how to right-click for context-sensitive options related to their selection, or how to find those options presented in the command bar. They know they can drag and drop items to move them around. They know how to change view modes. We hope that they know how to search their current location using the search box, and in Windows 7 we think we’ve made it much easier to discover and use the Preview Pane to make sure they’ve got the right result.

Searching Sharepoint from Windows Explorer in Windows 7

Searching a SharePoint site using the new Federated Search support in Windows Explorer

Much of the usefulness of building Federated Search into Explorer is our ability to take advantage of this knowledge and familiarity. This may seem obvious once you see it in action, but behind the scenes there’s quite a lot going on to make all of this happen. For example, some applications such as Microsoft Word already know how to work with web URLs. So opening a Word document from a web server is fairly straightforward. But the majority of applications you’ll encounter really only understand how to open files on the local machine or via standard network file sharing protocols. This includes everything from the built-in software like Notepad and Paint, to third-party software like Photoshop or iTunes.

To handle this case, we implemented a “just in time” download solution, which will download the file to the internet cache before opening an application or taking actions (like using the SendTo menu) which require local files. This lets us offer searches that are very “lightweight” from a server load perspective, where we display metadata and icons or thumbnails without ever requesting the actual file. Then if you take an action like previewing or opening an item, we will do some behind-the-scenes work to make a local copy of the file only if necessary.

That enables us to work with the existing application ecosystem without asking anything of developers. However, applications can also take steps to offer even better functionality in many cases. For example, Windows Photo Viewer has added support for non-file items. So if you open a picture result in the built-in photo viewer, it’s the photo viewer that downloads the item, not Explorer. This may not seem like a big deal, but it lets the photo viewer enable the forward and back buttons to jump to the next or previous result – and it will download that image on-demand. Starting at the PDC we began reaching out to third-party ISVs to encourage them to implement similar enhancements for Federated Search scenarios, and we will continue to offer guidance on how to best integrate with all of the newest Explorer features.

Finally, we support all the standard clipboard and drag-and-drop operations. So if you drag a Word document from a Federated Search query onto your desktop, it will be copied there. You’ll even see the familiar Windows Explorer copy dialog, with progress indication, cancel ability, conflict resolution, and so on.

But wait, there’s more! Windows Explorer is a great tool that many customers know and love. But some people use it without even knowing it. Countless Windows applications make use of what we call the Common File Dialog. This is a special Explorer window that lets you find and choose items to be opened or inserted into your current application, without ever leaving it. If you’ve ever clicked File and then Open or Save in an application menu, you’ve probably seen some version of this dialog. PowerPoint, for example, uses the common file dialog to insert pictures. That means from inside PowerPoint you can click Insert Picture, select the Federated Search link for your image repository, search for the picture you want, and then insert it directly into PowerPoint. This works for any existing application that supports the Common File Dialog, and there are a whole lot of them!

Inserting a picture into PowerPoint’s using Federated Search

Inserting a picture into PowerPoint’s using Federated Search

Our Federated Search solution is all about simple lightweight access with a common, familiar user interface. This has a lot of benefits as we described above, but there are also cases where a server’s web interface will offer its own benefits. This might involve advanced query building, browsing, or server work-flow tasks, for example. So Windows 7 builds a bridge to these content repositories. After doing a search against a supported location, you will see a “Search on Website” button in the command bar which allows you to seamlessly send the query up to the service’s web interface in the default web browser. You’ll also see the “Open File Location” menu item when you right-click on a search result. Selecting that option will launch the web browser to the specific location in the document repository where the file is stored.

This seamless integration of Federated Search within Windows allows customers to greatly simplify their workflow for getting at remote files while still being able to easily take advantages of the advanced functionality of content repositories.

Simple to deploy

Our next challenge was to make it easy for customers to get these new connections onto their machines. It wouldn’t be practical to ship Windows with a connection to every solution in the world, so we shifted to a way that would make it very easy for any web service to deploy a connection to their specific service.

The model we came up with is similar to the way you add favorites from the web today. A web service can place a link to an .osdx file somewhere on their web page (see Channel 9’s search page for an example). The .osdx file is a simple XML file that uses the OpenSearch description document format to describe how to connect to the web service, and gives the web service some control of how the data is presented in Windows Explorer. When a person clicks on the link, Windows performs an ultra-lightweight install process that adds a search connector to that web service and places a link to that it in the Windows Explorer favorites.

If you are an administrator in an enterprise environment, you will likely want to provide some pre-installed search connectors for your users to search the company intranet or a popular internal SharePoint site for example. You can do this by deploying the search connector (.searchconnector-ms) files to your users’ machines via typical deployment techniques such as imaging, group policy preferences or startup scripts. The beauty is that it’s just a simple XML configuration file and there’s no code that needs to get installed on their machines. It’s also possible to pin one of these as a link from the Start menu through group policy. In the group policy editor look for the policy in this area: User Configuration> Administrative Templates > Windows Components > Windows Explorer. The policy name is “Pin Libraries or Search connectors to Search again links and start menu”.

Launching a Federated Search of an enterprise Intranet from the Start Menu

Launching a Federated Search of an enterprise Intranet from the Start Menu

Easy to adopt

Of course this technology depends on having services that support it. Although there are only a few services that provide a .osdx for you today, there are many existing services that already support the basic OpenSearch requirements.

We’re already seeing positive initial reactions from enthusiasts and ISVs alike echoing that it is indeed easy to enable your service to work with our Federated Search platform. If you’re a developer and want to enable an existing web based service to support Windows 7 Federated Search, you’ll need to provide a web service that accepts an http GET request with the search terms embedded somewhere in the URL and be able to return the results as an RSS or Atom feed. These requirements are typically very easy to meet for most applications that already provide search services via a web browser.

Your web service results should include the basic RSS tags like <link>, <title>, <description>, <pubDate> to get started but there’s much more that you can include in the RSS output and customization you can do within the .osdx file to enhance the experience for the end user.

For more information, we’ve published the Windows 7 Federated Search implementer’s guide with detailed information on how to enable your data source to work with Windows Federated Search. There’s also a recorded PDC session that demonstrates how to build a Windows Federated Search compatible web service for an existing SQL database.

- Brandon Paddock, Scott Dart & Alwin Vyhmeister, Find and Organize

Leave a Comment
  • Please add 6 and 5 and type the answer here:
  • Post
  • Well, I've been using Win7 for about a month now and I'm generally pleased with the many improvements it has over the really frustrating Vista.  However, the Search engine is still quite frustrating for me.  I've posted similar comments in other posts and haven't had a real acceptable answer, just excuses.  This issue has also been addressed more than once by other users.  The issue is about the search engine failing to find files, right there in the current folder, because they aren’t indexed.  Now, I could live with the fact that the “default” search engine wouldn’t find them if non-indexed, but I should have a readily available Advanced Search Option where I can put the conventional XP search options:

    Search for files or folders named:

    Containing text:

    Etc…

    This search would be carried out solely based on the search filters entered, regardless of the settings for the "normal" searches.  The search would also be for current folders (and sub folders if option selected), including binary files.  None of the results would need to be stored for some indexing files, which wouldn’t affect “normal” searches.  Finally, when browsing to a different folder and selecting Advanced Search again, the last options I entered should show up by default, as it did under XP.  Now, this option would have to be available BEFORE doing the entire “normal” search.  Everything, could be left as is for default configuration, which should impact your “vision” of how you want Win to look.

    A nice way to implement this would be to be able to customize that “Static Magnifier” icon next to the search field to be an “Advanced Search” button.  When clicking on it, a window would expand under the tool bar (like it did under Vista) where I could put in the search criteria, as given above (NOT the ones under Vista!!).  You could also select from these options:

    - Indexed files only

    - Indexed and Non-Indexed files (this searches any know file types)

    - All files (this searches any file types, including unknown file types)

    Again, these options would be maintained for the subsequent searches.

    I really want to move to Win7.  But I use the advanced search features in XP almost on a daily basis.  The lack of such feature is a HUGE handicap under Win7.  

    P.S. I seem to recall available “inline” advanced options under the search field, but these are obviously impractical solutions for a daily use.

  • oops... little correction... replace instances of "non-indexed" with "unknown file types" in the main block

  • A new feature based on Search that allows searching across PCs and even servers in an Enterprise setting......

    http://www.stretchmarkscare.com

    http://www.acnetreatmentsite.com

  • Great article. Thought it looks more technical for me, i still like the search in Vista and Windows 7. But i still miss the search button available in XP, though there are tweaks to get it back. Check my Windows blog. http://www.techlikes.com/2009/07/windows-7-hidden-country-themes-trick.html

  • To be fair, the competitor you mentioned apparently gained a lot of footprint by removing the support for a whole class of processor.  It is worth doing a comparison for some of the issues raised for us as well, since one might find comparable disk footprint (for example printer drivers).  

    This is an area we pay close attention to and are always trying to balance consuming the least amount of space with recognizing that the vast majority of our customers obtain Windows as an image on a PC and don't have access to a DVD for installation so having drivers and hardware support on disk is important for recovery scenarios.  

    As far as RAM, this is of course another area we worked very hard on.  The feedback we've seen here reflects the work we've done I think.

  • windows 7 seems to handle enterprise data quite well. I just hope it works well in terms of compatibility with all the apps and games out there.

  • The current layout is not intuitive as there is no obvious indication what the relationship is between the name, the file path in parenthesis, the actual file location, and the current folder. All this information must be inferred by the user based upon where they were before the search happened.

    http://www.acaiberrystudy.net

  • i am having some problems with software not working with windows 7 but the updates are all coming out fast so it seems to be adapting quite well

  • most software actually seems to have updates faster than windows 7 was released there really is no waiting

  • This is a reallly important info for me.ove it!

  • For many Americans, the concept of success is a source of confusion. As a people, we Americans greatly prize success. We are taught to celebrate and admire the one who gets the highest grades, the one voted most attractive or most likely to succeed.

  • I estimate that almost every company has some unprotected customer-data on the internet...

  • Although Windows Search can drive me nutz as well as drive a computer into utter slowness, this is a very cool idea.

  • I love reading about Windows updates

  • Is there any tutorial where i can learn more about it, i am also getting some error this time. Please let me know.

Page 4 of 4 (60 items) 1234