Microsoft Enterprise Search Blog : search

The Search Developer Story in SharePoint 2010 - Query Interfaces

enterprisesearch — Fri, 20 Nov 2009 23:39:00 GMT

SharePoint 2010 includes a number of features that make the platform easier to use for developers. An improved Visual Studio integration, the addition of LINQ to the SharePoint platform, sandboxing for deployment, and the new developer dashboard are just a few examples of how developing and deploying SharePoint solutions have become much easier.

As a member of the enterprise search development team that has worked to bring FAST Search into SharePoint 2010, I can tell you that a lot has also been done to benefit developers of search-based solutions. SharePoint 2010 Search and the new FAST Search for SharePoint 2010 have been designed to share a common platform so that search developers can integrate with both SharePoint Search and FAST Search for SharePoint 2010 using the same query side interfaces. This means developers don’t have to learn new APIs or programming models, but can leverage the same object models, services and a common query language for both products.

SharePoint developers and architects implementing search-driven applications should understand the available integration options. Depending on requirements, tools, and preferences, one can choose from among several integration points, including a brand new object model in SharePoint 2010. Here’s a list of the different integration points with a brief description of each:

The Federation Object Model (OM)

This is a new search object model in SharePoint 2010. It provides a unified interface for querying against different locations (search providers), giving developers of search-driven Web Parts a way to implement end-user experiences that are independent of the underlying search engine. The object model also allows for combining and merging results from different search providers. Out-of-box Web Parts in SharePoint 2010 are based on this OM, and SharePoint 2010 ships with 3 different types of locations; SharePoint Search, FAST Search and OpenSearch. The Federation OM is also extensible, should you want or need to implement a custom search location outside of the supported types.

The Query Web Service

This is the integration point for applications outside of your SharePoint environment, such as standalone, non-web based applications, or Silverlight applications running in a browser. The Query Web Service is a SOAP based ASMX web service, and supports a number of operations, including:

Querying and getting search results

Getting query suggestions

Getting meta data, e.g. a list managed properties

The same schema is shared for SharePoint Search and FAST Search, and both products support the same operations. For querying, clients can easily switch the search provider by setting a ResultsProvider element in the request XML. A number of extensions are available for FAST Search, e.g. refinement results, advanced sorting using a formula, issuing queries using the FAST Query Language.

The Query RSS Feed

Certain scenarios, like simple mashups, may need only a simple search result list. The RSS feed is an alternative, lightweight integration point for supplying applications outside of SharePoint with a simple RSS result list. The Search Center - the default search front-end in SharePoint 2010 - includes a link to a query-based RSS feed. Switching the engine to the RSS format is done by simply setting a URL provider. Because of its intended simplicity, there are some limitations to what can be returned and customized in the query RSS feed. The object models or the web service integration scenarios are recommended for more advanced applications.

The Query Object Model

This is the lowest level object model, used by the Federation object model, the Query Web Service and the Query RSS feed. Both SharePoint Search and FAST Search support the KeywordQuery object in this object model. While the Federation OM returns XML (to Web Parts), the Query OM returns data types.

The Search Web Parts

Search Web Parts in SharePoint 2010 are common in SharePoint Search and FAST Search, and are now based on the common Federation OM. The Web Parts on a page communicate through a shared Query Manager, a central component of the Federation OM. This makes adding new Web Parts that interact with existing Web Parts simpler than before. For example, a new Tag Cloud Web Part for visualizing the query results can utilize the shared Query Manager for getting results. Developers will also be able to extend out-of-box Web Parts as they now are public in SharePoint 2010 (no longer sealed).

The Common Query Language

Both SharePoint Search and FAST Search support the Keyword Query Language syntax. This is the default query language for both products, and the end-user language supported from the Web Parts in the search centers (including the advanced search page).

FAST Search Extensions

FAST Search has a number of extensions beyond the standard SharePoint Search that are available on both the Federation and Query object models, and as well as on the query web service. Some examples are:

The FAST Query Language, which supports advanced query operators like XRANK for dynamic (query time) term weighting and ranking.
Deep refiners over the whole results set, and the possibility of adding refiners over any managed property
Advanced sorting using managed properties or a query-time sort formula.
Advanced duplicate trimming, with the ability to specify a custom property on which to base duplicate comparisons.
“Similar documents” matching.
FAST Search Admin Object Model for promoting documents or assigning visual best bets to query keywords/phrases.

Building powerful search applications is easier than ever in SharePoint 2010. FAST Search is now integrated into the SharePoint platform and developers of search-driven solutions and applications can leverage a common platform and common APIs for both SharePoint Search and FAST Search. This means applications can be built to support both search engines and then extended if and when desired to take advantage of the more advanced features available with FAST Search, such as dynamic ranking, flexible sort formulae, or deep refiners for insight into your full result set.

Arnt Schøning, Senior Development Engineer | Microsoft Enterprise Search Group

(o on Twitter as @aschoning)

FAST meets SharePoint - What's Coming in Search for SharePoint 2010

ntreloar — Wed, 28 Oct 2009 01:22:00 GMT

Last week was the 2009 SharePoint Conference in Las Vegas. The sold-out attendance of 7400 doubled the number from the previous SharePoint conference 1 ½ years ago. This is not too surprising given the incredible momentum of SharePoint and the fact that much of the event was dedicated to disclosure of the highly anticipated SharePoint 2010 release. Surprising or not, it was gratifying for us search guys to see the level of interest in the new search capabilities being disclosed for 2010. Several of the search-specific break-out sessions had as many people in the audience (>1000) as the entire attendance of our FASTforward’09 search conference in February earlier this year.

At that FASTforward conference in February, Microsoft announced plans to deliver enterprise search targeting two general solution areas: 1) business productivity applications, where the emphasis is on search driving employee efficiency, and 2) Internet business applications, where search is used to drive customer service and revenue. The disclosure of the new search options in SharePoint 2010 at last weeks SharePoint Conference amounts to the first deliverable of this strategy.

SharePoint as a whole has evolved from the original content management and portal platform of earlier releases into a complete “business collaboration platform”, and there are *a lot* of enhancements and new capabilities in SharePoint 2010. I won’t even attempt to summarize them all here. Instead, check out Jeff Teper’s blog post from early last week, which provides a remarkably good summary of everything that’s coming in SharePoint 2010. As Jeff points out in his blog, search is just one of several major categories of capabilities in SharePoint 2010, but “enterprise search is a big investment area for Microsoft” and an area where “we’ve added depth at all levels in 2010”.

There are two main enterprise search options coming with the SharePoint 2010 release:

1) SharePoint Server 2010 Search – the out-of-the-box SharePoint search for enterprise deployments.

2) FAST Search Server 2010 for SharePoint – a brand new add-on product based on the FAST search technology that combines the best of FAST’s high-end search capabilities with the best of SharePoint.

SharePoint Server 2010 Search represents an important upgrade to the existing search for SharePoint, while FAST Search for SharePoint 2010 is a completely new offering and the first new product based on the FAST technology since FAST was acquired by Microsoft in April 2008. Customers and partners familiar with search in previous versions of SharePoint will see many important improvements in 2010, regardless of which product they deploy. For example, there is a new People Search feature for expertise identification and search-driven collaboration, to name just one (see Jeff’s post for a good summary of these general improvements).

FAST Search for SharePoint 2010 adds a whole new level of search capabilities that are a superset of what comes in the out-of-the-box SharePoint 2010 Search option. Since there are now two search options in 2010, it’s useful to understand what is unique in FAST Search for SharePoint and when you might consider using it over the out-of-the-box SharePoint 2010 search. With that in mind, here are my 10 reasons to consider FAST Search for SharePoint 2010:

1) Content Processing Pipeline

For people familiar with the FAST Enterprise Search Platform (ESP), the good news is that the most valued capabilities of ESP have been brought into FAST Search for SharePoint 2010 and made easier to access and deploy through tight integration with the SharePoint management and development tools. The open framework in FAST ESP for creating custom content processing pipelines is a good example. Since it was first introduced in version 3 way back in 2002, FAST customers and partners have leveraged advanced content processing and advanced linguistic features to create a wide variety of novel search applications. This highly valued aspect of the FAST ESP will be available in FAST Search for SharePoint and has been architected and enhanced to take advantage of the SharePoint management interfaces and development tools like PowerShell.

2) Meta-data Extraction

Meta-data is used in search for faceted refinement, relevancy tuning, targeted queries (e.g. search only the authors field), and other general techniques designed to improve findability. The problem is that unstructured documents are often devoid of useful meta-data. The ability to automatically extract meta-data to create useful structure on otherwise unstructured documents is a feature of FAST ESP that will also available in FAST Search for SharePoint 2010. Importantly, FAST Search for SharePoint 2010 takes advantage of simple administrative tools and the concept of “managed properties” in SharePoint to support adding custom meta-data extractors very quickly.

3) Structured Data Search

Structured data search is possible with both search options in SharePoint 2010, but FAST Search for SharePoint 2010 adds an extra level of sophistication for searching data that contains numbers, dates, and other encoded and structured information. To start, the full FAST Query Language (FQL) is available to application developers who want the richness and expressiveness that FQL provides. This includes support for numeric and date data types, formula-based query operations, term weighting with the XRANK operator, and much more. Also, integration with the new Microsoft Business Data Connectivity services in 2010 means that ingesting structured data from external Line of Business applications is much easier in FAST Search for SharePoint.

4) “Deep” Refinement (Faceted Search)

Previously only available in SharePoint search through 3^rd party add-ons, faceted search, called “refiners” in the default search interface (SharePoint Search Center), is now native in the out-of-box SharePoint 2010 search. FAST Search for SharePoint adds to this the ability to deliver faceted search across results sets of any size while retaining precise counts on the refinement facets. This is critical for research and analysis applications where precise counts on facets are important decision making criteria. (You can see examples of deep refiners on FAST ESP powered sites like scirus.com and dell.com.)

5) Visual Search (Document Thumbnails and Previews)

Visual document thumbnails and previewer Web Parts will be out-of-the-box with FAST Search for SharePoint 2010 to help users more quickly judge what is relevant in a search result list. This includes a graphical previewer for PowerPoint presentations based on Microsoft Silverlight that allows users to quickly find the “one slide” of interest without having to open up the entire presentation.

6) Advanced linguistics

The quality of search against text data is highly dependent on the ability to apply the right language-specific processing techniques. FAST Search for SharePoint 2010 builds on the FAST ESP heritage and Microsoft tools to include advanced language processing (linguistics) for dozens of languages, including optimized processing for Chinese/Japanese/Korean.

7) Visual best bets

SharePoint already supports the concept of search Best Bets – managed results delivered with the search for specific queries. FAST Search for SharePoint adds to this the ability to render visual best bests in the form of images and even videos. Management of search best bets, both standard and visual, is through the standard SharePoint administrative console.

8) Best-in-class development platform

FAST Search for SharePoint 2010 builds on the comprehensive development framework of SharePoint 2010. The customization options range from configuring out-of-the-box search behavior (best bets) and user interface controls (Web Parts), to extending existing functionality using public Web Part code and SharePoint Designer, to creating brand new components and functionality with the available APIs. For FAST ESP aficionados, no compromises have been made in the area of extensibility with FAST Search for SharePoint, but many of the customizations in ESP are now much easier to do.

9) Custom search experiences (per user/profile)

FAST Search for SharePoint 2010 includes the same level of relevancy tuning available to FAST ESP. It will be possible, as it is in ESP, to create custom relevancy models tuned to differences in content sources, application needs, and user contexts. User context simply means that different users can have different search “contexts” that enable experiences optimized for their specific business needs. User context can be used to set the search sources, relevance rank profile, linguistic processing features, and other search features by user or user group. In an enterprise search setting, this means that a Sales Director does not have to see the exact same results as a Product Designer for a given query, even if they are searching the same sources.

10) Extreme Scale and Performance

Scale and performance of the out-of-the-box SharePoint 2010 Search has been dramatically improved – with proven scalability to 100 million documents and more. For FAST Search for SharePoint 2010, the exact same scale-out model that exists in FAST ESP has been preserved to enable extremes of content (e.g. number of documents to search), queries (e.g. the number of queries or query rate), or both. This means search solutions that can support billions of documents and thousands of queries per second.

There is much to like about what is coming with search in SharePoint 2010 and more information than I’m able to share in one blog post. You can add to the list above the general benefits of search enhanced by all the other tools and services of the SharePoint platform - including content management, communication and collaboration, business intelligence, system administration and monitoring, application development, and so on. As I’ve pointed out in previous posts, search doesn’t exist in a vacuum, and the ability to integrate and interoperate with other business productivity functions is critical to actually acting on a search result. From this point of view, SharePoint and it’s compendium of integrated services, simply makes search better.

The first public beta for SharePoint 2010 will be available in a few weeks. This will include beta bits for the standard search in SharePoint 2010 and FAST Search for SharePoint 2010. I hope you’ll be able to try out these new search products and features. In the mean time, you can learn more about what’s coming in search for SharePoint 2010 by going to the SharePoint 2010 preview site at http://sharepoint2010.microsoft.com/.

Nate

A Focus on Search User Experience

ntreloar — Tue, 14 Jul 2009 21:17:00 GMT

It’s happening… slowly … but it’s happening.

Attention in search is finally shifting from a focus on low-level features and relevancy models to looking at the whole user experience for information access. I, for one, am very glad to see this trend. Of all the enterprise technologies out there, few are planted so squarely at the interface of humans and machine as search. And yet, for many users, the search input box and a list of blue links is still the pinnacle of a search user experience – a user interface model that hasn’t changed appreciably in over 10 years. There is room for improvement.

So what exactly is happening? I’ll point out three things:

1) New Books on Search User Interfaces and User Experiences

Three recently published books have put some focus on human-computer interaction (HCI) and search. The first book by Ryen White, from Microsoft Research, and Resa Roth, published earlier this year, covers the topic of exploratory search. From the abstract:

Exploratory search has gained prominence in recent years. There is an increased interest from the information retrieval, information science, and human-computer interaction communities in moving beyond the traditional turn-taking interaction model supported by major Web search engines, and toward support for human intelligence amplification and information use.

The second book, Daniel Tunkelang’s on faceted search, looks at a particular interaction pattern that is now a mainstay of most commercial search platforms. Daniel, co-founder and Chief Scientist at Endeca, can speak with some authority on the topic of faceted search since his company was essentially built on the idea.

The third book, and perhaps the most ambitious, is from Marti Hearst, a respected researcher in information retrieval and text mining at UC Berkeley, who has recently released for online reading (print version coming in September) a comprehensive review of search user interface research.

A general theme, with the first two books especially, is on user models for search that are interactive and iterative. They address, in part, the fact that users are not very precise in communicating their information needs in an ad hoc query. While there is some evidence that keyword queries are getting longer, the oft-referenced 2.3 term average query length still demands user experiences that don’t just try to deliver the best possible results on the first attempt, but that can help the user ask a better question through contextual navigation, iterative feedback and refinement options.

2) New and Evolving Examples Online

Beyond the three more academic works above, there is also evidence that commercial search applications are focusing more on search-based user experience. In a post last month, I referenced a couple Microsoft/FAST customers, Oodle and Globrix, who have put a particular emphasis on user experiences built completely around search. Other sites, like Getty Images’ Catalyst search take advantage of the uniqueness of the domain (image search) to create rich and engaging experiences built on search.

On the wider Web, Microsoft launched the Bing “decision engine” in May with query disambiguation features built in. Even Google has relaxed its keep-it-simple position by adding search options to enrich the user experience. Compared to domain-specific enterprise search applications, the Web search engines are just beginning to dip their toes in the water, but otherwise the same theme exists: search user experiences that are more interactive, iterative, and conversational.

3) Search UI Design Patterns

Finally, the past couple years have seen efforts to formalize UI design patterns for search. Peter Morville has championed this idea and posted a nice compendium of discrete search patterns with example screen shots on Flickr (also see his wiki). The idea of cataloging UI patterns for search is so that the good patterns - those that have been proven to work well and to result in a positive user experience - can be promoted and reused. There is also the concept of “anti-patterns” or patterns that have been shown to have a neutral or negative impact on user experience. (As an aside, Peter’s catalog of patterns focuses on GUI patterns – many of which will be familiar even to non-practitioners. In my post on search and Natural User Interfaces , or NUIs, I mentioned that these new “touch and gesture” UIs do not have established patterns yet for search. It is truly a greenfield and it will be interesting to see what patterns emerge.)

As said, I’m a fan of this focus on user experience in search and also of the formalization of best practice design patterns. I’d like to see it all go a little further, though. Having a set of discrete and generic patterns is helpful, but even better will be having best practice patterns that are oriented toward specific business processes where search is used. Understanding these meta patterns in enterprise search is especially important in order to understand user experience differences between search for Research, search for eCommerce, search for Customer Service, search for eDiscovery, etc... Some of these differences are in the search features themselves, others are in how search interfaces with other non-search features and workflow (e.g. shopping carts in eCommerce or communication tools for collaborative research). For example, the “product comparison” view is something common in eCommerce applications and, while not obviously a search UI element, its rendering is dependent on search results.

In time, I expect these meta patterns to evolve into user experience and UI templates (customizable) that will help organizations quickly stand up search front-ends that take into consideration not just how people search (functional patterns), but why people search (process patterns).

Nate

Observations from the Text Analytics Summit 2009

ntreloar — Wed, 10 Jun 2009 04:56:00 GMT

One of the hard parts about organizing a conference like the 5^th annual Text Analytics Summit, held last week in Boston, must be selecting the industry case studies. Text analytics is a highly specialized, but broad reaching topic that has applications in life sciences, financial service, legal, retail, government, media, and entertainment, to name a few. Any one of these industries could have filled the conference with interesting examples.

As it was, most of the case studies and vendor briefings at this conference were about Voice of the Customer or Market Intelligence. I suspect that some attendees might have preferred a little more variety in the cases presented. The absence of any government case studies, for example, was conspicuous, but understandable given the special nature of that domain. We’d all probably have needed security clearances to attend those sessions anyway. Overall, I appreciated the more commercial/consumer focus and felt that the conference organizers did a great job of finding representative examples and balancing the practical (vendor briefings and case studies) with the theoretical.

As a first time attendee to the conference, I was interested in just getting the lay of the land in text analytics, but I was also interested to learn how people were answering the “what’s next” question. It came up several times over the 2 days during Q&A and panel sessions and there were different takes, but I paid close attention to three, in particular, that resonated with my own observations looking through the lens of enterprise search.

Trend 1: ETL-like Tools

Ok, this is not really a trend in text analytics, but it is one in enterprise search that is informed by text and data analytics.

Many of the vendors at the conference demonstrated graphical tools designed to simplify the process of building text analysis “pipelines”. These tools look very much like the Extract, Transform, and Load (ETL) tools that have been around for many years in the data integration world. The difference is that the text analysis versions of these tools focus on operations for handling unstructured text. For example, named entity recognizers are a common text analytics task for automatically recognizing and tagging things like person names, company names, and locations in text.

This ETL “pattern” exists in enterprise search, as well, where information must be extracted from a source repository (e.g. an email archive), transformed into an enhanced, canonical representation (e.g. annotated XML), and loaded into a database or index for searching. The demand for graphical tools to manage the ETL process for search has not been as high as for text or data analysis. I think this partly because, for search applications, it is usually a one-time set up process and not an iterative modeling exercise as it is with text analytics. It may also be because historically the operations performed on content before it’s indexed for search have not been as sophisticated as the operations performed for in-depth text analytics.

This is changing. To start, extensible pipeline processing frameworks that incorporate advanced text analysis capabilities have become more common in enterprise search products. By now, most mainstream enterprise search platforms include entity extractors, for example. We are also seeing more ETL-like graphical consoles for managing content integration and analysis.

The adoption of these tools and techniques for enterprise search is motivated, in part, by a desire to more easily harness text analytics features that increase search precision and create richer search experiences. It’s also the case that, while text analytics shares a heritage more with information retrieval (search) than with business intelligence (BI), it includes technologies relevant to both and sits smack in the middle of the convergence between these two spaces. Sue Feldman and Hadley Reynolds of IDC reinforced this role of text analytics by describing it as a cornerstone of Unified Information Access during their Market Report at the conference. Given this, it shouldn’t be surprising to see that, as text analytic tools and concepts have found their way into BI applications, traditional BI tools and concepts, like ETL, are finding a place within enterprise search.

Trend 2: Empowering the End User

Another topic that popped up at various times during the conference was the challenge of delivering the richness of text analysis tools to users other than specially trained analysts. As with traditional BI tools, many text analysis tools assume a trained user or “analyst” capable of designing sophisticated workflows or advanced analytical models. One question posed to a speaker after he finished describing his text mining process was “when do you think you’ll be out of your job?” - meaning, when will the tools be so easy to use that your end users won’t need you to do their investigation for them?

I’m sure this exact question was asked at a conference of professional research librarians some 15-20 years ago - back when online search services and later Internet search engines were becoming easier and easier to use and obviating the need for “professional searchers”. The answer is likely the same, too. There will always be specialists and “power users”, but as the tools become easier to use, end users will become more empowered to do their own increasingly advanced analysis.

In practice, we are seeing more applications that combine conventional search with advanced text analytics in ways that bring a more powerful search experience to relatively unsophisticated end users. Silobreaker.com is a clever site that combines the richness of text analytics within what is fundamentally a news search application. Unlike other news search sites, Silobreaker offers options and tools that help to uncover and discover interesting and potentially novel connections and patterns in the news. There are still some usability challenges with a consumer site like Silobreaker, but I like it as an example of ad hoc search converging with iterative knowledge discovery.

The trend toward empowering users with more than just a search box and list of blue links also reaches into less “analytical” consuemr applications. Two examples are www.oodle.com and www.globrix.com. Both sites show the power of applying analytics to both structured and textual data (classifieds in the case of Oodle, real estate postings in the case of Globrix) in what are otherwise fundamentally search applications.

Trend 3: Taking Sentiment Analysis to the next level

Sentiment analysis is the ability to recognize the mood, opinion, or intent of a writer by analyzing written text. It is sometimes called the “thumbs up, thumbs down” problem because the most common application is establishing whether a writer is positive or negative on a particular subject. In this form, it is often used to analyze written product reviews (see this example on Microsoft’s new Bing Web search).

Sentiment was a much mentioned topic at the conference. This is not surprising given the focus on Voice of the Customer and Market Intelligence – two areas where accurately establishing the sentiment of customers and consumers toward products, services, and brand is highly desirable. One of the presenters at the conference was Roddy Lindsay from Facebook. I missed that session, but it doesn’t take much imagination to appreciate the possible applications for text analytics and sentiment analysis, in particular, with the information available on Facebook and other social networking platforms.

Every vendor present had something to show or say on the subject of sentiment analysis, but all the panelists in the vendor-only panel acknowledged the difficulties of increasing the precision of sentiment classification. Currently, the number tossed around is 80%. That is, a sentiment classifier will get it right about 80% of the time compared to human judgments. This number is higher in some applications - for example, when analyzing short, strongly opinionated product reviews. It is lower when analyzing longer pieces of text where just fixing the subject can difficult – like this blog post.

Progress is being made, though. The first step has been a shift away from “document-level” sentiment to “topic-level” sentiment. This allows sentiment classification to be more accurate when confronting documents, like this post, that touch on and offer opinion on multiple topics. It also helps with more concrete problems like the ones represented in this sentence:

“Acme’s new P40 digital camera has a good viewer, but its controls are awkward.”

While it’s relatively easy for a human, it takes some heavy linguistic lifting for a machine to recognize that the sentiment of this opinion is directed not just at Acme or at the P40 digital camera, but specifically at the viewer (positive sentiment) and the controls (negative sentiment). It’s ever trickier establishing what the word “its” refers to in the 2nd part of the sentence. Is it the Acme P40 itself, or just the viewer?

Sentiment is admittedly a niche topic, even within text analytics, but getting it right matters a lot for enterprise search applications in eCommerce (think product reviews), Market Intelligence (reputation tracking and competitive intelligence), eDiscovery, and Government Intelligence. One presenter suggested that all the remaining hard problems in sentiment analysis will be solved, at least academically, in a couple years. It will be interesting to see how soon these improvements surface in real-life applications.

Nate

Search and Natural User Interfaces (NUIs) - Part 1

ntreloar — Mon, 27 Apr 2009 19:08:00 GMT

About five years ago or so, I participated in a conference panel where the question was asked: “What will search interfaces look like 20 years from now?”. I had just seen Steven Spielberg’s sci-fi film “Minority Report” starring Tom Cruise, so I referred to the scene where Cruise’s character is interacting with a futuristic looking visual display and using appropriately dramatic gestures to grab, spin, shrink, expand, and otherwise manipulate the various news stories and images floating on the display.

I heard later that Spielberg, while developing the script for the film, had consulted a number of futurists to create as realistic picture of the year 2050 as possible (from the point of view of those futurists at least). Interestingly, over the past several years, that scene has become a conceptual benchmark for so-called natural user interfaces (NUIs), to the point where if you search for “minority report” in your favorite Web video search engine you’re as likely to find examples of prototype NUI products as you are trailers for the actual film. It’s not a stretch, imo, to say that the film has inspired and perhaps even accelerated advancements in NUI products and technology.

There are now many good and real examples of NUIs and even some actual products that come close to the vision in "Minority Report", but despite the impact the film appears to have had on the development of NUIs, there is a very strong connection to search that gets overlooked. Cruise’s character in that scene is searching. His various gestures and other contortions are queries, navigation, and refinements intended to help him find answers and collect information. Granted the depiction is not quite up to the vision of the smooth-voiced computer on Star Trek, but it’s a step beyond the keyboard and mouse and, if you look past the theatrics, I think it paints a realistic view of not just the future of natural user interfaces, but of the type of natural search-driven user interfaces we will be seeing soon… in much less than 20 years time.

Nate