Designing search for the Start screen

re: Designing search for the Start screen

Drewfus — Tue, 25 Oct 2011 16:42:42 GMT

@mil_

"What is the best device I should be using to control all these screens? A tablet of course or a remote control that looks like a tablet, let’s say a 7” tablet with the Metro UI. So that actually proves the point that Metro UI is for tablets and not for desktops."

No, it doesn't. All it "proves" is that a household full of wall monitors might be best controlled via a tablet. For a house with one or two wallscreens, direct touch would be useful, just as it would be for a display built into a refrigerator, or other household appliance.

"I do not want to transition to a full screen application just for a few seconds. It is tiring, stupid and unnecessary."

I timed the transition from Desktop -> Start (and the reverse), and it seems to be around 0.75 seconds. This is below the approximate 1.0 second threshold that usability expert Jacob Nielson identifies as "...the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay." www.useit.com/.../responsetime.html In contrast, the start menu goes close enough to being within the 0.1 second barrier, which "...is about the limit for having the user feel that the system is reacting instantaneously..." However, in the case of Start-Desktop transitions the user will have to readjust to a totally changed screen display, as well as processing movement during the transition interval. This would mean getting perilously close to a genuine disruption of the users attention. Not good. Microsoft will most likely need to reduce the transition time right down, and minimize the cognitive overhead as best they can, before Win8 goes RTM.

"When I am using my computer my main task is not to search for applications or look at live tiles, I have work to do. If I am video conferencing with somebody and I have outlook open as well as Visual Studio, I don’t need/want an application of any type going and covering my desktop."

I'll just make a point i've already made in a slightly different way - don't let Microsoft's new terms for old things mislead you. If you want an example of what i mean, go to www.bing.com - and there is a very simple Metro style UI - a solid background and one big "tile" in the middle displaying content, plus a field for user input. That tile could just as easily be your video conferencing app, or your newest inbox entries. A Metro tile is just the analog to an HTML <div> - "reimagined" as the building-block for web-based app interfaces. A combination of tiles is called a group, but you could call it a webpage, or a window, or a ... banana. The point is we now have a web-standards based application interface model, albeit incomplete. The other thing about tiles is that they can (or will) be created in any size required. So not just 1x or 2x, but also 0.1x, 10x, fullscreen, eighth-screen, quad-screen, fit-to-content. Whatever suits. Don't assume any limits, unless Microsoft tell you otherwise, and maybe not even then.

"Do I really need to scientifically explain why the telemetry data is used wrongly in an effort to support a UI that is made for tablets and desktop challenged simpletons?"

If Microsoft justify their design decisions using objective data/evidence + usability testing + trial & error + existing research + well known design principles + maths, you are going to have to do a lot more to counter their arguments than just assuming your own conclusion. Telemetry is useful for optimization, but to optimize implies that the fundamentals of the thing being designed or refined must already be in place, so these must have had a different source of inspiration. Do F1 teams design their cars using telemetry alone, or do they use telemetry for fine-tuning?

"Is Windows “difficult” to use because I am getting lost constantly when searching for my applications? Why do they assume I am always searching for applications? Why do they assume it is so important to me than I am willing to sacrifice my whole desktop even for a few seconds/minutes? Is this what their telemetry data tells them?"

I partially agree with you here, in that Microsoft seems to be trying to persaude us that the start menu is so bad and yet so crucial to users that we should regard it as the Achilles Heel of Windows that absolutely needs a major overhaul - to put it mildly. Of course, the counter-argument is always going to be that since, as the telemetry tells us, 88% of app launches occur from places other than the start menu, then to that extent the argument that the start menu is some grossly flawed thing must also be 88% irrelevant. The problem for the "pro-start menu" group is that the argument is a two-way street - if you're not using the start menu much for app launch or for search, then why so much angst over the occasional mildy interrupting Desktop -> Start -> Desktop cycle, if you're getting stuff like fullscreen search results and data services into the bargain? Aren't we sailing perilously close to a storm in a teacup here?

"I am not going to be touching my 3 desktop monitors now or in the future."

Fine, but touch is touch and Metro is Metro. Besides, i reckon you will touch your monitors, occasionally :-)

"Searching for applications/files is not what I am paid to do when I am sitting in front of my computer."

Almost none of us are, but here is the fascinating thing (that i predict we'll see covered in a future post) - most users spend an inordinate amount of time searching for files, apps and webpages. Web search time is something we are well aware of, but apparently users spend some amazingly high proportion of each computer-facing hour searching for stuff, particularly files. There are several reasons, including the poor file naming habits of users, and the fixed heirarchies that existing filesystems force us to use. www.useit.com/.../filedeath.html So i don't agree with you when you say - "“Full screen” is not the same as “full use”, it is just a lazy way of using the desktop that appeals to novices but is against professional users of the desktop" - in the case of search. On the contrary, if anything should be fullscreen in Windows, it's search.

"Microsoft seems to confuse the message of Windows Everywhere, which is perfect for the OS kernel with the message that every device can have and should have the same basic interface. I find this later message to be childish and wrong and the only reason to go down that path is if they really want to change their user base completely."

Again, your last point is true, if you assume your own conclusion (that Metro UI is for tablets and not for desktops) - but why?

re: Designing search for the Start screen

Ehsan — Tue, 25 Oct 2011 15:24:48 GMT

Windows team ! i have some questions and plz..answer me !

maybe someone above has asked this but as there this complain about differentiating the apps..files.settings into categories .. and how that doubles the clicks (taps on tablets) to find a file o settings when you're not intending for an searching an app ?

so you tell me.. you need more room to show the results and avoid the confusion..but you're actually doubling the taps / clicks aren't you ?

if you insist on doing it your way..ok .. but why don't you apply this workaround ?

if a user hits win+F for example..it still gives you the files category BUT it's not files..it's files FIRST

if he hits win+W then it gives you settings FIRST search result ??!

And i have another question that i be glad to be answered

earlier this week mr ballmer said in press cf that " I don't want to be a scientist to be able to use a (android ) phone..so

since you both work in the same company.. .. I ask this..

haw could I know I had to hit win+W to get the settings results and not be a scientist ??

seriously..i used developer preview for 3 weeks at least (and i'm a soft eng by the way ) and I didn't notice it until I saw that on your blog..maybe I'm dumb !?

re: Designing search for the Start screen

Computermensch — Tue, 25 Oct 2011 14:49:42 GMT

And finally. With the virtual start menu - and possibly virtual shelll - Windows is getting ready for its future. It has 95% market share. It's so huge - nobody really invests so much money in developing the infrastructure except Microsoft. And may be Goolge coming up - except they still does not really care about innovation in the OS because they are using Linux - and have to give stuff away.

MacOS is notorious for having most of BSD code in its kernel. Android is Linux. Both dependend on open source license so every investment possibly lost to the public. So nobody really invests.

With the virtualization starting to come into the desktop level we can finally get on with the game and say, hey - the roads have been build. Let's focus on some new stuff instead. And i.e. the MacOS at some point in the future can be Windows core in part rebranded at the deskttop level. Just as it already is - but BSD. Same for Android - but Linux. They all prepared for it choosing stuff that runs on Intel technology.

Let's get on with it. There are new ventures to be done.

re: Designing search for the Start screen

Computermensch — Tue, 25 Oct 2011 14:26:31 GMT

We can also add that the Start Menu because of exactly that role is the most important object in the operating system controlling the end user experience.

Technology from Microsoft is extremely great but its under the hood

Microsoft needs to make the Start Menu and thus the end user experience as just a good techonology (characterized by almost no weaknesses) as the NTFS (like the introduction of transaction control in Vista and Windows 7). Complete hidden but great. Now in Windows 8 a new file system is expected by some. So NTFS is not good enough in Microsoft opion. Yes, a new technology is may be needed. But thats under the hood and nobody "consumer" like ever sees it or directly experience it.

But the same ambitiousness from Microsofts part should be there regarding the most important thing for the end user experience. Where's all the software and automatics - the smart stuff - beyond looks and graphics?

Again virtual start menu. Come on! Don't do a Start Menu with an Apps list - don't do it likke 1995.

Think end user expience has to be in value, in the possibilities - in USE! Not just pictures or imagination.

Do the same great stuff there as elsewhere. Bottom up is great ... but top down too if you have to prioritize. Building Windows 8 is an imaginary process ... theres no way to start with the bottom first and do a technically strong and very useful UI in a later release. You can't do such a UI first and have no a weak engine under the hood.

Do both. Great with the strong engine. That was also the case with Windows Vista - it was the core for Windows 7 - like Windows 2000 was for Windows XP.

Now WIndows 8 - new core, WinRT, possibly protogon and much, much more. Very busy doing core technology. Which is great.

Only to avoid the Vista thing - remember the UI too! And its not about some nice pictures - but useful.

Vista had an animatied desktop. Never mind.

The technology at the time for Vista was very good and it was the foundation of Windows 7 - Vista rebranded with A NEW DEKSTOP!

So please get it. Personally, I wrote a feedback note then critizing the lack of renewal in taskbar during Vista. It came with Windows 7!

In Windows 7 I worried about the lack of renewal or innovation in the taskbar and start menu. It came with Metro.

But now I worry about the usefullness. It has to be strong technology like what to do for every other component in Windows. Everything is so good.

Then why is there no smart real good techonology in the UI? I'm not thinking about the looks. The looks are rather great. But that Start Menu can not really do anything software cool. Nothing.

Same with the apps list.

They are just appaling in behaviour - but looks ok (the tile stuff). Following that idea, those tiles could be smart of stuff. Or at least I would want that virtual start menu.

You go under the hood and stuff is really cool in Windows

Why is that? What happens at the desktop level? Nobody cares about writing cool software and smart algos for that.

I took a look in process monitor following the work of the Start Menu and Apps List. The apps list just scans folders across the system and presents them icons in the apps list. That's it. Hey, we don't wanna do it the cool way. I want to write a new application pool instead or something like that? No tiles, no stacks, no virtualization. No computer science? Only psychologi and medical science?

Please, please - what I really worry is, that there will be no way to build changes into the beaviour of the Start and also the Apps LIst, because you want to control that complete because of reality, consumer trust and the appstore. Meaning locked up unflexible Start Menu and Apps List - like a new locked up Media Center presentation with no ways of changing that ... and a real forever locked up tech Start Menu (not looks, but FUNCTIONALITY and no virtualization) ... untill next Windows release.

Meanwhile new start menu and launcher comes up - only they will not be able too collaborate with the appstore, because variability and managed API's that can EXTEND THE BEHAVIOUR in the Start Menu was ignored.

I.e. is it wrong to say that from Microsofts perspective the success of the app store depends of the Start Menu. I.e. can anbody install something from the appstore if the are not using that Start Menu? So do great stuff for the Start Menu under the hood and virtualize it. Also with a managed API variants can be required to implement certain behaviour and not overide others. Shoud not be that hard to do.

A lot less harder that a lot of new stuff in Windows 8. Please, more effort here. The artists are doeing great, but more computer stuff ... software stuff. An Start Menu for 2012 that would impress people in Computer Science. Touch is not new or special. It's old. The market interest is new. So let's get more than that.

Allow us to do more with launching programs. The most important part of the user objects.

You already know what I want to do. Spatial Launch Objets. Just provide the API's so we don't have to break collaboration with the app store.

On the phone we can access the live camera feed ...

As I put in earlier I don't mind you doeing you own Start Menu that way - just that in can be replaced. If you don't want it replaced put more software into it - other than the artists, psychologist and other interpretive specialists. Provide more engineers that will write virtual code as well.

But I do mind that you're doeing it the Vista way again. More software into the user expiernence. Think about what you did for the taskbar, right?

It became smarter. It could do stuff.

Now think about how much of that you're doeing with the Start Menu. Search is just appaling. That's delegating the responsibility to the user ... go discover and find it yourself.

I am not saying it easy - I'm just saying the allow the object to be switched. Then we can redo the start menu. Everybody can have launch the way the want. Evyerbody can optimize the user experience. Everybody is different. Let's use differtial programming for the Start Menu.

Virtual!

re: Designing search for the Start screen

mil_ — Tue, 25 Oct 2011 07:16:17 GMT

@Drewfus:

Please read again the example you mentioned

“Now what interface would be preferable for controlling systems like that - the Metro UI with touch, or Desktop with a mouse?”

What is the best device I should be using to control all these screens? A tablet of course or a remote control that looks like a tablet, let’s say a 7” tablet with the Metro UI. So that actually proves the point that Metro UI is for tablets and not for desktops.

“What exactly is the problem in temporarily "losing the desktop"? Is it the transition itself, or the loss of attention, or something else?”

I do not want to transition to a full screen application just for a few seconds. It is tiring, stupid and unnecessary. When I am using my computer my main task is not to search for applications or look at live tiles, I have work to do. If I am video conferencing with somebody and I have outlook open as well as Visual Studio, I don’t need/want an application of any type going and covering my desktop.

When I am driving, I am not going full screen with the in car radio interface because I am changing stations.

My problem with the telemetry data is that they compare oranges and potatoes by saying “look, you now have more space to search for things”. Yes, I don’t even need telemetry data to show me that I can display more results on a full screen window than an 8th smaller Window the old start menu represented. So what?

Do I really need to scientifically explain why the telemetry data is used wrongly in an effort to support a UI that is made for tablets and desktop challenged simpletons? I don’t think I have to prove anything when there is a history of Windows that spans almost two decades. As I said before there was nothing stopping us from “inventing” this “new” way of using Windows a decade ago. The technology required was there one way or another (maybe the smooth transitions would be a bit hard to achieve though).

All these years Microsoft was trying to convince us that the desktop UI could fit into the Mobile Phone…Windows Mobile 6.0, 6.5 etc. Then the market “explained” to them in billions of lost market share that they were following the wrong path. Now they are trying to fit the Phone UI into the desktops…maybe this time they are not completely wrong because they will get all the novice users to like it.

Is Windows “difficult” to use because I am getting lost constantly when searching for my applications? Why do they assume I am always searching for applications? Why do they assume it is so important to me than I am willing to sacrifice my whole desktop even for a few seconds/minutes? Is this what their telemetry data tells them? Does it also tell them that I am interested in animating tiles that distract me from my work constantly without having the option to restrict the size of the Window that displays them? These are the wrong conclusions, it is not what I want as a desktop user and the only reason they try to defend that position is because they are trying to herd us towards that direction.

I am not going to debate the rest of your points because they represent suggestions on how to unify the two paradigms, and probably they are correct if the main objective was correct. But I disagree completely with unifying the two because they are as I said two different ways of using a computer and they will never merge without breaking one of them completely.

The main points are, I am not going to be touching my 3 desktop monitors now or in the future. A full screen application is something I want to have full control of. Searching for applications/files is not what I am paid to do when I am sitting in front of my computer. I know where my applications are and I don’t have tens of thousands of them now and I am not going to have tens of thousands of applications in my desktop in the future. I want a different UI tailored for the device it is used with and make full use of its capabilities. “Full screen” is not the same as “full use”, it is just a lazy way of using the desktop that appeals to novices but is against professional users of the desktop.

Microsoft seems to confuse the message of Windows Everywhere, which is perfect for the OS kernel with the message that every device can have and should have the same basic interface. I find this later message to be childish and wrong and the only reason to go down that path is if they really want to change their user base completely.

re: Designing search for the Start screen

Andrea B. Previtera — Tue, 25 Oct 2011 06:41:29 GMT

@Steven Sinofsky

There's one thing that telemetry can't address: psychological matters. I believe you're disregarding the psychological attachment of desktop users to the Desktop (Open windows, taskbar, icons) as a "home" to their computing needs. Adding a separate screen for most of the critical tasks is understandably a need for tablet usage - and the screen you're building is stylish and functionally superior to every other tablet interface out there. That's were MSFT always shined, after all. But for a desktop scenario, you should allow it to blend somehow with the desktop itself: as an overlay, as a half-screen that slides out of one the screen sides, etc. - without actually "hiding" the desktop, without leaving it and making it disappear from the user's view. This has a very bad psychological impact, that's in my opinion what's generating most of the negative answers to the start screen.

Also, Aero should be somewhat metro-fied for the always underrated question of visual consistency - but that's another matter.

re: Designing search for the Start screen

Irfanfare — Tue, 25 Oct 2011 04:50:38 GMT

Great post, like your previous ones. I Like the way you explain things through demos, stats and graphs. Very persuasively done. Love the Search feature in 8, definitely easier than 7. The only thing that irks me is IE10, its less responsive on my 1.25 RAM, 6 yr. old HP desktop so much so that I have to frequently go to the desktop view. Also wish it had a Home button just as in its desktop view.

re: Designing search for the Start screen

Drewfus — Tue, 25 Oct 2011 04:10:20 GMT

@mil_: Picking up on a few of your comments.

"When I am referring to “scaling”, I mean the Metro UI and not the kernel of the OS."

Ok, fine. However, i don't agree with the general consensus on this blog and in the forum that the Metro UI is great for hand-held devices but doesn't scale well to laptop/desktop PCs. When you go from icons to tiles as the target points, and then start adding app data and app content to the tiles, the bigger the screen you have, the better off you are. Then if you consider combining tiles into groups and regarding the group as a "window", plus the need to zoom out to see all you groups and scroll to see all your tiles, then you are definately at an advantage with a larger screen, even in a relative sense. So i think the idea of Metro being a tablet UI "shoehorned" into a desktop UI is false - and the converse is true. Metro is absolutely designed with big monitors, and multiple monitors, in mind, and not only will it scale well, but these environments are where it will shine.

Take this as a (currently) extreme example. Consider a house like the M & B Gates residence with walls covered in screens that display various types of content based on the preferences of the occupants and visitors. Now what interface would be preferable for controlling systems like that - the Metro UI with touch, or Desktop with a mouse? Of course, it's a "no-brainer". So there's one nice example were the critics of Metro on larger systems are just wrong. The question for me, is not - does Metro scale up? - rather it's - will Metro work ok on sub-netbooks and small tablets?

"Come on, that is a joke…i.e. pressing a button and losing your whole desktop just to perform a search. That looks so ridiculous on my 30” monitor that I won’t even debate it. The only thing I would say is that it is not Job’s or Steven’s eyes that are suffering from the change of contract/brightness and focus every time I press the WinKey on the Windows 8."

What exactly is the problem in temporarily "losing the desktop"? Is it the transition itself, or the loss of attention, or something else? I agree that the abrubt change in contrast/brightness is currently a problem. (Part of that problem is due to apps using stark white, battery-sapping backgrounds. IMO plain white is too bright and should never be used - a light beige or cream hue is far easier on the eyes, especially in dim lighting environments.) Regarding the altered brightness and "jarring" nature of the transition, there are several things that could be done, and i think Microsoft have to some extent deliberately avoided smoothing-out the Metro -> Desktop -> Metro transitions in the DP to see how much people can tolerate. I guess they have the answer now. However, i can imagine things like combining/locking the Metro and Desktop background color and wallpaper would help, as would replacing the slide-back, slide-in affect with a rapid fade-out, fade-in of the tile/window layer (possibly overlapped). That might mean users having to replace their more graphically rich desktop wallpapers with something more bland that doesn't spoil the look of Start, but so be it (or just have a prominent slideshow tile, as compensation). While i'm on the subject, the other tweaks i would make are:

1. The start menu button should look less like the Win7 button than it does in the DP. It should get the Metro color scheme (white flag on green base), and the running app look, so users perceive the start button as representing an app, and not a pull-up menu. Currently the start button is too much of a "lie" (it breaks user expectations, causing a slight loss of trust in the OS). The context-menu gets an 'Open Start Menu Programs' Explorer link so users can quickly rebuild their taskbars.

2. The Desktop tile should be removed, and be replaced by a Desktop group, containing by default, IE and Explorer tiles, plus tiles for whatever icons the user pins to the taskbar (@desktop). Perhaps the Desktop group should get smaller, rectangular tiles, to make this group look a little more like the start menu.

3. System tray functionality should be completely removed from the taskbar (with the possible exception of date/time) and moved to Start. Desktop is just another destination (from Start), so it should no longer host OS related functions & notifications.

4. An unrelated point i'm going to add regardless, is that tile visibility should not be restricted to unlocked sessions. The user should be able to flag tiles as 'Public' (non-confidential), so that when they lock their logon sessions, these tiles stay visible so users can observe live data.

The point i made about telemetary had nothing to do with liking or disliking the new Task Manager personally. Instead, the fact is that everything new in Win8 that has a UI is being designed with telemetary as a guide, or, as a design tool, is probably a better way of putting it. I don't understand why people can't except this, or why people think the telemetary data totally drives the outcomes, with no respect to any other factors, including "the art". There are multiple factors at play here, and MSFT are not using telemetary "fullscreen", like you seem to think they are. Frankly, i think many people (not yourself) are attacking the use of telemetary in Win8 because they can't or can't be bothered articulating what problems or issues they have with the new stuff, or simply want to pretend that MSFT is some big, dumb organization with no "artists", and if only they would listen to me (who has the great design sense and skills), then all would be well. Well to those folks i say - keep dreaming.

re: Designing search for the Start screen

Will — Mon, 24 Oct 2011 22:54:41 GMT

"All my slighty negative comments are not even published here, way to hear users when you can only see positive comments..."

They don't care what we think. They figure we'll buy whatever they peddle as long as its in the store. But if Microsoft PCs are going to be crippled into being tablets, why not just buy a tablet? Windows 8 is all the more reason to buy an Android device. And if Windows usability and multitasking is going to crippled, why not buy a Macbook Air? If Windows becomes as crippled as Mac, I'll just go to Mac. If Windows is going to be given an ugly tablet interface and made into a tablet masquerading as a laptop, I'll just go to Android whose interface is way better than Metro (Android interface is more Windows-like actually, since it mimics your standard Desktop type interface that both Windows XP / 7 and Mac have).

re: Designing search for the Start screen

Will — Mon, 24 Oct 2011 22:50:02 GMT

The problem with the old search (as in XP, Win 7) is the FILESYSTEM not the interface. Microsoft clearly doesn't do any indexing UNTIL you search. Well, duh, that's going to be slow. But how does this idiotic Metro crap solve that problem? It doesn't.