Notes on comments.
Welcome to our blog dedicated to the engineering of Microsoft Windows 7
Thanks to everyone who provided comments and sent me mail. I definitely appreciate the discussion we have kicked off. There’s also a ton of energy in our hallways as this blog started. It seems like a good thing to do to start off is sort of an introduction to the Windows development team. This post provides an overview of the team that is represented by this blog.
Before diving into the main topic, let’s talk a bit more about what to expect from this blog. First a few words on the comments and emails I’ve received. I’ve received a ton—most of the weekend was spent reading emails and comments. There are definitely some themes. I would say by and large the reception has been very warm and we definitely appreciate that. The most frequent request was to discuss Windows performance and/or just “make Windows faster”. There’s a lot to this topic so we expect to talk about this quite a bit over the next months. There are many specific requests—often representing all possible sides of an issue such as some folks saying “please get rid of (or don’t do) <x>” and then other folks saying “whatever you do it is really important to keep (or do) <x>”. A big part of this blog for me personally is having the discussion about the multiple facets of any given issue. Even something that sounds as binary as performance proves to have many subtle elements. For example, some folks suggested that the best thing for boot performance is to not start anything until idle time and others suggested that the delay loading feels like it slows them down and still others have suggested that the best approach is to provide a startup manager that pushes everyone to choose what to start up. All of these have merit worth discussing and also demonstrate the subtlety and complexity of even the most straight forward request.
Second, much to the surprise of both Jon and I a number of folks questioned the “authenticity” of the post. A few even suggested that the posts are being “ghost written” or that this blog is some sort of ploy. I am typing this directly in Windows Live Writer and hitting publish. This blog is the real deal—typos, mistakes, and all. There’s no intermediary or vetting of the posts. We have folks on the team who will be contributing, but we’re not having any posts written by anyone other than who signs it. We will us one user name for all the posts since that keeps the blog security and ownership clear, but posts will be signed by the person that hit publish. (If I participate in the comments I will use my msdn name, steven_sinofsky.)
And third, what frequency should folks expect and when do we get to the “features of Windows 7”. When we wrote that we would post “regularly” we meant that we don’t have a schedule or calendar of posts and we don’t want to commit to an artificial frequency which generally seems inconsistent with blogging. We do expect to follow a pattern similar to what you have become familiar with on the IEBlog. FWIW, on my internal blog no one has yet accused me of not contributing enough. :-)
As we said in the introductory post we think it will be good to talk about the engineering of Windows 7 (the “how”) and the first step is establishing who the engineers are that do the engineering before we dive into the product itself (the “why” and “what”).
So let’s meet the team...
It is pretty easy to think of the Windows team as one group or one entity, and then occasionally one specific person comes to represent the team—perhaps she gave a talk at a conference, wrote a book or article folks become familiar with, or maybe he has a blog. Within Microsoft, the Windows product is really a product of the whole company with people across all the development groups contributing in some form or another. The Windows engineering team “proper” is jointly managed by Jon and me. Jon manages the core operating system, which is, among many things, the kernel, device infrastructure, networking, and the engineering tools and system (all of which are both client and server). I am part of the Windows client experience team which develops, among many things, the shell and desktop, graphics, and media support. One other significant part of the Windows product is the Windows Media Center which is a key contribution managed along with all of Microsoft’s TV support (IPTV, extenders, etc.).
There’s a lot to building an org structure for a large team, but the most important part is planning the work of the team. This planning is integral to realizing our goal of improving the overall consistency and “togetherness” for Windows 7. So rather than think of one big org, or two teams, we say that the Windows 7 engineering team is made up of about 25 different feature teams.
A feature team represents those that own a specific part of Windows 7—the code, features, quality, and overall development. The feature teams represent the locus of work and coordination across the team. This also provides a much more manageable size—feature teams fit in meeting spaces, can go to movies, and so on. On average a feature team is about 40 developers, but there are a variety of team sizes. There are two parts to a feature team: what the team works on and who makes up a team.
Windows 7’s feature teams sound a lot like parts of Windows with which you are familiar. Because of the platform elements of Windows we have many teams that have remained fairly constant over several releases, whereas some teams are brand new or represent relatively new areas composed of some new code and the code that formed the basis of the team. Some teams do lots of work for Server (such as the VM work) and some might have big deliverables outside of Windows 7 (such as Internet Explorer).
In general a feature team encompasses ownership of combination of architectural components and scenarios across Windows. “Feature” is always a tricky word since some folks think of feature as one element in the user-interface and others think of the feature as a traditional architectural component (say TCP/IP). Our approach is to balance across scenarios and architecture such that we have the right level of end-to-end coverage and the right parts of the architecture. One thing we do try to avoid is separating the “plumbing” from the “user interface” so that teams do have end-to-end ownership of work (as an example of that, “Find and Organize” builds both the indexer and the user interface for search). Some of the main feature teams for Windows 7 include (alphabetically):
I think most of these names are intuitive enough for the purposes of this post—as we post more the members of the team will identify which feature team they are on. This gives you an idea of the subsystems of Windows and how we break down a significant project into meaningful teams. Of course throughout the project we are coordinating and building features across teams. This is a matter of practice because you often want to engineer the code in one set of layers for efficiency and performance (say bottom up), but end-users might experience it across layers, and IT pros might want to manage a desktop from the top-down. I admit sometimes this is a little bit too much of an insider view as you can’t see where some interesting things “live”. For example, the tablet and inking functionality is in our User Interface Platform team along with speech recognition, multi-touch and accessibility. The reason for this is the architectural need to share the infrastructure for all mechanisms of “input” even if any one person might not cross all those layers. This way when we design the APIs for managing input, developers will see the benefits of all the modes of user interaction through one set of APIs.
The other aspect of our feature teams is the exact composition. A feature team represents three core engineering disciplines of software development engineers (sde or dev), software development engineers in test (sdet or test, sorry but I haven’t written a job description externally), and program managers (pm). Having all three of these engineering disciplines is a unique aspect of Microsoft that has even caught the attention of some researchers. In my old blog I described dev and pm which I linked to above (I still owe a similar post on SDET!).
The shortest version of these roles is dev is responsible for the architecture and code, pm is responsible for the feature set and specification, and test is responsible for validation and the ultimate advocate for the customer experience. Everyone is responsible for quality and performance, each bringing their perspective to the work. For any given feature, each of dev, test, and pm work as a team of peers (both literally and conceptually). This is a key “balance of power” in terms of how we work and makes sure that we take a balanced approach to developing Windows 7. Organizationally, we are structured such that devs work for devs, sdets work for sdets, and pm works for pm. That is we are organized by these “engineering functions”. This allows for two optimizations—the focus on expertise in both domain and discipline and also the ability to make sure we are not building the product in silos, but focused on the product as a whole.
We talk about these three disciplines together because we create feature teams with n developers, n testers, and 1/2n program managers. This ratio is pretty constant across the team. On average a feature team is about 40 developers across the Windows 7 project.
We also have core members of our engineering team that work across the entire product:
Some have said that the Windows team is just too big and that it has reached a size that causes engineering problems. At the same time, I might point out that just looking at the comments there is a pretty significant demand for a broad set of features and changes to Windows. It takes a set of people to build Windows and it is a big project. The way that I look at this is that our job is to have the Windows team be the right size—that sounds cliché but I mean by that is that the team is neither too large nor too small, but is effectively managed so that the work of the team reflects the size of the team and you see the project as having the benefits we articulate. I’m reminded of a scene from Amadeus where the Emperor suggests that the Marriage of Figaro contains “too many notes” to which Mozart proclaims “there are just as many notes, Majesty, as are required, neither more nor less.” Upon the Emperor suggesting that Mozart remove a few notes, Mozart simply asks “which few did you have in mind?” Of course the people on the team represent the way we get feature requests implemented and develop end to end scenarios, so the challenge is to have the right team and the right structure to maximize the ability to get those done—neither too many nor too few.
I promised myself no post would be longer than 4 pages and I am getting close. The comments are great and are helping us to shape future posts. I hope this post starts to develop some additional shared context.
First off, I'm so glad that you guys are being a little more approachable. Microsoft as a whole has always seemed a little aloof from it's users, I'm hopeful that this will lead to a really great OS.
Now I've got Vista, XP and loads of Linux and BSD OSs and I've tried them all (well the Linux and BSD stuff that'd work... or that I could get to work, I should say). What I concluded from the tests was that overall Windows is still the best. It's the only one that really works reliably in all arenas (Well unless you can code in which case Linux is probably better but then aren't you writing the OS to do what you want as you go?). Vista has some great improvements, I have plenty of RAM and find it vexing why I'm waiting whilst a ton of spare power is left unused, but it seems hampered by the amount of things it's trying to be ready for.
To my mind the next Windows release should be capable of all the things that Vista can do, the thing I'd change is to make it so that it's not using up resources needlessly to try and be ready right now for everything. Make the install more replete with options to opt in and out of the various add ons. Also I'd like to see an improved version of Vista's resource throttling so that the OS can eat huge amounts of resources when needed but also shrink back to nothing when it's not doing anything.
Anyhow I look forward to getting my hands on the new windows and making my mates howl as I once again break down my computer for a tinker :)
When I first read about Windows 7 in a computer magazine, the authors of the article were all about this new gadget, another new tool, one-click web publishing ect...
And I said, oh dear, seems like Windows7 is going the same bad way as Vista.
Vista has all kinds of features, 3D desktop and I don't know what else. But an operating system has ONE core function, and that is to operate the system. And this function Vista fulfills most unsattisfyingly. It is slow, and instable.
Now I certainly do understand that I am not Microsoft's only customer, and that there are many people who want those tools. So here is my suggestion:
I would love to see Windows7 being a pure operating system. That is, the only thing it does, should be: Manage the hardware, organize the files. That and nothing more. Nothing. no Paint, no Calculator, no windows messenger no internetexplorer, no mediacenter, no word pad, no one-click publishing, no adressbooks no 3D desktop, no windows zooming up from the taskbar (in terms of powerpoint animations: all windows should simply "appear" one second: no window, the other second: a window) no transperacy effects, no windows firewall, NOTHING. simply: run the hardware (that is configure networksettings, manage the totally encryptet harddrive (the one vista function I miss in XP)) and give me the most simple graphic user interface you can design to organize my files and my hardware.
But this core function, the operating system has to fulfill FAST and STABLE. I want it to do nothing but that, but that I want it to do always, reliably and under whatsoever circumstances might come.
If I want my computer to perform anything else but that, I can install some software designed specifically to do that one Job I want it to do.
No as I said, I understand that many people do want these tools. So why not let the user choose during the installation of Windows what he wants to have installed and what not. And when I write not installed, I mean not even one single bit concernig this programm on the harddrive. Windows XP sure does allow you to "deactivate" Messenger, outlook express etc... but this does not mean that every BIT of the programm on the harddrive is erased.
I want a computer with not one single bit that I don't need on the harddrive and with no single unnecessary process running. Or in Terms of the ToyotaProductionSystem: a LEAN Computer
"and launch this software when I want the computer to the Job, and shut this software down again after it has done the Job"
"If I want my computer to perform anything else but that, I can install some software designed specifically to do that one Job I want it to do."
I do like Vista as it is right now, and I think Microsoft made Windows Server 2008 perhaps the best OS *ever*.
That means 95% of the work is already done for Windows 7. Because of that I have both high and low expectations for the next release.
High, because you already have a good foundation, so now you can concentrate in aspects of performance, usability, UI consistency, etc.
Low, because I really can't imagine such a greater leap to from Vista/2008 to Windows 7 that the one already made from XP/2003 to Vista.
Is it Fully in Manage Code or not ?
I am loving the blog so far, it's a perfect amount of interaction with the community and comes directly from the source, hopefully it will clear up some mis-understandings.
One thing I am tired of hearing when I read differnt blogs and such is the term (Win FS). I have heard so many different thoughts behind this term that no one really knows the correct answer.
Please do an in dept post about Win FS and finally set the record straight about how much of it is included in Vista (if at all), explain exactly what it is and how it's different, exactly what the state of it is, what the future holds, and anything else important to clear up the air.
I'd really like a post dedicated to it, so next time someone comments on Win FS on some blog or forum I can paste the link to the blog post by you, to say, "this is actually what Win FS is".
Please make Windows 7 the first version of Windows that doesn’t need antivirus, antispyware, or any of the other performance killers we absolutely have to install on previous versions of Windows.
Be advised that I for one do not use any indexing service, and I do not know of anybody else who enjoy wasting precious CPU capacity on a feature I neither want nor need. Just to make everything perfect a lot of companies have bundled similar functionality with their products, so my files are thoroughly indexed, and it’s just about impossible to permanently disable them all. Search and organize sounds like a nightmare.
This does not mean that indexing should be removed altogether, but repeatedly scanning and indexing everything on the disk isn’t what anybody really wants.
I personally hate pathnames like the following:
C:\Documents and Settings\All Users\Documents\RAD Studio\5.0\Bpl
C:\Program Files (x86)\Microsoft SQL Server\80\Tools\Binn
My current PATH is nearly 1800 characters because everybody thinks that their product is so great that users couldn’t possibly require a slew of other tools installed on the computer.
One possible solution would be to embed the required paths for a program in the shortcut used to invoke the program.
WPF is a great step away from the legacy of Windows 3.0 (and earlier versions). Please let us access similar functionality from regular C and C++ programs. It’s just a scene graph so that shouldn’t be too hard.
Install MSMQ by default – it’s so useful, but for some odd reason MS have in their infinite wisdom elected to make this part optional. RIAs would be ever so much easier to develop if we could assume that this functionality was available – even on mobile phones.
The world is asynchronous and multithreaded – programmers who don’t know how to employ synchronization primitives should not be protected from their folly on the OS level. (Like the single threaded apartment model and the scripting APIs).
It would be awfully nice if the windows message queue was redesigned to accept variable length messages. Memory should be managed by the queue and GetMessage should just return a pointer to a message header with a variable length buffer that was valid until for that thread alone until the next call to GetMessage. It should also be possible to service such a message queue from multiple threads. I know this can easily be achieved using other mechanisms, but the point is to significantly reduce the number of calls an application has to make to the kernel.
A Window, or any other UI construct, should not belong to a thread. It should be the other way around where the programmer could accept the creating thread as the default execution context for the UI construct, or if he so chooses assign one or more threads to queue associated with the UI construct. The UI construct should in other words be associated with a queue, and a queue should be serviced by one or more threads, possibly from an OS level threadpool.
Threads should have their own resource managers, so that exiting a thread would also perform cleanup of handles and memory allocations done purely in the context of that thread. Done right this could do wonders for server development. Something like:
long WinMain(hResourceManger theDefaultResourceManger)
hResourceManger theThreadResourceManger = CreateResourceManger(theDefaultResourceManger);
hThread theThreadHandle= CreateThread(theDefaultResourceManger, theThreadResourceManger, &Function, etc…
long Function(hResourceManger theThreadResourceManger, etc…)
hMem buffer = AllocateMemory(theThreadResourceManger,4096);
hFile f = OpenFile(theThreadResourceManger,”somefile.txt”,etc…);
// Exiting the thread at this point will not cause any resource leakage
Hopefully we will see some really programmer friendly improvements to Windows, they are much needed – both by the developer community and Microsoft. Eye candy is nice, but it is nothing more and nothing less. If Microsoft wants to sell loads of Windows 7 they have to provide something significant and useful. C and C++ is still king when it comes to serious development –.Net is nice enough but neither Microsoft, Oracle or IBM is developing core functionality of their enterprise level servers in managed code; they do support .Net which is nice, but C and C++ is still their primary development languages for large scale application development.
Making the life of C and C++ developers easier, and introducing APIs that are so useful that migrating to Windows 7 makes economical sense all around could do far more for the success of the new OS than just about anything else I can think of.
If there is one thing that I really miss about Media Center and the Media Player is the support of codecs. I always need to install codec packs for viewing my videos. Several of my friends choose not to use media player/center and instead they download VLC Player. If you integrated the most common codecs in the Media Center/Player you will get more users. An integration of the codecs in the K-Lite Codec Pack would be a perfect choice!
Could you explain the interesting turn of events, Microsoft has confirmed that Windows Server 7 will be a minor release, which is rather confusing to me. The NOS will beer a similar name to the now on the market Windows Server 2008 only with a slight addition 'R2'. What was so hard in going with Windows Server 2009 or Windows Server 2010? The out of date naming scheme would to me reflect badly on the products marketing and significance to those interested in upgrading.
I am confused to be honest with you, but as Microsoft continues to build this thing, it seems that there will be more confusing turns along the way. My understanding and I hope this is it, but it seems the features in Windows 7, client and server will reflect version 7 not the core part of the 'NT Kernel'. I want the next release of Windows to clearly reflect meaning to the end user, I just don't want it to be some release that is edging on almost sounding like a glorified Service Pack.
A very informative post that goes into the intricacies of building Microsoft Windows, Steven has really given a good perspective on the people who are responsible for delivering the features and building the platform that both end users/developers can use to build products and be productive from. Steve pointed the irony of what so many people seem to despise about Windows its size, its too big, too big to manage, yet the amount of comments and request seem to reflect the growing need based on customer demand for more to be added to Windows. Relevant issues the Windows 7 Team have discussed based on what users are asking about is to make Windows even faster, and it is indeed a challenge, but I am glad to see it become a priority in this release.
When Steven mentioned the indexer, he was talking about the Windows Search service, not the old "indexing service" that is an optional install for legacy compatibility.
The Windows Search service already integrates very tightly with the filesystem and does not "waste CPU cycles." It mainly uses CPU and I/O time that would otherwise be *wasted* doing nothing.
It also does not but "repeatedly scan and indexing everything on the disk" as you claimed. It only indexes files when they are added or changed.
We've also shipped the Windows Search 4.0 update, which contains a great many improvements to the indexer. If you have problems with it, or feel it has a negative impact on your system performance, please post a comment on my blog or in the Windows Search support forum on MSDN.
I have just a request,
I'm tired of programs like vlite to spoil the OS of all the unused features it preload, is it possible to have similar tool during OS installation?
Standard: typical installation average features installed
Expert: let you choose wich macro features you want to install, like the basic personalization of Office (ex I want only word/excel)
Super User: let you choose in depth every single component that you want to install and run (with a decent explanation of what that component do and for wich services it is usefoul)
You sound like you are referring to an article that went around the internet about how nice and compact Mac OSX is because it contains no legacy code and what not. Paul Thurrott proved that article completely false and since Mac is based on UNIX, it is in fact older than the NT system. Plus, as he explained, what is the point of having a compact operating system with the abundance of large and cheap hard drives these days?
Also, I love the fact that the folders no longer contain "My" this and that and are no longer all inside of the Documents folder. Music and videos are not documents, they are media and should be kept separate. That said, I think that there could be changes to the file structure made by incorporating WinFS.
Finally I think that the registry should be removed altogether. It is a big weak point for Windows. Something similar to how Mac splits up the registries instead of having one central location would be nice.
Actually, you can make an Amadeus analogy here, but MS is not Mozart, they are Salieri, recognizing their own mediocrity and fuming while the young wunderkind (Apple) keep creating inspired and elegant works of pure genius :)
much of the problem as I see it is perceived delays in getting to our information.
slow boot times are maddening.
slow responses from explorer are equally maddening. why on each visit to the control panel do the icons not just appear. why do we have to search for what icons to display each time? surely this should be a fairly set area, so doesnt need to be checked off against a list each time?
going to the explorer view is similar. each time is greeted with a check to see if the drives are the same!
now since I spent money on a really fast pc this isnt as bad as on my laptop, but it could be so much better IMHO. My Macbook is of similar spec to my windows laptop, but the responsiveness is so much better. I coudl shut down the Macbook, add more memory and start it back up again in the time that windows was still shutting down!
it's the same with SQL I notice, not that this is part of the 7 development, but if you want to import data it spends an age working out what formats you might want to be able to import, which in reality are largely going to remain the same throughout the life of the pc!
I have great hopes for 7, as I did with Vista until I got on the beta program!
One question I'd like to see answered is "Why Windows 7"? What problem are you trying to solve?
Is to fix Vista? Is it simply to have a new product on the market?
I have to say that I'm distressed to see how large a team is working on the project, however. How can it help but be complicated with so many cooks?