AdamU's WebLog : Recruiting

Will it Blend? That is the question!

adamu — Thu, 01 Feb 2007 22:47:00 GMT

We're having fun watching the BlendTec commercials since we've announced that the product code named Sparkle is now Blend.

Since my last post 6 months ago in July where I visited a company in Raleigh NC, I've been amazingly busy.

I've visited Roxio in Santa Clara, CA.
I've done a recruiting trip to U Penn in Philly
I've done a recruiting trip to Shanghai

Most importantly, I've shipped the Interactive Designer Sep CTP, the Blend Beta 1 in Dec, the Blend Beta 1 Japanese in Dec, and the Blend Beta 2 in Jan.

I hope that everyone that's seen Beta1 and Beta2 come away with how deeply the product team cares about quality, and how hard we're working on delivering a great user experience. If you've visited the sparkle/blend newgroup, I hope that you understand that the majority of people answering your questions are people who've been working for years on building this product, and that they love hearing how you are trying to use the product, and are anxious to take your issues and try to resolve them.

We've got two more big milestones coming soon to hit for Blend: RC1, and English RTM. And then the RTM for an additional 8 languages (German, Japanese, French, Italian, Spanish, Korean, Simplified Chinese, Traditional Chinese). For me personally, I've also got two more recruiting trips planned in the next 3 weeks: MIT and CalTech.

So I'll likely be scarce for a while, again, but only because I'm working hard on hiring great people, and driving us to completion on a great product for Pro Designers to build rich compelling UI experiences.We fixed almost 700 bugs between Beta 1 and Beta 2. We added a couple key features that we were missing.

Adam

We don't make the software you use, we make the software you use better....

adamu — Fri, 23 Jul 2004 12:13:00 GMT

BASF used a variation on the above as it's corporate tagline. Michael Hunter, the technical lead for my test team, used this as part of his auto signature for quite some time. It should be the motto for every software test/qa organization.

For those who don't know him, Adam Barr worked for microsoft from 1990 to 2000, and wrote a book called proudly serving my corporate masters. He commented on a comment to a post I'd made about testing at microsoft. Here's a couple excerpts from his comments.

So what I am trying to argue in the book is that testers are just as important as developers, in fact they may be more important, but it's not because they are as good developers as the developers, it's because they are good testers! That should be reason enough to respect them. In fact I make the claim that Microsoft started out in the "era of the developer", moved to the "era of the program manager" around 1990 when Windows 3.0 shipped, and has now moved to the "era of the tester" -- except nobody realizes this, and it is hurting the company.

Anyway you can read the book (it's linked to online from my website if you don't want to buy a copy) to get the full argument, it's scattered throughout chapter 3, on pages 40-60. But please understand that I was trying to raise testers up to equality with dev and PM. I'm not sure what "He further rants negatively about testers especially in chapter 4" means but I don't recall doing any negative ranting about testers. I rant negatively about the ATTITUDE towards testers. The quote on p. 50 about "for a variety of reasons they are viewed as lower on the pecking order than program managers and developers" is not me bragging about the way things should be, it's me lamenting about the way they are.

Adam, thanks for clarifying your books points. I think that Adam and I are in agreement about the way things were, and that attitudes needed changing and understand that testing is currently a bottleneck in delivering quality software. I believe that the companies views have changed significantly in the last 2-3 years. Bill Gates quote from May 2002 : http://www.informationweek.com/story/IWK20020517S0011

INFORMATIONWEEK: When we were on Microsoft's campus in March, it became clear that some people there think they've been developing quality software for years; in other words, it's not a new concept to them. And we got a sense that a few people were a little bit defensive about it.

GATES: Well, let's separate out quality from security. Microsoft in terms of this quality stuff--we have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We're more of a testing, a quality software organization than we're a software organization.

Now where Adam Barr and I may disagree: I want to hire people who are both great testers as well as being great developers; I don't see these skills as mutually exclusive. I took my current job to start a new group 2 years ago. From day one we've worked to hire people who have strong testing experience or aptitude, and are skilled developers. Given the resource constraints, and the longevity of software servicing, as a manager, I must pursue 100% test automation as aggresively as possible, and have the automation completed as close to feature completion as possible. To build the framework necessary to deliver on this goal, you have to have strong developers. It's simply not a simple task.

Think about it like this: students at school, or developers from other companies, have lots of knowledge on how to solve typical problems in software engineering. How to design a 2 tier or 3 tier data bound application; how to draw bezier curves; how to implement pixel shading support, etc, etc. There are books for every aspect of software engineering. Need a sort? Just look it up. How about jpeg compression? Again, just look it up. I'm not saying that everything developers do is regurgitate code. There is room for lots of creativity. But there just aren't books or classes yet on how to write a test harness that can be shared by a development team and a qa team to run both unit tests and functional tests, that runs in process and out of process with respect to the application being tested. How about designing software that automates the installation of Operating Systems and other prerequisites? How about building a verification system that can test that your bezier curve is actually rendered correctly?

That's why I believe that during the next decade, the hardest problems we have in software engineering are in the realm of testing. It may not be glamorous work; the code we write never gets put onto a single customer's computer, yet quality is still the single most important feature we ship in every product. If it works as expected, people will have the ability to love it (of course, it also has to do something they care about for them to actually love it); if it is buggy, people will hate it, no matter how many killer features are in the box. Getting the product to market in a timely fashion at that quality bar, and putting the product in a position to be sustained without continually pulling resources away from the next version, unless you have unlimited resources (hah!), requires you to have quality test automation. To implement that, of course, brings me full circle back to my point on hiring people who are both great testers and great developers.

Preparing for technical positions at MS: part deux: Industry route

adamu — Wed, 07 Jul 2004 04:32:00 GMT

If you've been working (or idling) for a year or more since you hung up your full time academic cap and gown, the microsoft recruiting machine considers you an industry candidate. Many of my comments in the prior post about campus candidates will apply to industry candidates as well. In fact, I would say that ALL of it applies. They still do spare time coding projects outside work; they are constantly trying new technologies and reading to keep themselves technically sharp. The pick up on technology trends and can expound on where they think they should go. They are smart AND hard working. But for industry, the bar goes up (and it should!); they now have practical experience that should translate to a more polished set of skills.

Preparation

So back to how to prepare yourself. If you are a developer and you want an SDE or SDET position at microsoft, the best way to prepare youself is to write code, read code, dream code. There's no secret here. Have it reviewed by others so that you get better. You'll want to take a 4 pronged approach to what you code:

A solid general knowledge across the board in areas like file I/O, 2d/3d graphics, string manipulation, databound apps, whatever you haven't tried, broaden your horizons. You never know what kind of interview question you will get, and having a broad knowledge from which to drawn on will help you.
At least one area of deep expertise. Something that you've written tons of code in, and have much deeper knowledge of the problem space such that you could be a strong resource for your team in this space. You should be, above all, passionate, about this area, whatever it is.
Use multiple languages and learn the pros/cons of each. When you're asked one of my favorite questions “What's your favorite coding language“, you'll be able to answer most correctly “that depends on what the problem is.“ It goes back to the old saying, if you only have a hammer in the toolbox, everything looks like a nail.
Become an expert on your tools and leverage tools to make yourself better. Think about a master trade craftsman. If you asked him why he bought his DeWalt (or Ryobi, or Snap-on, etc, etc) tool, what kind of answer would you expect. Ask him what the advantages are with it over other tools he could be using. Why does he need a special diamond tipped wet saw? Because it ensures clean cuts on hard ceramic tile. For a software engineer, tool knowledge equates to efficient coding. Understanding what your tools are good at, and where you might upgrade demonstrates that master craftsman approach. Why do you use Lutz's Reflector? Why do you write all your code using EMACS? What do you hate about intellisense? How does late binding affect your coding? What 10 features do you wish you could add to your coding environment?

Interviewing

During the interview process, you'll be asked lots of questions, from coding problems to open ended questions designed to test your creativity, to fundamental questions like math/logic based puzzles designed to evaluate your approach to problem solving.

Coding: I would tell you to try writing code on paper or in notepad or on a whiteboard. For today's rad tool environments, intellisense coding is so helpful that it can be crippling you if you need to interview.
Creativity: I don't know how to develop this, either you have this or you don't.
Problem solving: I don't know of a single way to prepare for these type of things, except to have a solid mathematical foundation and be both creative and inquisitive. If you don't feel your math skills are up to par, upgrade them! Generally coding across a broad space can keep you sharp enough to solve these kinds of problems.

And of course, don't be afraid to ask what kind of question this is. Why is a man hole cover round? Is it a logic problem, or a creativity problem? That really depends on who is asking it.

Preparing for Technical Positions at MS: part 1: college campus route

adamu — Tue, 22 Jun 2004 21:23:00 GMT

Matt asked about how to prepare for a technical position at Microsoft. Since there are two separate recruiting engines at Microsoft, I’m going to break my comments up into college candidates and industry candidates to match the paths that people typically enter the company.

College Campus Candidates

I interview a lot of people at college campuses as well as during full interview loops at Microsoft since I took my current position 2 years ago. I've visited 7 college campuses and two foreign countries (Bulgaria and Romania) and interviewed around 200 candidates during those trips, and I have had about 20 or so campus candidates interview for positions during that same period. Obviously, a BS or MS in CS at a school with a good CS program is probably the best way to prepare yourself technically for any technical position at Microsoft. It probably goes without saying, but I'l say it anyway: regardless of your degree or your GPA or your class rank, not just getting the grade, but developing a deep understanding of the curriculum is what will best serve you. That GPA on the resume that’s reviewed by a screener at a recruiting fair, or seen by the schools recruiter, may get you 30 minutes with me, but you'll never get back to Redmond for a full loop if I see that you came out of school with a degree in CS and have gaping holes in your fundamental knowledge.

So back to how to prepare yourself. Here’s some of the things I see in the best candidates:

They are involved in coding activities outside school projects. Coding competitions, cross engineering projects like robotics with a mechanical engineer and electrical engineer, interns writing code for a company, highly involved in local programming user groups, teachers aide for coding classes so that you see a lot of other peoples code, freeware/shareware projects to learn about a new technology – perhaps building an app for a pocket pc to understand how it works, etc.
They keep up with today’s technology, are passionate about it, and can form opinions on where it is headed. A couple common interview questions are
1. “If I gave you $5000 to spend on any technology related items, what would you spend it on?” Be prepared to talk about all the hardware components in a PC if you want to buy one, what software and why you’d buy, and the advantages of your printer or camera of choice over others in the market.
2. “Given $10,000,000 dollars to invest in the tech sector, what would you invest it in?” Wireless, 3d graphics, display technology, what trends do you see, or breakthroughs are you dying for that you think would have a broad appeal?
I’m not sure how you go about changing who you are, but I’ll tell you that the best candidates I see are hardworking and fundamentally smart. I see a lot of 3.8+ GPA folks, and they work really hard to get the high GPA. But too many of them seem to be so focused on getting the grades that they failed to get an education, or they just aren’t that fundamentally smart (or it was a bad day; it happens to everyone), because in my interviews many don't do too well. I’ve talked to other recruiters for other companies, and this is a major difference between Microsoft and other companies. They tell me that the GPA will tell them that they know their stuff and can get things done. Well, for me, that’s half of it. I want that, sure, and I want to hire the person who will actually make us better as a company with their ideas in addition to their energy and effort.

What I look for when someone applies for STE

I mostly look for fundamentally smart people with a real passion for technology who can generate test cases and understand enough about how computers work to understand what code is doing. I also look for determination and inquisitiveness. Then I look at if this person can grow their coding skills. OK, they've never done any OOP, but can they code functionally? When I am making the decision for my team as the Hiring Manager, every STE I hire I believe that they have the coding skills to eventually move to SDET/SDE. Hey, I am in developer tools, I want every person I hire to be an expert on our product, and so I look for the potential to become an expert, which would mean being a developer. That is not the bar across the entire company, so when I am interviewing for Microsoft on campus, I look for the first part (fundamentally smart, real passion, understands computers at a moderate to high level), never gives up, and an inquisitive nature. I generally ask some testing questions, but because there are really few classes that teach how to test software, I am looking for aptitude, not ability. Red Flags: when I see someone come through with a degree in IS rather than CS, I drill very hard on their technical skills. Most seem to make that move because they aren't enjoying or just don't get the coding aspects. If this is the case, you will have a tougher time with me in making the STE bar. Not impossible, but it is more difficult.

What I look for when someone applies for PM

The best PM candidates have a MS/Phd in CS, excellent oral and written communication skills, understand the underlying technology, can build prototypes for proof of concept work, and are great negotiators. Pretty much, the best pm’s could be in Dev or QA if they wanted to be, but they have plus communication skills and a strong customer focus that they would rather focus on tying all the features together at a higher level rather than drilling into one area of technology. Warning signs: When I see someone with a BS in CS, then masters in business, and they want to be a PM. While I think that I understand the logic of this course, it rarely pans out well for candidates. They've lost any tech edge they may have had (they haven't taken a technical course in 2 years), and can’t talk about fundamental coding or design concepts any more. The other red flag: when people haven’t done the leg work to understand what the PM role is. You mean I can immediately manage people out of college? Wow, I want that job! PM's manage features and process, are not people managers (lead PM's would do both). Customer focus isn't enough in this role as well (I see many BS in IS trying for the PM role); you have to know your technical stuff too.

What I look for when someone applies for SDE/SDET

These people love to write code, live it, breath it, bring their laptop to the interview and are probably coding while their waiting for me to be done with the person in front of them in the interview queue. OK, that might be a slight exaggeration, but really, the best way to get technically proficient at a SDE/SDET core competency of writing code is to, well, write code. Lots of it. All the time. Candidates don't need to have perfect syntax, I'm not expecting that, but do they have a firm grasp on the concepts? Do their eyes light up as we talk about optimizations on a simple but elegant solution to a coding problem? Do I see that passion? For SDET, the bar goes up slightly in that I am looking for an additional skill. I look to see if they can test their own code. Sure, everyone tests their own code, right? It’s just part of the development process, right? Well, not exactly. Can they step back from the fact that they wrote the code, and look with a critical eye on their implementation? There are 4 types of developers from a QA perspective.

1. Can write code, can’t test other people’s code, can’t test his own code.

2. Can write code, can test other peoples code, can’t test his own code.

3. Can write code, can test other people code, can test his own code.

4. Can write code, can’t test other people’s code, can test his own code.

#1s don’t last too long at Microsoft. I characterize them in a very general way as either not enough experience or smarts to go beyond just writing code, or that they have such a huge ego that they believe that they never have bugs in their own code, and anything else being written doesn’t deserve their time. #2, they enjoy poking holes in other peoples designs, but have too big an ego to believe that there are holes in their own code. Ideally, we want to hire all #3 types for SDE or SDET, but we really need them in SDET roles. Why? Well, for test code, there is no additional organization that test’s the test code to ensure that it is working correctly, so these people are the last line of defense between the bugs in their code that cause test code to not uncover flaws in the shipping code. #4s are ok as SDE as well, they can test their own code, they just don’t help their peers that much via code reviews. Mostly, these people see one solution to a problem, and when people throw a different algorithm at them to solve a problem, they struggle grasping it, so reviewing someone else’s implementation is challenging for them.

I'll talk more about the industry recruiting route next time around.

You've got Questions, I've got answers.

adamu — Fri, 18 Jun 2004 12:14:00 GMT

I got some questions on my last blog post, so I’ll do my best to answer them now:

Q. How hard is it to get a job at Microsoft?

Well, that’s a tough question to answer. There are hundreds of variables there. If you know someone and they submit your resume as a referral because they know your work, and you do legwork on open positions that you feel would be an excellent match (and you carefully chose which ones you are best suited for, because recruiters dislike nothing more than someone who applies for 27 different open positions just hoping to get any of them), then you’ve significantly improved your chances at getting at least a phone call from a recruiter.

It also depends on if you are in school still, or have been out of school for more than a year, because the recruiting process is handled differently for ‘industry’ candidates and ‘campus’ candidates.

No matter which bucket you fall into, you should read Gretchen and Zoe’s blog to understand all about how recruiting works.

Q. How does it work at Microsoft for a job change to occur (such as Test Engineer to Development Engineer)?

The short answer: you would interview for the job. The long answer: one of the great things about Microsoft is the ability to not only move between teams working on various technologies, but move between job functions as well. All the job openings that we have (did I mention that I have 3 openings on my team?) are posted both internally and externally. If you were to stay within a functional discipline, you would most likely go through what we call an ‘informational’ interview process to see if the job is a good fit, etc. If both the hiring manager and the person applying for the position wished to continue the interview process, then typically a formal interview loop with several people would be scheduled. Sometimes this is waived, but that is strongly discouraged. For functional changes, a formal interview is mandatory.

Q. Is there a central body that determines if someone can code well enough to take a development job, or is up to the individual managers for the different teams?

Coding, while a very valuable skill to have, is just one of many skills necessary to succeed in many jobs, but you really shouldn't focus down strictly on coding, as you need to meet the bar across a broad skill set to succeed. There are people that can code, and have the interpersonal skills of a yak. But to get back to the question, there is no central body; the Hiring Manager for every job makes the final hire/no hire decision at the end of the interview.

Q. What does someone have to do to show he can code?

Again, that’s generally demonstrated during the interview process.

Most of these questions are focused primarily on a move from Test to Development. That can be a common path, but by no means is there any ‘norm’. I’ve seen dev to pm, pm to dev, test to pm, pm to test, dev to test, writers to qa, even moves into recruiting and marketing. Gretchen (and Zoe, who you should read if you are really interested in understanding recruiting and interviewing at Microsoft) made a post on how she loves what she is doing, and that she’s certain that there is another career here at Microsoft for her.

Of course my focus is QA, and as a company, we are working on retaining more senior developers in quality roles. In fact the Technical Lead in Test and the Test Architect positions within QA organizations are becoming more frequent as we build even more complex systems and the technical demands in QA grow proportionately with the complexity of the systems being tested. While the path into management is one road for growth in QA, (and I will readily admit that for some time it was the only path for growth) that is no longer the case.

I have a personal jihad of sorts about the type of people we need to hire into QA. I’ve spent my career at Microsoft in developer tools. In order to be an expert tester for a developer tool, you really have to be a developer. So we’ve always hired a lot of developers on the teams I’ve been on. On my current team, I hired all developers who can test (software design engineer in test, or SDET). Every single person on the team writes code; in fact, they write a lot of code. Sure, we write test plans to define what code we will write; we write design documents for our tools, and we do the occasional buddy test/manual test/bug bash thing. Mostly we focus on test automation for a variety of reasons. There’s lots of opinions on efficacy of automated vs manual testing, and I’ll leave that for a future post topic.