Software Test Engineering @ Microsoft

Differences between Encarta Premium 2006 and Student 2006

2005-08-08T23:09:00Z

The back of the box for Encarta Premium 2006 has a nifty feature chart that I've been unable to find online. Here's a posting of that chart, containing the differences between the North American versions of Encarta Premium 2006 and Microsoft Student:

	Encarta 2006 Premium	Microsoft Student 2006
Trusted Content
Articles	64,000+	66,000+
Photos and Illustrations	26,000+	26,000+
Videos and Animations	300+	300+
Sound and Music Clips	3,000+	3,000+
Map Locations	1.8 million	1.8 million
Links to Web Sites, Preselected by Editors	29,000+	29,000+
Interactive Resources
Videos from Discovery Channel	Yes	Yes
Interactive World Atlas	Yes	Yes
Encarta Kids	Yes	Yes
Dynamic Timeline	Yes	Yes
2-D/3-D Virtual	Yes	Yes
Virtual Flights	-	Yes
Read Aloud	Yes	Yes
Games	Yes	Yes
Useful Tools
Encarta Search Bar	Yes	Yes
Dictionary, Thesaurus, and Translation Dictionary	Yes	Yes
Quotations	Yes	Yes
Internet Resources
Web Companion	Yes	Yes
Article Updates	Yes	Yes
Live News, Radio, Weather	Yes	Yes
Homework Resources
Student Tools for Microsoft Office	-	Yes
Graphing Calculator Software	-	Yes
Online Math Homework Help	-	Yes
Project Starters	-	Yes
Literature Guides (Book Summaries)	-	Yes
Style Guide	-	Yes
Additional Features	-
Hard Disk Install	Yes	Yes

-Greg

I Love the MVP Program

2005-04-15T08:07:00Z

Encarta's MVP, Stephen Stakey, has just launched the Encarta SupportZone- you can find it here. As he says, it's a place to "find how-to guides, videos, and links to other Encarta-related websites". He's already spent a lot of time in the newsgroup, dug up a lot of the pertinent information that's come over time, and conglomerated it into a centralized location.

This is a great example of how cool the MVP program can really work. Stephen's active on the Encarta newsgroup, has been involved in Encarta Betas for *at least* as long as myself, and does a great job keeping in touch with the product team. It's really enlightening to have a user on the outside who is a great contributor to the product and is a big fan of the app that we spend so much time working on.

Love the site, Stephen!

-Greg

Encarta Blog is up and running!

2005-03-23T23:29:00Z

The Encarta Team blog is now up and running with it's very first post. Take a look here!

The IE Team blog is currently the best team blog I've read, but I hope our Encarta blog is ultimately of similar quality.

BTW- Reading the IE blog it really came in handy today. I got to teach myself how to use the Fiddler powertool during an internal MSN Encarta bugbash of the MSN Encarta Beta website. It's one of the best testing tools I've ever used- I was able to throw a host of bad inputs into some of our Edit boxes, watch how the inputs are handled by the server, and then "fiddle" with the data at various times during a communication to see how extensive error handling is implemented.

I didn't find any bugs using the Fiddler in my first attempts, but I'm sure it'll help me find some issues in the future.

Greg

Reasons to be a Test Developer

2005-01-26T02:40:00Z

Steve Rowe has an excellent blog post on Three Reasons To Consider Being a Test Developer (aka SDET). Nihit Kaul also follows up with some interesting comments here.

One piece I'd like to add about the advantage of being a Tester (whatever the job title : ) is that you'll learn your underlying product literally *better than anyone*. It's true that PMs have put their sweat and blood into creating detailed Specs, and Devs have literally built the application from the ground up and have an extensive knowledge of underlying algorithms- it's their own code, after all! Ultimately, I would have to argue that the Tester of a feature is going to have the most knowledge of the in's, out's, upsides, and downsides of the features.

Why?

It's our job! We've spent such a long time hacking away and learning our feature, that we've inadvertantly picked up full expertise from a true power user's perspective. When we think of a user scenario that's outlandish but possible and should be supported, a Tester is the only member of a product team with the freedom to go ahead and try it. PMs have their hands tied, and Devs are already busy off writing more code. An SDET is free to write automation to learn exactly where the product works, and exactly where it doesn't. And, they're probably happily doing some exploratory testing when they become inspired to search for a potential bug that may have been hiding.

From my own experience, when a question comes up in a meeting like "How do we handle environment condition X when executing feature Y?" - all eyes at the table will drift to the Tester, and they'll usually know the answer. :-)

Greg

Why does Software Crash #2 – The Access Violation (continued)

2005-01-18T15:47:00Z

In our last post (here), we took a look at an Access Violation of a user mode application attempting to access memory in kernel mode address space.

It's interesting to note that we can't just go reading and writing memory in the kernel- if we try to blindly access another application's memory, we'll get the same AV error (on an NT OS, anyhow). Let's take a look:

int foo;

int * ip = (int*)16777216; //Address 0x01000000, in another process' address space

foo = *ip; //Crash!

This will crash with the following error:

Unhandled exception at 0x004173c8 in cpractice.exe: 0xC0000005: Access violation reading location 0x01000000.

A few notes:

On most machines, you'll see a Watson dialog instead of this unfriendly crash dialog. I've used the `unfriendly' crash dialog here for demonstration purposes.
AT posted some interesting comments in the other post here. The win32 API has tools for sharing memory between processes, and the above code was legal on DOS, which was without a protected memory space.

Why does Software Crash #1 – The Access Violation

2005-01-13T00:58:00Z

Pop quiz: what does this line of code do when executed?

int foo = (*((int*)0));

If you’re an astute reader, you can solve the answer just by reading the title of the blog post. But, more interestingly, let’s forget about that buzzword and analyze exactly what’s happening. The access violation is probably the most common crash in unmanaged software, so let’s break it down piece by piece to discover what’s happening.

int foo;

int * ip = NULL;

foo = *ip; //crash!

On the first line, we declare an integer variable named “foo”.

The second line declares a pointer to an integer named “ip”, and initializes this value to NULL (which is simply a fancy term for 0). Pointer variables “point” to a location in memory. In this case, we initialize it to point to the address NULL, or 0, of memory. Location “0” is in area of memory reserved by our operating system.

The third line attempts to grab the value of address 0, which is prohibited, and assign it to our variable “foo”. Our operating system smartly catches this and says “No way- you can’t see my reserved memory location!”. Our operating system shuts down this badly behaving application with an error like the following:

Unhandled exception at 0x004173c8 in cpractice.exe: 0xC0000005: Access violation reading location 0x00000000.

This error message is a bit cryptic, but it's understandable if broken down. 0xC0000005 is the error code designation for an Access Violation. 0x00000000 is the location that we tried to read (this is our NULL value!). And, 0x004173c8 is the arbitrary memory location where our application happened to be running at the time of the crash.

AV's can happen for both reading and writing, which is part of the reason that they are so common. Pointers in general are one of the most challenging topics in computer science, which is yet another reason this crash is so commonly seen.

-Greg

“Why does Software Crash” – The series!

2005-01-12T02:14:00Z

Starting this week, I’ll be starting a series on the nuts and bolts of software crashes. Complex software has a seemingly infinite number of ways that can force a dreaded crash dialog, and I’m up for the task of enumerating why many of these things happen.

I plan on starting from examining short C programs that crash, and working my way up to broader environment issues that can cause instability on a user’s machine. As always, feel free to ping me as I go along and I’ll address any questions as they come up (and I’m sure I’ll have some questions myself as I write this series!)

-Greg

White Box security practice

2004-11-03T16:12:00Z

Mike Howard has posted some examples of buffer overflows by incorrect use of "safe" string functions. It's an enjoyable exercise- take a look here!

Greg

Difference between STE and SDET roles at Microsoft

2004-10-07T00:38:00Z

What’s the difference between a Software Test Engineer (STE) and a Software Design Engineer in Test (SDET)?

The Microsoft career site has two excellent definitions for the Software Test Engineer and Software Design Engineer in Test roles at Microsoft. In my experience, these clearly defined role definitions have very vague applications here at Microsoft. After all, aren’t we all just assuring that the software that we ship is the highest quality possible?

Historically, it seems to have made a lot of sense that these two positions had distinct roles. Years ago, Software Test Engineers were testers who had limited programming experience. I’ve heard particularly frightening stories from Microsoft old-timers where Software Test Engineers were used as monkey labor- “Run this test fifty times, and call me if this breakpoint gets hit”. STEs didn’t need a strong CS background to get their jobs done- they just needed a lot of patience, and basic understanding of *using* computers.

The Software Design Engineer in Test role was reserved for Testers who were capable of writing tools and/or automation. I’ve also heard stories from other old-timers about STE hires that write tools in their first half year on the job having their job title switched by management to SDE/T.

It’s easy to understand the difference between STE and SDET with a black and white example. If you’re testing an API, then yes, you’ll surely be an SDET (how can you do any testing without writing a line of code?) Conversely, if you exclusively black box test a user interface (without tools/automation), then you’d have the job title of STE. The gray area between these two extreme examples is huge, though. I don’t know any STEs who don’t end up writing at least *some* code. In my personal experience, I’ve spent the past three months on the job writing automated test cases. Like many of my fellow STEs, this doesn’t strike me as uncommon. I'm sure there are plenty of STEs that in practice write more code than SDETs! This difference is particularly pronounced across different product teams at Microsoft.

Management seems to be pushing for a unification of these two roles, probably for this reason. Both roles dedicate themselves to product quality, both are reliant on strong fundamental understanding and application of CS principles. A quick look on the career website seems to back this up- there are currently 326 active postings for SDETs, compared with 44 for the job title of STE. This doesn’t mean software testing is going away- I only see it as a reflection of the increasingly technical nature of testing software at Microsoft. It looks like the job title that the company is shifting to is SDET, which is more a mouthful but also more impressive sounding.

In short, don't be surprised if the name of this blog becomes "Software Design Engineering in Test at Microsoft" to match the way that the management wind seems to be blowing. If it comes to that, hopefully Chris and I can come up with less unwieldy blog title! :-)

-Greg

A Great Use For Blogs

2004-09-09T18:26:00Z

I'm going to head off today and talk about something in the blog world, and not at all about testing.

Every now and then I hear of a new use for blogs that I really like. Most of us are used to the standard paradigm, I write in a blog so my family and friends can keep up with me easily, I write in a blog to share my ideas about the discipline I work in. But I was talking to a friend of mine who came up with a great use for the technology, and thought I'd share it here.

He's the computer teacher at a private school down in California, and for this year he decided it would be great to set every teacher up with a blog. They can use this to post homework assignments, communicate status updates, and update parents on what they'll be covering in class over the next week; all kinds of stuff.

One of the teachers even took it further. She enabled comments on the blog and the first day had her kids go out and do some online tests and post the results as comments. This got all the kids involved and interested in the blog. It was a great success.

I was talking to him this week and he said this has been a huge hit with everyone. He used Blogger to set up all the pages, and raved about their interface (my personal blog uses Blogger, I'm also a big fan.) Most notably, all the teachers have to do is send an email to a specific address and it gets posted up on the blog, complete with formatting.

It always makes me happy when blogs make roads into non-technical sectors. I think they're a wonderful communication tool and have lots of great uses ahead of them. Stories like this remind me they have a lot of places to grow to. I look forward to the day when people stop having to ask what a blog is, then give a puzzled look as I try to explain it. I fully expect it to be just like the web, email, instant messaging, and plenty of other communication tools - soon enough blogs will just be another tool we use to keep in touch and share information.