The Value of Certifications

Published 27 October 06 06:54 PM

A friend recently sent me to a blog entry about Microsoft certifications by Oren Eini.  This was really a bummer of a read for me – primarily because I’ve been a follower of his blog for a while and have used his NHibernate generics library in several projects.  That said, there are several points that I do want to address.

1) Oren described a question where the only distinction between the correct and incorrect answer choices was a Boolean value on a method call – and of course, the candidate wasn’t told what that parameter represented.  The comment was that a question like the one described is “fatally flawed”.  Bottom line – I agree – that type of question sucks.  In fact, in the new generation of certs (.NET 2.0 exams) we have been actually using questions like that one as examples of what not to do when we train people how to write questions.

2) Oren described a question where somewhere in the question was a concatenated SQL string and the correct answer was something along the lines of ‘dr.GetInt32(0);’.  So first off, I haven’t seen this question, but from the looks of the description, I’m going to guess that since ‘dr’ looks like a DataReader class instance, the offending query is probably up in the question body.  So while that doesn’t take away from the fact that concatenating SQL strings is a horrible practice, if in fact I am correct about the placement of the offending code, I can tell you why it was probably done that way – to save space.  Plain and simple.  Using parameters would require more lines of code, which would make the question longer, which could potentially cause the candidate to have to scroll – and this is something that we try to avoid.  If anyone knows of the specific item in question here, please validate or invalidate my assumptions here.

So past those specifics, I did want to additionally speak to Neal’s comment.  Neal says:

“Any ‘certification’ that can be achieved by a grad developer being shoved through a week long bootcamp really doesn't give me any faith in the grad (for wanting their food pre-chewed) or the certification (for the complete lack of any real effort or knowledge it takes to acquire).”

Wow – so what are the poor college grads to do?  Are we essentially saying that these folks should basically go out into the job market with their fingers crossed that someone will like the font on their resume better than those of their competition?  And in fact, if certifications are simply evidence that an individual can study a bunch of material, pass a series of tests, and get a dimploma…uh…er…I mean a certificate, then really why go to college at all?

In my opinion, there is a huge difference between having knowledge of something, and having the ability to evaluate a context and make decisions about how that knowledge should be applied – and it is that distinction that has been the driver behind the new 2-tiered structure for Microsoft certifications.  We should absolutely provide a means for college grads to differentiate themselves from peers when going out into the job market.  However, that differentiation cannot carry the guise of experience – and this was one problem we saw in the old MCAD and MCSD programs.  Hence, we created the Technology Specialist) tier of certifications.  So a college grad carrying this distinction would be able to say “I studied about ASP.NET, jumped through the hoops of taking this exam, and can now tell you that I know how to write ASP.NET applications – though I can’t tell you a thing about building a threat model or designing to scale to 100,000 concurrent users.” (much like I said about my computer science degree when I graduated college)  For that type person, we created a separate tier of PRO exams.

So the question in my mind is this.  Is the vision behind the new generation of certs wrong, ill understood, or just not well enough broadcasted?

Filed under:

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# Nathan said on October 30, 2006 2:52 PM:

I love how you took the value of a 4 year degree in computer science and brought it down to the level of a Microsoft certification.  

The difference is that with the degree, a graduate should have the tools to pursue a wide varity of careers in IT because s/he has received an education in the basics of math and computer science.

The bootcamp grad from a certification program is qualified to drag and drop crap into the designer in visual studio, using as many wizards as possible and have no understanding of what is going on behind the scenes or why.

It's worth pointing out that I have yet to write enterprise level software that uses the wizards or generated code.

# hdierking said on October 30, 2006 3:38 PM:

"should" is the operative word - and one that only has meaning because the educational institutions are rigorous enough to produce individuals who live up to the expectation that you describe.  And to jump around a bit - in building business applications, I have used my formal math and CS education about as much as I have used the wizards in Visual Studio (with the one major exception being regular expressions - those are just too fun <g>).

Please understand that I'm not trying to equate certifications to a university degree.  However, I think it's fair to say that in terms of the job market, the two serve at least one similar purpose - they allow individuals the opportunity to show distinction between themselves and their peers.  After all, I can get an arguably better education in math and computer science by simply reading a lot.  However, I still won't be able to compete with someone who has that 4 year degree.

So from your comment, it sounds like one problem is that our exams are able to be passed by individuals who can do little more than "drag and drop crap into the designer".  Is this a fair assessment?

# Matt Stevenson said on November 2, 2006 5:30 PM:

What it comes down to is that one does not need a thorough understanding of what is going on behind the scenes to know how to use a technology.  If that were the case imagine what getting your CS degree would be like.  I will be the first to admit I had no understanding of assembly when I wrote my first C++ program.

The purpose of the Microsoft Certs (from my perspective) is to say that this person knows enough about this Microsoft technology to work on an application that uses it.  

I would say that the root of our little problem here is not the exam itself, but the process of examination in general and attitudes of senior developers.  One can give the same complaints given here to any set of exams.

An example could be taken from an abstract algebra exam.

Question: If you have a Group G of order 6 and a coset of G, H, with 2 elements, what is the index of [G:H].

Anyone in the class could tell you to find the index you just do |G|/|H| or 6/2 = 3

But you would have a lot fewer people in the class who would be able to explain where the formula comes from and the underlying principles at work here.  The fact is the exam isn't over the underlying principles and that is not what the question was asking.  These exams are not over CLR, they are about implementing Microsoft technologies.

Secondly, from what I have seen many developers look down on Microsoft Certs, because in the end you can pass them just knowing how to "drag and drop crap into the designer".  As it turns out someone can just drag and drop in a designer and have a working application.  That was kind of the idea behind the thing: allow people make software easily without the tedious effort of writing code that needs little logic.

I suppose its worth pointing out here that I have not used the designer as well, nor have I used Assembly, Scheme, Prolog, self-written linked lists, binary trees, merge sorts, or integrals in enterprise level applications, but if needed I can.

# jk said on November 12, 2006 12:30 AM:

I'm in the process of studying for my MCTS and, frankly, it's not very much fun trying to learn strategies for navigating badly written multiple guess questions.

As a contrast, I just got my Red Hat Linux certification.  Not only was it a joy (and a real challenge) but the entire exam was hands on: here's the issue, go fix it.  There is absolutely no way I could have passed it with rote memorization, because the only way to pass it was to solve real problems in a real environment.  I can honestly say that I had to prove myself to get my Red Hat cert in ways that reflect real world situations.  I wish the same could be said of a MS cert.

The vision behind the certs is wrong.  They should demonstrate real-world "can do", not ability to learn how to get good at choosing answer A, B, or C.

# jk said on November 12, 2006 10:39 AM:

Just to follow up-- how amazing would it be if the MCST/MCPD tests were actual hands-on tasks?  

Imagine, for example, being given installers for VS + SQL and being told to install everything and implement an application that passes a battery of test cases?

So the "test" that Microsoft would give is simply to provide the appropriate set of failing unit tests: these could certainly be quite complex unit tests, and could easily test anything that the silly multiple choice test covers.

"Passing" is simple: do your unit tests pass?  Test cases could be made to cover any topic: scalablility, cultures, threading, datasets, you name it . . .  As long as my unit tests pass, it doesn't matter if I used wizards or wrote it all by hand.

Now that's an interesting test.  

# M.H. said on April 16, 2007 10:10 AM:

I for one would love it if the exams covered actual scenarios instead of multiple choices as jk described above.

I'm currently training for 70-536 and I'm frustrated to say the least. When running MeasureUp I'm constantly at 65-70%

I did one test swiching between Visual Studio, MSDN and Google and guess what my score was...

My point being that in any normal situation you would have access to those resources - memorizing the entire MSDN Library (wich i percive this test is about) is just silly.

# hdierking said on April 16, 2007 11:24 AM:

So we've definitey heard the request for performance based testing and that is something that we are investing tons of resources in right now.  I am curious about one thing though from your commments...

Do you think that performance based testing will by itself fix the "MSDN memorization" problem?  Originally, I just assumed that it would - I mean how cool is it to fire up VS on a test and go to town??  However, I'm curious as to whether it is equally possible to create a performance based test where you are essentially asking the same type of memorization question in a new (cooler) format.

Here's why I bring it up - one thing I keep hearing over and over when people talk about performance based testing is the word "scenario".  The quality in scenario-based testing that seems to show relevance is that it tests multiple points of knowledge simultaneously, and in context.

I'm not trying to be the buzzkill for performance based testing by any stretch.  Just trying to identify exacty why everyone assumes that it will solve all of life's problems.

thoughts?

Leave a Comment

(required) 
(optional)
(required) 

About hdierking

I am currently the Editor-in-Chief for MSDN Magazine. I joined Microsoft in 2006 as a product planner with the certification team at Microsoft Learning. Prior to that, I spent my career as a developer and later as an architect. My main technology passions include pretty much anything on language theory, agile development, and service-oriented architecture.
Page view tracker