Based on some recent events, I find it necessary to post this as a notification to all those who take certification exams. This applies to certifications across different vendors, not just Microsoft.
During the opening screens of the exam, before you reach the actual question portion, a non-disclosure agreement (NDA) is presented. Like most exam candidates, myself included, the thought process is to get to the exam content so you can complete the exam, get your score, and get out the door and back to your busy work life. This is a normal and accepted behavior and I’m not advocating a change on that aspect.
What I am posting about is reading, understanding, and adherence to the NDA. If you are not sure what it is or what is contained in it, take the time on at least one exam to read it completely. Essentially, by clicking the Accept button on that screen, you are agreeing to a legally binding document. So, in a nutshell, what are you agreeing to?
In essence, you are agreeing that when you have completed taking the exam, you will NOT discuss with anyone, at anytime, in any forum, in any form whatsoever, what was covered on the exam. You are not permitted to discuss the content of the questions in any form. The only content you are permitted to discuss is what is publicly available in the form of exam preparation guides. The content on the exam is intellectual property and is owned by the exam provider such as Microsoft.
Exam developers like Cisco, CompTIA, Microsoft, Oracle, etc. spend considerable time, effort, and cost to create certification exams and the content that is required to adequately test candidates on the material necessary to achieve the certification. This effort helps to ensure that the certification you acquire when passing the exam(s) provides you with a valid measure of your skill and knowledge for that technology.
By talking openly on what the content of the exam is, you are in essence making it easier for those who come after you to pass the exam without really knowing the content. You also make it easier for the brain dump sites to get a head start on what the exam covers. By not discussing the content of the exam with anyone, you help to ensure the validity and value of the certification you just earned.
Certification providers include in the NDA the fact that if you are caught divulging this information, you can lose your credential and become banned from further certifications. They are within their rights to enforce this although I am not aware of Microsoft doing this in the time I have been here. If you value your certifications and want them to continue to hold value in the industry, you need to partner with your certification organization to help prevent piracy and ensure that those who achieve the certification, actually deserve it.
Comments and questions welcome.
Well, yes and no.
Yes, everyone is supposed to do the right thing. But when so many people (including many of my students) prepared the exam and passed mainly based on these brain dump sites, they don't care.
I've seen students sharing these brain dump questions, even asking me questions from these brian dump sites, and I warned them to not use it. Two common responses: "Why? They're just exam prep questions." "Come on Bing, we just want to pass the exam and everyone is using it."
I really don't know what to say to my students. I can't say these questions are so similar to the actual ones -- or I'm breaching the NDA and confirming they're real. I can't say you're supposed to use only Microsoft approved material, because for many PRO level exams (especially for development), there is no MOC, no e-leanring, no books.
A "Skills Being Measured" list is the only thing officially available. But, have a look at what's in there, e.g. 70-564 "Choose appropriate controls based on business requirements. May include but is not limited to: user controls, server controls, built-in controls, custom controls, third-party controls, Web parts".
Don't you think it's too broad (i.e. not specific enough)? Students asked me "what third-party controls we should learn and use"? Well, even though I know they misunderstand the requirement, I can't even say in what aspects Microsoft is testing on "3rd-party controls". In the end of the day, students spent long time playing with different controls from different vendors, then took the exam and complained to me "There is no qustion on any particular ......." (well, I don't want to breach the rule.)
Because of the unclear requirement, they wasted time and effort. Yes, it's not bad to learn things not actually required for exam. But for exam prep, students only get bad impression on Microsoft.
In short, I think MSL shouldn't rely on people doing the right thing. And I don't believe brain dump sites get real questions inlcuding "exhibits' are from exam takers. There must be some dodgy testing centre involved. And, please, make the requirements more specific.
Just my 0.02
Hi Bing, thanks for your thoughts, I'll offer some responses to help you understand a little better how I would respond to these students.
"Come on Bing, we just want to pass the exam and everyone is using it."
"I really don't know what to say to my students. I can't say these questions are so similar to the actual ones -- or I'm breaching the NDA and confirming they're real."
1. By telling the students that these are actual exam questions taken from the real exam, you are not actually breaching your NDA at all. You are stating a simple fact. In order to breach your NDA, you have to discuss the question content with your students in terms of what is actually in the question.
2. What you should be telling your students is that if they want the certification because they place a value on it, then they should "earn" the certification and not cheat to get it. If they do use brain dumps or other cheating methods, it has no meaning to them and is a waste of their time and money, so they shouldn't pursue it.
It's no better than going to the many online sites that offer a Bachelor degree in whatever you want, without actually attending a University. When these candidates go for a job interview, they will be identified as a hack or someone who has gotten a credential falsely because they will not know the technology, only what they memorized.
If they want the certification as a way to obtain employment, they need to understand that their choice to use a brain dump site is only hurting them in the end. Employers know that these sites exist and they also know that there many "paper certified" people out there. As a result, they don't care that you have a certification anymore. So the students are only contributing to the very thing that is preventing them from achieving a goal in the first place.
To your other points;
The lack of Microsoft training materials for certain exams is not an excuse to cheat. I don't agree with that argument and I think it's a cop out for not dealing with the real issue at hand. If the student has the proper training and the required experience as listed on the prep guide, the exam should be a no-brainer to pass without any training kit. True, a training kit can help provide targeted learning for a student but so can the prep guide and MSDN, SQL Books Online, or TechNet documentation. The fact that a specific, targeted, training resource is not available is not an excuse to cheating on an exam.
As for the detail or lack thereof in the prep guides, again, not an excuse to cheat. If we were to provide more detail, students would learn only that information that is necessary to pass the exam. That, in my opinion, is no different than providing a brain dump. The way the prep guide is designed is real world. If I need someone who knows how to work with server controls in an ASP.NET application, I need someone who knows how to work with all server controls, not just a specific server control.
The exam is designed to test the candidate on their knowledge of a topic or technology. With that technology, there are numerous tasks associated. The tasks may encompass more than one method of accomplishing the task and a question or exam cannot possibly cover them all.
What you are asking for is no different than going to your high school math teacher and asking him/her exactly which aspects of algebra will be covered on the final. Will it be solving equations? Will it be simplifying equations? Will the questions ask about solving for x or solving for y?
Having said all this, it's important to note that you are missing the point of my original post. When taking an exam, you agree to an NDA. Violation of that NDA carries certain actions that can be taken against the candidate.
It's not a Microsoft issue. As I stated in the original post, any certification provider will have an NDA that a candidate must agree to prior to being allowed to take the test. So the fact that we don't give precise details on a prep guide or provide training materials for every exam we create, has nothing to do with the NDA.
The post was made because some people have written information on public forums about the exam content. Most that we have contacted were not aware that they had violated an NDA. When asked if they knew there was one and if they actually read it, they replied no.
This is meant to be an awareness post, not a rehash of a lack of training materials or clarity on a prep guide although I'm not saying you aren't able to voice your opinion on those matters.
As for how the content ends up on brain dump sites, I'm not able to comment on that but do know that we take it very seriously and are actively working to resolve the issues and deal with the offending parties where possible and appropriate. It's a daunting task and not one that can be completed or solved overnight.
Thank you for the Write Up. It is something which I agree every Certificate Exam taker (MS or otherwise) must adhere too.
I was wondering if there were any efforts to make the exam even more categorized and results based on scores. For e.g. Increase the depth of understanding by adding questions of increased difficultly and practical experience on the same topic and allow scoring to impact the certifical level given (graded from A to F etc).
In reality during our 'college' education, after understanding and reading the subject, we used to rely on the last 5 years question papers to get a general idea about the kinds of questions and marking pattern.
The idea to do the same for Certification Exam seems parallel to this. Every Certification at the very basic level is a proof of attempt and participation. An excellent score is the ability to crack the exam code.
I have attempted a couple of exams in the last month and posted general overview of the exam pattern and some topics which I found interesting or otherwise difficult. You can read it here http://geekswithblogs.net/DotNetReader/Default.aspx
I do not think I have done anything outside the NDA. Please feel free to let me know if this is not accepted.
Regards and Cheers
I'm certainly aware of the NDA, and I'm opposed to braindump sites. That said, I think there's a bit of a grey area when it comes to discussing these exams. For instance, I wrote a bit in my blog after I did an Exchange exam (see URL for relevant post):
"One significant point about this exam is that you have to be familiar with the command line. I can't go into too much detail (due to the NDA), but I can say that about half of the questions involved cmdlets, so if you use the GUI for everything then you'll fail this exam."
I don't see anything wrong with that: I'm not helping people to cheat (they still need to learn the cmdlets), I'm just giving them a rough idea of whether they're ready for the exam, so that they don't waste their money and fail it. However, the prep guide doesn't say anything about cmdlets vs GUI, so based on the post above have I broken the rules?
Thanks for the comments. As for the efforst to make the exam more categorized, I have to say no. We are actively looking at ways to move all of our exams to performance-based testing and away from a standard multiple choice style exam though. We are still a little way off from completing that task yet.
In looking at your posts that you link to, I don't see much in the way of information on question content being divluged. Realistically, your posts are not much more than what can be gleaned from reading the prep guides.
Based on what I read on your blog post, I don't see it as being a violation of the NDA. Discussing the type of questions may border on NDA violation but you're not actually telling anybody what a specific question covered. You merely indicate that the are of testing used cmdlets more that the GUI. It's actually good information for potential candidates and allows them to go back and study both ways.
The NDA at the start of the exam is the most difficult thing to explain to people as they believe you can tell them almost everything about the exam and get upset when you do.
I have taken to the method that if it has been mentioned in the wild by MSL then its ok to discuss. For example on 70-536 there was a live meeting on the whole thing and a web cast from TechEd that went through a lot of it.
There was a post on the MSL blog that detailed what could and couldnt be disclosed but it doesnt seem to be working. I even included it in the forum guidelines for the Training and Cert Forum on Technet. Any chance it could be published again as part of this to give people a couple of examples so that they can set their "NDA compasses" in the right direction
Thanks for the post. I agree with you, as usual.
Do you think you can find the post you are talking about? I would be more than happy to repost it.
It used to be at http://borntolearn.mslearn.net/2009/03/ten-things-we-cant-tell-you and I think it was Ken Rosen who posted it. Seems to be a dead link now which is a pity
I'll see if I can dig it up from some archive somewhere. At the very least, Ken should still have it somewhere on his computer.
In regards to question sharing and the like, and I don't know if it has already been done, but update the questions say once a month, that would make it impossible for people to memorize the questions. Or is that simply not feasible? too expensive etc.?
And by the way thanks for an awesome blog, been following for quite some time not without posting:)
Having an item pool large enough to do this would be a great idea but unfortunately, it is cost prohibitive. Not only from the perspective of writing enough items to do this, but also from the process and cost involved to rotate questions in and out of the exams. It's not just our systems involved but those of the exam delivery provider as well.
Thanks for the blog support too!
When will the exam results released for MCTS 2010 Beta?
The information on the beta exams indicated that the results will be available in appcoximately eight weeks or when the exam goes live.
Please note that this is an estimate only and various factors may affect the actual notification times.
I'm taking my vacation time preparing for the exam, taking a day off, driving 4 hours round-trip and sitting 4 hours (for beta) watching flickering monitor from 80s. And on top of it you expect me to read and understand NDA?
MS and MeasureUp should concentrate on more important things like to finally realize that mouse have a WHEEL and it suppose to WORK. Especially then the most of the pages in the test layout expect that monitor is a 5 feet high and only 6 inches wide. I complain to MeseareUP with no avail. If one of my programmers layout any program to make it a pain to use... If you think it's so important - make it 5 lines and ask to read it out-laud before the exam.