What Happens to My Comments in Beta Exams?

What Happens to My Comments in Beta Exams?

  • Comments 18

If you have taken a Microsoft beta exam and made comments on the questions at the end, you may be wondering where they go.  Does anyone read these?  Do they make a difference or do we discard them without a though?

Rest assured that each and every comment is read and taken into consideration.

Our rationale for beta exams has nothing to do with allowing a limited amount of people early access to the exam for free.  These betas are designed to validate the questions and answers with as many people as possible within the beta time frame.  The purpose is to allow those who were not involved with the exam creation and item selection phases to provide an objective view.  In much the same way as software is sent out for beta testing, this allows us to have fresh eyes on the exam in the effort to catch unwanted errors.

Once the beta exam period ends, the exam scores for each and every question goes through a thorough statistical analysis indicating the performance of each an every question across all the candidates. 

During this phase, each and every comment is read and evaluated and compared to the satistical analysis.  A new group of subject matter experts, from outside of Microsoft, are brought in for a focus group.  Some of these people are taken from the pool of beta candidates that scored very well on the exam, and some have not taken the exam yet.

They review each question along with the comments and then work with our team internally to make decisions on the items such as;

1) Keep the item if it scored well, was statistcally good and is a valid and relevant question that tests the candidate's knowledge well.

2) Fix the item if it is a good test of knowledge but may have some other issue in terms of correctness or hard to understand etc.

3) Remove the item if is a problem from the comment standpoint, scores poorly or any other reason that rates the question as poor.

So, if you wondered what happens to your comments, I hope this helps to answer that question.  Knowing this, I hope that on future beta exams, you take the time to provide us with relevant, constructive feedback that can help make a difference and a better exam for all.  Comments such as, "This question just plain sucks", does nobody any good.  And yes, it is in quotes because it is a direct quote from an item feedback.

For those that take our beta exams, thanks for your comments and helping us create quality, relevant exams.

 

Gerry

Leave a Comment
  • Please add 6 and 7 and type the answer here:
  • Post
  • If you have taken a Microsoft beta exam and made comments on the questions at the end, you may be wondering

  • Many thanks for explaining the total process, very informational.

  • Thanks for the info. I always leave a bunch of comments (just took all three 3.5 exams in beta).

    Just wondering, I've taken lots of beta exams but for some reason I always find out about them through someone else. Is there no beta exam email list that people can get on?

    Thanks!

  • Hi,

    I've only done three beta exams in the past, and always ran out of time when commenting. I've marked every question i didn't find to be perfect for review, and added a comment indicating what bothered me (that was usually for half of the questions). I've then written a few short comments on what i thought was not clear - and always ran out of time before i was able to comment on all the questions i had marked. As far as i remember, i had 15 minutes of time for the comment period each.

  • Some of the comments I wrote in my 40+ beta exams:

    - "None of the choices provides the correct answer.  The correct answer is using a third party tool to do that."

    - "After reading the question next to this question, I know how to answer this question."

    - "If A is correct, then B should also be correct.  That means both A and B are not correct.  If answer C is correct, you don't need to do the first step mentioned in the question.  I don't know the meaning of answer D, but that's the only possible choice."

  • Hi Bryant.  Typically the beta invites go out to people who have participated in the past in betas, people who are on the MCP lists and MCTs.  We do open them up to the public at times as well.

    There is no external beta mailing list however.

    Gerry

  • Hi Lukas.

    Beta exams allow 30 minutes for comment time at the end of the exam.

    Unfortunately, we cannot extend that time as there is a maximum seat time at the exam providers to ensure equal opportunity for exam scheduling for others in the schedules.

    I do understand that sometimes, 30 minutes is not enough to comment fully on 100+ questions but I hate to think there is that much issues with that many questions on the exams.

    My only suggestion would be to use bullet points to make your case in lieu of full explanations.  We can discern your meaning from point form as well.

    Gerry

  • S.Y. Paul, we need to see you in some of our item writing or selection sessions then.  :-)

    Gerry

  • I'm wearing the same pants, for the third day in a row. I hope no one notices. In other news, I have

  • I'm wearing the same pants, for the third day in a row. I hope no one notices. In other news, I have

  • Gerry, i could swear that on my 70-647 beta i only had 15 minutes...

    30 minutes sounds about right though.

  • Hi Gerry

    This is completely off topic, but other than adding comments to your blog I couldn't see a way to contact you.

    Just a general question really. Do you have any plans on adding something that fits inbetween MCPD/MCIPT and MCA? In my view all the MCPD/MCIPT exams are pretty basic and can be easily obtained with minimal experience and a book or two (I'm certified and talking from personal experience here). But then I'm years away from being ready to even think about applying for the MCA. Do you think you'll ever develop something to fill the gap? Maybe something like Sun does, where you have to actually develop a java solution?

    Sorry for spaming your post!

    Rob.

  • Hey Lukas.

    The live exams only offer 15 minutes but the betas should have 30 minutes.  Note I say "should have".  Not to say there wasn't a glitch in the system and you only did get 15 minutes.  

    Gerry

  • Hi Rob.

    I can't for certain at this time what will happen between the pro level and the MCA.  There is room in there for another level but we also want to re-evaulate the current pro level certs and determine if there is room to change or add to them.

    We are also working on some different technologies to help make the exams more relevant in terms of how they test but we are still only in testing stages at this time.

    Keep reading this blog for announcements on new plans etc.

    Gerry

  • For the MCITPs, the PRO exams are always easier than the TS exams.

    PRO exams test high level things, TS exams test low level things.  Unfortunately, those high level things you can easily learn from just reading a book, or reading some marketing materials from Microsoft.  On the other hand, those low level details in TS exams usually require you have some degree of actual handons to get familiar.

    I suggest Microsoft reverse the level of the PRO and TS.  People need to pass the PRO exam before they take the TS exam.  TS certificate should be moved to a high level.

    Those sales and marketing persons who only need to know the overviews of the products take the easier PRO exams.  Those real technical supports and engineers who need actually work on the products instead of just selling them should also take the TS exams.  The PRO exam doesn't need to be renewed.  The TS exams need to be renewed every 3 years.

    Microsoft should also suggest the partners increase the salary of those real technical personels as their work is more difficult than those sales and marketing people.

Page 1 of 2 (18 items) 12