Pondering the Complexity of "Mastery" Assessment

I’m pondering a little bit today. I’m imagining a bit about what it might be like to see a multiple choice test for the first time…stepping into my “time-n-place” machine…

I’m “time-n-place” traveling back to where people learned things by working side-by-side with someone who had more experience. This is a place where tests don’t measure what you know or what you are able to do. Demonstrating a thing you make, or a process you follow, or a piece of the whole that you finish is how someone can tell if you know your stuff or not. “Doing” is the way you show mastery — the way you move from apprentice to master.

Now consider, if this is your mindset, putting a multiple choice test in front of a person that laid out parts of the process or the vocabulary used in the job. And as with all multiple choice tests, there is no talking, no explaining of answers. You’re either right, or you’re wrong, depending up on the choice you select and regardless of the subtleties or intricacies of the process you’re “proving” you know.

How difficult that must be. How incredulous the master would be to even consider that as a worthy substitute for actually just SHOWING what you know.

But the test-maker wants to be sure there’s no confusion on the multiple choice test, so he creates questions that are clearer. That are less confusing. That have an easier right/wrong response. And maybe, for the more intricate explanations, you’re allowed to write essay or short-answer responses.

What do we lose in that “refining” of questions? Complexity? Subtlety of angle? Pressure? Nuance of response? Might it just be easier and more precise to just show the process? To do the thing you’re trained to do? Because if that thing is the actual thing the master cares about, why ever would she waste time creating — and refining — a test that doesn’t really do the process justice?

What do we gain with the multiple choice option? Mass standardization. The ability to have multiple people “demonstrate mastery” at the same time. The ability to say, “Yes, ALL these people have mastered that thing, that process and are now ready to be masters themselves,” when, in reality, that’s simply not true. It’s a mere shadow, if even that, of true mastery.

Pondering, pondering, pondering…