What would you grade, or on what do you wish to be was graded?

One of the unfortunately all-but unavoidable elements of our educational system is the assignment of grades to students’ work. I won’t go into why this is hard to avoid in this post; rather, I want to speculate on what should be graded.

Arnold Pears recently posited “‍‘‍learn the skills‍’ should be synonymous with ‘‍pass the course‍’.‍” This assertion works well in courses where skills are the desired outcome (such as is the case in the programming courses he was speaking to). Variations can be easily derived for course with other principle objectives. But I cannot recall a course in which the objective and the grade were fully correlated.

Suppose, for a moment, that you are a teacher and have access to full immersive three-dimensional replay of every action every student ever takes with respect to your course. You can watch every motion of their eyes as they read, play back every note they take in every lecture, observe every facial expression, listen to every verbalization, etc. What of this, if anything, would you use to judge student understanding?

One thought I had was that if I am to test people based on their understanding, the process they go through to develop artifacts is more important than the artifacts themselves. Fortunately, I have some ability to test this.

There’s a student I sometimes tutor in office hours this semester who appears to bumble along completely at random, no where near the solution, and then suddenly it all clicks and he erases everything he has and writes the correct solution in one go. I know another who works up in tiny little steps a large number of incorrect solutions and then starts winnowing them down, removing the worst until a relatively sound skeleton remains to be fleshed out. Some students grab a relatively close example and modify it to solve the problem; others work from the ground up based on their notes and memory.

I can tell you a lot of things about students I tutor. I can tell you who’s creative, who “‍gets it‍” right away, who thinks of themselves as brilliant, who struggles, who’s patient, who gets distracted, etc. I can easily identify the 5-10% I’d give As. But, those aside, the bulk of the students are hard to judge. I can make some kind of rough ranking, but I know it’s based as much on listening skills as aptitude. And I know it doesn’t closely correlate with examinations.

There’s something comforting about grading based on “‍correctness‍” of answers. Even on fuzzy topics with obfuscated replies, there’s something concrete about it. You can take each question as if it stood alone and let some arbitrary formula combine the results. But frankly, the results just don’t matter that much to me. There are questions that are narrow enough to shed reasonable light on understanding, but there aren’t resources to provide them over more than a random selection of topics. There are questions that are hard to answer without integrating many topics together, but an incorrect result on such a question is hard to grade. And there are a whole lot of questions that measure test-taking skills more than the subject.

But even if I could collect the full process used by each student, how would I grade any part of it? Is the splat-and-cull approach a B where the copy-and-revise a C? If a student takes 30 minutes but spends 15 of them staring out into space, is that better or worse than a student who spends 15 minutes uninterrupted? Is there any proper mapping between student actions and grade?

I’ve never been one to put much weight in the idea that skill and intelligence can be quantified. Some people are clearly better than some other people at some tasks, but in between are a lot of people I can’t rank even on that task. And yet I expect to be giving grades to students for the rest of my life. There is some part of me that hopes I’ll eventually find a good way to do so.