Graded assignments and secret exams.
This week in the course I teach for new teaching assistants we discussed “problem students,” which might more accurately be described as “students TAs have problems teaching.” This involved discussing (among other topics) the difference in motivation of some students and their TAs, and as I was preparing to address them on this I realized that some of the difficulties these differences of priority create are the fault of course designers.
A class provides two potential rewards readily noticed by the students11 A third, the likelihood of enjoying lectures and activities, widely influences student course selection and engagement but is not generally acknowledged as an objective. : learning and grades. Learning is both its own reward22 I have yet to encounter evidence that there are people who do not enjoy learning. and enabling greater pleasure and power to influence the world. Grades are desired in themselves33 Grades are like the “points” in the game of school. and as a form of praise and as a gatekeeper to later opportunities and employment.
Instructors generally44 Except when they want to minimize time spent on teaching or when they believe that their goal is to find rather than create smart people. want to maximize student learning: it is an objective in itself. They also generally55 Except when they want to used them as a carrot or stick… want the grades given to mirror the level of learning66 Some measure this in progress, others in understanding. that took place. They tend to have at most residual appreciation for the seeking of grades themselves; a not-uncommon attitude is “the ideal student seeks learning and ignores grades”. Because they trust that learning will lead to good grades they believe that a student that learns well will get good grades too.
Some students do what the instructors want and focus on learning. Most also focus on grades to some degree, as witnessed by the practice of “studying for the exam.” Some, however, focus on grades far more than on learning, and these are often seen as “problem students” and likely cheaters.
There are many ways to foster learning and many ways to assign grades. Two in particular that are widespread (in my field at least) could almost have been designed to promote the desire for grades superseding the desire for learning. Even if that priority inversion does not result in cheating, it still inhibits students engagement in learning.
Computing courses at every institution of which I am aware assign programming tasks to be done outside of class time. These are almost always designed to provide practice and experience (to teach) and are also almost always graded. This practice of assigning graded homework tasks is not restricted to computing, but it is widespread in it.
For some students, this practice works fine. They do that task at least in part because it is graded and in the process gain the experience and practice needed to reinforce and actualize the concepts and skills the course is attempting to teach.
But for others, help is needed, perhaps because they failed to master preliminary material and now find themselves faced with an activity that starts from a place they have not yet reached. This is not a bad position to be in; courses generally have more than one student and each learns each topic at a different pace so some will take longer to master some topics than others. This is why we have office hours and TAs and such. But for graded assignments it creates a dilemma:
To maximize learning, the best action is to go back to previous material and reinforce the missed concepts.
To maximize grade, the best action is to get the assignment finished on time whether you understand its content or not.
This pull of the two main motivators in different directions engenders much of the problems that my TAs report: students who want TAs to get them past the parts they don’t understand rather than TAs who teach them to understand those parts themselves. It also motivates many of the most common forms of cheating I discover, such as unauthorized teamwork and stealing solutions from other students or the web.
Another extremely widespread practice77 As a student I had only two exams that did not fit this practice. One was a sham created by an inept and lazy teacher who had to show his exam to his superior but had taught nothing all term and wanted a high pass rate; the other was a publicly announced “you will have X minutes to write a program (without consulting any external source during that time) that does Y” and worked very well. is the administering of exams that would cease to be good measures of understanding if their content was made public prior to their administration.
For some students, this practice works fine. They do not know which more-or-less random subset of material will be tested so they prepare equally across the course’s concepts and then demonstrate understanding on the examined subset that is well correlated to their understanding of the whole.
But for other students this does not work well. When enough concepts have not been mastered that a grade on the entire set is unlikely to be as high as they want, they are motivated to discover the particular subset that will be tested. It is much easier to learn that 12345 (mod 10) is 2 than it is to learn how to perform modular exponentiation, and if that is the only modular exponential you’ll ever need then why learn the general rule instead of the specific case?
Reactions to this problem vary from what may be the most disheartening question I am asked on a regular basis88 “Will X be on the exam?” to the much more troubling case where the average on the 2pm section’s scores on the exam is a full letter grade higher than the average on the 11am section’s scores on the same exam. I’ve known instructors to feel they have to re-write an entire exam because the fire alarm went off ten minutes into the exam and the secrecy was lost, and have even known people to discover their exam was stolen and distributed prior to being administered.
There are other models of promoting student learning and evaluating student understanding that do not have the same99 Not the same problems ≠ no problems. There is no silver bullet… problems listed above. These generally1010 There are alternatives that do not satisfy both properties, such as the blend of self-paced instruction and the opportunity to repeat failed assignments until they work that is sometimes called “Mastery Learning.” If someone in my readership knows how to scale mastery learning to classes with 500 students, please let me know. share two properties: learning and evaluating are decoupled and knowing the content of evaluations in advance does not permit learning a small sample of course material.
Learning and evaluating are decoupled. There is not grade awarded for “succeeding” at a learning activity.
When learning activities carry no grade weight there is a general fear that some students will chose not to learn. That is of course their prerogative, but the motivational impact of grades can remain if the second property is also met.
Some instructors believe their students require the structure of due dates to keep them from procrastinating learning until it is too late. I have no evidence either way if this is true, but suspect that the scheduling of frequent decoupled evaluations would have similar motivational impact.
Knowing all knowable information about the evaluation in advance does not substantively reduce the understanding required to be positively evaluated.
Designing such evaluations is not simple, and generally involves per-student randomization via computer. The goal of such randomized exams is to define large enough sets of problems/questions that memorizing the answers to all elements of a set is more effort than is gaining the understanding necessary to answer a particular element of the set. The challenge is to ensure that the union of all such sets is equal to the set of information you wish students to learn.
There are additional technical and logistical issues with randomized exams, not the least of which is the difficulty in grading them.
There are reasons that courses are run the way they are, some good and some not-so-good. I myself have selected portions of my course design based on at least the following motivations:
Repeated pedagogical research has demonstrated that it maximizes learning.
Informal lore suggests it is a good way to teach.
I tried it and it seemed to work.
People I admire use a similar approach.
Other techniques have not worked so I’m trying something I hope will.
I suspect there must be a better way, but don’t know what it is.
It’s not the best way I know of, but it is the best I know of that I can implement in such large classes with so few assistants.
It’s not the best way I know of, but I’m still working to implement the infrastructure necessary to do it a better way.
It’s not the best way I know of, but I’ve asked for and been denied the physical resources (space, computers, etc) to do it the better way.
I know there is a better way, but am overworked and haven’t had time to implement it.
I know there is a better way, but have not yet learned how to do that.
I suspect there must be a better way, but don’t know what it is.
It’s what the person I learned from did and I haven’t questioned if they were right.
It’s a bad habit, but a hard one to break.
I’m not even aware I am doing it.
I confess that it is only recently that structuring my course in a way that provides some students with an incentive to cheat left the bottom of this list for me and became something I was aware of doing. Hopefully this blog post can help a few other teachers make a similar realization.
Looking for comments…