The Problem. My Research Question: The usual procedure for peer-instruction using clickers in large classrooms is to post a multiple-choice or true-false question, let students enter their responses, show the class the distribution of responses (without indicating the correct answer). If more than 15-20 percent answered incorrectly, let students talk to their neighbors for 1-2 minutes about their understanding of the concept (the 'peer-instruction' part), then let them vote again. If there is an increase in the number of correct responses, the interpretation is that some students who had answered incorrectly the first time came to understand the concept correctly and that the peer-instruction was the cause. This assumption may be incorrect. It is possible that some students game the quiz by assuming that the correct response is the one the majority chose on the first round; they change their answer to that without any learning having taken place. Evidence: on one true-false question I gave my class in 2005, ~60 percent chose the incorrect answer on the first vote. After showing the histogram and giving time for peer-instruction, I had them vote again, and this time the number choosing that incorrect response increased. Either those who had the misconception were unusually effective in persuading their neighbors to their point of view, or some students simply changed their answer to the most common response. The purpose of my project is to determine whether the latter occurs, to try to quantify it, and, if it occurs, to develop ways around it. [January 15, 2008; edited May 28, 2008] See the link below to a set of concept-test questions I designed for use in Spring, 2007. Summer 2007 Sample concept-test questions

Methodologies I will conduct this project in Introductory Biology 151 at UW-Madison. This is the introductory course that most biology majors take here. I teach the sections on evolution and diversity, a series of 15 lectures late each spring semester. I propose the following procedure to test the hypothesis that improvement in correct responses seen in the second vote is not completely attributable to students’ increase in understanding of the concept as a result of peer instruction. Reason: After seeing the histogram of responses to the first vote, some students may change their answer to the most frequent response in the first vote, with little or no change in their understanding of the concept. To test this, I will administer concept-test questions using clickers and the peer-instruction approach in two ways, in randomized sequence. In each, questions will be multiple choice, with two or more choices, or true-false, and designed such that students with a misconception about evolution are likely to be distracted by one or more of the incorrect answers. 1. Standard approach. Students will choose a response and enter it via their CPS clickers. The histogram of responses will be shown, but without indicating which choice is correct. After seeing the histogram, students will talk to 1-2 neighbors for 2-3 minutes and come to consensus about which answer they now think is correct. They will then vote again using their clickers. 2. Hidden histogram approach. Same as above, except that students will not be shown the histogram of responses in the first vote before they discuss and vote a second time. I will calculate the mean percentage increase (or decrease) in correct responses to format (1) and to format (2). For analysis, questions will be paired, one from each group, with the questions matched as closely as possible for correct answer. If the mean increase is significantly greater in format (1) than in (2), it will support the hypothesis that some students are gaming the concept tests. That is, they are changing their answer to the more popular choice in round 1 and are not improving their understanding of the concepts. I will include the following assessment question following one of the standard-approach questions described above: “On the concept-test question we just did, I A. did not change my response in the second vote; B. changed my vote from right to wrong as a result of discussing the concept C. changed my vote to the response that got the most votes in round 1 D. changed my vote from wrong to right as a result of talking to my neighbor(s), and I feel that my understanding of the concept improved E. changed my vote from wrong to right as a result of talking to my neighbor, so I don’t feel I really increased my understanding of the concept.” The self-reporting on this question will be another way to get at what goes on during peer instruction. Summer 2007; modified May 28, 2008

Project Summary A number of studies have shown that using clickers in large-lecture contexts can enhance learning. Based in part on these findings, I and the two other instructors in my lecture section of Introductory Biology 151 at UW-Madison used clickers last year (spring, 2007). We let students use their clickers to respond to a multiple-choice or true-false question designed to test their grasp of a concept. If more than 15-20 percent answer incorrectly, we move to 'peer instruction,' i.e., have students turn to one or two neighbors and discuss with each other which answer is correct, then we let them answer the same question again. Typically, the number responding correctly the second time increases, suggesting that more students now 'get it' as a result of discussing the concept with their peers. But there is a potential confounding factor behind these results. Not all of the increase in percent correct responses is necessarily the result of increased understanding. Some students may simply be changing their answer in the second round to the one that received the most votes in the first round, on the assumption that that is the correct answer. That is, they may be gaming the quiz system and not increasing their understanding at all. My project is designed to determine to what extent this goes on.

List of Helpful Resources and References Crouch & Mazur 2001 U. Colorado handout Preszler et al Noel handout Duncan Clicker do's and don'ts Anderson U. Guelph

Preliminary Findings, Results, Conclusions, and Implications

Career Relevance and Impact The impact of this study will depend on the results. If there is evidence for a significant effect of gaming the clicker-administered concept-tests, I would adapt my use of these questions in class so as to counter that effect. I anticipate that this could be as simple as not showing the histogram of responses to the class after the first vote. I see the project as a way to delve deeper into the varieties of student thinking in response to the peer-instruction approach that may lead to my fine-tuning the use of this active-learning technique. On a somewhat larger scale, as chair of the large, team-taught Intro Bio 151-152 course (30 faculty instructors) I am in a position to encourage the use of clickers and peer instruction by the faculty in that course. There are already signs of interest in using this technology among some of the faculty, so use of clickers may well expand beyond the three instructors who used them in the Spring of 2007. Results of my project will certainly be of interest to them.