Course Background In this section, briefly describe the discipline, type of course, learning objectives, and types of learners for which the course was designed. The OLI introductory statistics course was modeled on Statistics 201, a first semester freshman service course typically attended by 250 students per semester at Carnegie Mellon. The course is required forfor all humanities and social sciences students, and similar courses are required for engineering and business students. The specific goals for the introductory statistics course are to teach basic ideas & logic of data analysis, help students develop a critical approach to study designs, data, & results and to understand what statistics is all about (big picture). The Statistics course emphasizes conceptual and critical understanding of statistics and utilizes statistics software to minimize computational mechanics.

Pedagogical Motivation What pedagogical, teaching or learning challenge(s) were you trying to address in developing this course? For example, you may want to discuss a particular learning problem that your students faced, or difficulty in teaching a particular concept that is hard for students to visualize. Our goal in all OLI courses is to promote the relevance and coherence of the domain of knowledge. Promoting relevance means teaching students how the skills and formal systems transfer to real world situations outside the context of instruction. Promoting coherence means teaching students how the discreet skills they are learning fit together in a meaningful “big picture�. In designing the course, we studied the challenges students face in learning statistics. We found that students have difficulty choosing appropriate graphical displays and statistical tools (e.g., many of their analyses were inappropriate or at best not directly relevant to the question). They often failed to interpret their results with respect to the question of interest (e.g., students would say they were finished with the problems almost immediately after producing a display or statistic). Generally described, students do not take a systematic approach to solving problems, their behavior appeared instead to be driven by the menu options of the statistics package or by a random process-of-elimination strategy. Since the course is online, we also focused on addressing the challenge of setting the "course path" and leading the students through it so that there is a sense of "beginning and end". Since students are on their own in the online course, how can we make sure they do not get lost in the details, how do we create a course where students will feel that they are "walking safely through it"

Learning Principles and Learning Activities Describe the specific learning principles that guided the course design and specific activities that you designed to address the pedagogical, teaching or learning challenges. A high level goal of the Statistics course is for students to understand the “big picture of Statistics� in other words understanding the process of: (1) producing data (sampling data from a population and considering study design issues), (2) conducting exploratory analysis on the collected data, and (3) making inferences from the sample back to the population of interest. The big picture gives students an organizational structure through which they learn the material. Studies have shown that students learn new material better when it is presented in a structured manner. The underlying organization of knowledge is represented in the structure of the course and reinforced in the navigational structure. At any point in the course, the student can answer the question, where am I and how does what I am learning now fit into what I have just learned and what I am about to learn. Learning is an active process. The course features short economical sections of expository text interspersed with frequent opportunities for students to learn by practicing the target concepts and skills. Learners work with instructional applets which are simple interactive applications that demonstrate concepts, engage in exercises using a statistical package, and take comprehension checks through multiple choice and short answer questions. Following many of the exercise, the students are given an open text box to explain their experience. These open text boxes give students the opportunity to reflect on what they have learned and engage in self explanation then compare their explanation to the explanation an expert would give. The contents of the text boxes are also stored so that an instructor using this course can review the explanations given by the students. We provide instruction in the problem-solving context and give immediate feedback on errors. StatTutor is a cognitive tutor that is embedded in the OLI Statistics course. StatTutor presents students with data-analysis problems, guides students through solutions (as they need it) and works in parallel with a statistics package. Throughout the course, we use mini-tutors that also support students with hints and feedback. We explicitly give the students a framework for seeing the solution relevant features of data analysis problems, the role type classification of variables. This framework is referenced as part of the hints that are given by StatTutor and mini-tutors as students solve analysis problems. In this way, we encourage students to recognize the relevant characteristics of problems within statistics. The role type classification of variables for statistical analysis is a somewhat abstract concept. We teach it to the students in a specific hands-on context of working through a data analysis problem. If done properly, explicitly helping the students to represent their strategies at higher levels of abstraction will increase the probability of positive transfer to other problems. The value of worked examples and contrasting examples are well documented in the learning science literature. Actively studying worked examples and contrasting examples helps students learn how to do a procedure and also to recognize the solution relevant features of a problem. We have found a process that works well in encouraging students to actively study worked examples and to identify the significant features in the contrasting examples. First we give the students a multi-media demonstration showing the example being worked out; we then give the student a partially worked example to complete in the form of a mini-tutor; we then present the student with a similar problem to solve on their own.

Tips for Teaching What advice do you have for instructors who might want to use this online course? Become familiar with the course structure starting with the "big picture" level, and down to the structure within units, modules etc. This structure of the course is the "pedagogical foundation" on which all the material rests. You'll find that It is not necessarily the **way** you were thinking about intro stats so far or the **order** in which you presented the material. Discuss the pedagogical idea of StatTutor prior to the first time students use it. Even though one of the benefits of the course is self pacing, you should give deadlines for when portions of the course need to be completed. Monitor the students progress through the grade-book and feedback reports, and let the students know when you feel that they are falling behind or not understanding concepts. I was afraid that students would feel like they are being "spied on", but one of the students I emailed to wrote back to me "Thank you for your concern, it adds a more personal feel to the online course." Use the contact time you have with the students effectively. This is an opportunity to clarify points that are misunderstood or to discuss interesting case studies that are relevant to the material for that week. One way that can be effective in creating an interesting class discussion is to have students complete a short "survey" (on paper or using the the response tool in the online course) in which they are asked: - What do you think are the 3 most important points of the material you went over this past week? - Give one example of a place in the course where you a bit confused, or that you wished you had someone there to clarify something for you. When you portion the material for the students give "bigger chunks" in the first two sections (EDA, Producing data), and smaller ones when the material is getting harder (probability, inference).

Impact on Teaching and Learning What has been the effect on your own teaching of designing or using this course? What has been the effect of using this course on your students' leaning? Describe your assessment of the course and the results. Include or link to assessment tools, example student work or resources that demonstrate the change. The design process created an environment for an open discourse about teaching statistics among several members of the statistics faculty. Working on the OLI course and meeting with the other OLI faculty has fundamentally changed and improved the way I teach. Thinking through how to explain concepts via the online course to students I will never meet, has given me insight into how to present concepts to students that I work with in the classroom everyday. We conducted a small assessment of the EDA module of the course during a summer session of the course at Carnegie Mellon. The students learned the EDA portion of the course entirely from the stand alone online course and then learned the remainder of the course through the traditional lecture and lab format. The students taking the online version of EDA performed as well or better on the EDA material on the exams. In a post course survey when students were asked whether they would recommend or not recommend the online course to others who wanted to learn statistics their responses were: 77% Definitely Recommend, 23% Probably Recommend, 0% Probably not Recommend, 0% Definitely not Recommend. We are currently conducting a evaluation of the full online course. During the current 2005 Fall semester at Carnegie Mellon 20 students who are enrolled in the traditional lecture and lab statistics course were randomly selected to take the entire semester course from the online course rather than from the traditional lecture and lab course. We will report the results of this study when it is completed.