Jill M. Klefstad's WisconsinTeaching Fellow Project

Assistant Professor-Early Childhood Education

Contact Information: klefstadj@uwstout.edu, 715.232.2345

Research Question

What impact do Critical Incident Questionnaires (CIQ) have on student's ability to think critically and construct critical questions?

Fall 2007


Context for Research

Hutchings (2000) identifies the taxonomy of questions which characterize the scholarship of teaching and learning. These kinds of questions focus on: 1) what works; 2) what is; 3) visions of the possible; and 4) formulating a new conceptual framework which shapes thoughts and practice.

The focus of my research investigated a vision of the possible question. The following questions framed my research: If students are introduced to the art of questioning which provokes a deeper level of thought and engagement in their learning, will they internalize the process to their own learning and teaching?

The Problem

Critical thinking is about how one approaches problems, questions, and issues (Facione, 2007). People who are critical thinkers have been known to have a critical spirit or a probing inquisitiveness, a keenness of mind, a zealous dedication to reason, and a hunger or eagerness for reliable information (Facione,2007).

According to Johnson (1990), teachers who are good questioners motivate their students and stimulate their thinking at a higher level. Research shows modeling good questioning techniques and asking good questions will assist students in making observations regarding their learning process and enhance their learning (Black, 2004; Potts, 1994; Thom, n.d.). In order to be an effective critical questioner, a person must become knowledgeable about the use of critical questioning within their discipline content and understand, prepare, and practice this technique (Hutchings, 2000; Paul, 2005).

Critical thinking can be best understood by viewing the art of Socratic questioning. The Socratic Method evokes responses from students that are more than one word or agreement/disagreement statements.

The following types of questions can be used to pose problems and stimulate students to think critically: 1) Questions of clarification, 2) Questions that probe assumptions, 3) Questions about viewpoints and perspective, 4) Questions which probe reasons and evidence, 5) Questions that probe implications and consequences, and 6) Questions about the questions (Saskatchewan Education, n.d.)

Previously, I have used a Critical Incident Questionnaires to study student dispositions about their learning and their ability to reflect critically. The current investigation will explore the impact which Critical Incident Questionnaires (CIQ) have on student's ability to think critically and construct critical questions.


This project conducted in the Fall of 2007 used surveys, critical incident questionnaires, critical question logs, and critical reflection journal entries to collect evident of student learning.

Surveys

Students were given two descriptive surveys that included questions on a Likert scale and open ended questions. The purpose of the survey was to gather information regarding student's understanding of critical thinking and critical questioning. The surveys were given in the beginning and at the end of the semester. The survey results compared the percentageof students who agreed with the statements in the beginning of the semester and at the end. The open ended questions were analyzed to find emerging themes.

Critical Incident Questionnaires(CIQ)

Step 1

Classes were held on Tuesday and Thursdays each week. On Thursday, students completed a Critical Incident Questionnaire (CIQ). The purpose of the CIQ was to give students an opportunity to reflect and write about what they learned in class that week as well as to define any frustrations they felt were preventing them from learning. The questions on the CIQ were written as a model of thoughtful critical thinking questions.

Specifically, the questions asked students to state assumptions they held, changes and modifications they could make to facilitate their learning, and to define ways their learning would impact their teaching. The last question on the CIQ asked students to write one critical question based on the lectures and discussions that week. Using a guide of Socratic questions linked to Bloom's taxonomy, students were encouraged to write the higher level thinking question and to identify the type of Socratic question they wrote.

Critical Thinking Log/Discussion Board

Step 2

Students recorded their critical thinking question into a Critical Thinking Log. The log asked students to write their question, identify the question for a young learner or adult learner and name the type of Socratic question created. Students posted their question to the discussion board allowing members from their cohort group (4 students) to read and respond to on question in their individual critical question journal. The instructor periodically commented on the posted questions in order to guide the process of identifying and writing Socratic questions.

Critical Question Journals

Step 3

Cohort members read and reflected on each of the critical questions posted by members of their group. Each member then chose one question from the discussion board and wrote their response in their Critical Question Journals. This process allowed students time to think more deeply about another critical question, formulate a prior question that could be asked as well as a follow-up question. Students composed a response to the original critical question. Each week, students’ added questions and responses to their critical question journals thus allowing for a snapshot of their progress in constructing and responding to critical questions.

Survey

Socratic Questions/Blooms Taxonomy


Critical Thinking Journal Rubric

Critical Incident Questionnaire (CIQ)

Critical Question Log

Project Summary

This project was conducted in the Fall of 2007 in a Social Studies curriculum course for Early childhood majors. One term often used with students is the term critical thinking. Yet, it was uncertain that students knew what the term meant nor how to apply it to their own learning and future teaching. My study investigated the following question and objective about critical thinking.

The Learning Question:

What impact do Critical Incident Questionnaires (CIQ) have on students ability to think critically and construct critical questions?

Objective:

To investigate how the use of weekly Critical Incident Questionnaires (CIQ) and critical journal reflections impact a student's ability to construct and answer critical thinking questions.

Results:

The results of the study which explored the impact of Critical Incident Questionnaires on students ability to think critically and construct critical questions showed that students meta-cognitive awareness about critical thinking increased throughout the semester evident in the students ability to recognize, evaluate, and apply the art of critical thinking to their own learning and teaching.


Annotated List of Helpful Resources & References

The resources below were used to compile the background information I used as well as ideas for the survey, reflection rubric, and critical incident questionnaire.

Ashby, A. (1996). Questioning critical thinking: Funny faces in a familiar mirror. Issues of Education . Community Colleges, Princeton, N. J.

Black, S. (2004). Habits of Thought. America School Board Journal, December, 191 (12).

Brookfield, S. (1997). New Directions for Adult and Continuing Education, Fall, 75, 17-29 .

Facione, P. A.; Facione, N. A. (2007). Critical Thinking: What it is and why it counts. Change 39 (2 ), March/April, 38-45.

Hutchings, P. (2000). Introduction to Opening Lines; Carnegie Foundation for the Advancement of Teaching. Retrieved June 12, 2007, from

http://carnegiefoundation.org/dynamic/publications/approaching.pdf

Johnson, N. L. (1990). Questioning makes the difference. Beavercreek, OH: Pieces of Learning.

Paul, R. (1990). Critical thinking: What every person needs to survive in a rapidly changing world. Rohnert Park, CA: Center for Critical Thinking and Moral Critique.

Paul, R. (2005). The state of critical thinking today. New Directions for Community Colleges, Summer, 130, 27-38.

Potts, B. (1994). Strategies for teaching critical thinking. Eric Document, ED385606.

February 1994.

Saskatchewan Education. (2007). Understanding the Common Essential Learnings. Critical and Creative thinking. Retrieved June 26, 2007 from http://www.sasked.gov.sk.ca/docs/policy/cels/el4.html#e13e14

Ongoing


Results

The results of this study showed that the students metacognitive awareness about critical thinking increased throughout the semester evident in the improvement in their ability to identify, construct, and answer critical questions.

Survey Results

Students took a pre/post survey to gather information regarding their understanding of critical thinking and critical questioning. The survey results compared the percentage of students who agreed with the Likert statements in the beginning of the semester and at the end.

Questions from the surveys which demonstrated the most changes in students awareness of critical questioning and their own metacognition were Questions 1, 6, 2, 4.

Survey results showed that students agree that critical thinking is about finding an answer or a response

Question 1: Critical thinking is driven by seeking answers to questions. In September 8 percent agreed with that statement compared to a 48 percent rate in December. Question 6: A critical thinking question always has a response. In September 12 percent agreed with the statement however in December 48 percent of students agreed

There was an increase of students confidence in regards to the meaning of the term critical thinking from September to December. Question 2: I am confident about the meaning of the term critical thinking. In September, 27 percent were confident about the term critical thinking whereas in December 80 percent of the students felt confident in understanding the meaning of critical thinking.

An increase occured in the percentage of students agreeing with Question 4: The questions a person asks reflects their level of thinking. 46 percent of students agreed with the statement in September and 64 percent agreed with the statement in December.

Students were given opportunities throughout the semester to become aware of their own preference to learning in relation to being asked questions or being given the answers. Question 7, I prefer to learn by having others ask me questions. In September 27 percent of students agreed with that statement but in December 40 percent of students agreed, In comparison, Question 8, I prefer to learn by being given the answers, 19 percent of students agreed in September but 12 percent agreed in December showing that students recognized that greater learning occurred when questions were asked rather than when answers were given.

Critical Thinking Logs

Students ability to recognize, evaluate, and apply critical thinking was measured by analyzing students critical logs and their written journal responses.

Students kept a log of their weekly critical questions which they posted on the discussion board. The purpose of the log was to track a students ability to write critical thinking questions based on course content from the week. The students used a Socratic question guide to help them formulate their critical thinking question. Students were asked to identify which one of the six specific types of Socratic question they modeled their critical question after.

The research data was analyzed to find the number of different types of Socratic questions asked throughout the semester. Socratic questions range from low level thinking to a higher level and include; (1) open ended questions, (2) questions of clarification, (3) questions that probe assumptions, (4) questions that probe reasons and evidence, (5) questions about viewpoints or perspectives and, (6) questions that probe implications or consequence.

Results showed that the most frequent type of Socratic question students asked was the Viewpoint/Perspective (Level 6) with 79 percent of students asking that type of question. 52 percent of the students asked questions that probe reasons and evidence (Level 5), 28 percent of students asked implications or consequence (Level 6), followed with 26 percent asking clarification questions (Level 2). The open-ended questions were only asked by 1 percent of the students (Level 1). The Social Studies course content encourages probing of assumptions about learning and thinking however, only 17 percent of students asked that particular type of Socratic question.

Critical Thinking Journals

In this study, students were asked to apply critical thinking by writing and identifying Socratic questions that evoke a deeper level of response.This study also analyzed student journal responses to a weekly critical question posted on the discussion board.

The journals were scored on the following criteria: 1) type of Socratic question was identified and characterized as a question for young children or adult learners, 2) journal entry included a prior question and follow up question to the critical question posed, 3) response to the question was thoughtful and prompted further questioning and 4) quality of writing was free of grammatical errors.

Critical journals entries were submitted three times for review throughout the fall semester. Each entry was worth ten points and the instructor scored the first and third entry while the second entry was scored by peers. The results of the data show the percentage of scores between eight and ten points. The results reflect only the overall score of the entries the researcher scored (first and third entry) because peers tended to score their fellow colleagues rather subjectively, giving them all tens.

The percentage of the scores between 8-10 on the first journal entry was 88 percent. In journal three, the percentage of the scores between 8-10 dropped to 54 percent. The drop was caused by some students to respond to questions which were not considered critical thinking questions and to write pre/follow up questions that were not critical questions. Although students used the Socratic guide to create their original critical question it was evident that they did not use the guide when formulating the pre/follow up question. Some students continually used the same beginning question such as what/how with no variation and thus no attempt to compose a critical question. This could be attributed to the end of the semester crunch and the desire to complete the semester long assignment.

Conclusion

In conclusion, the results of this project showed that through the use of the Critical Incident Questionnaire (CIQ), critical logs, and critical journals, students meta-cognitive awareness about critical thinking was impacted evident in their ability to successfully recognize, evaluate and respond to critical questions.

Percentage of Agree Statements

Summary of Questionnaire Response

Critical Thinking Line Graph-September

Critical Thinking Line Graph-December

Socratic Types of Questions

Journal Entry Graph

Overall Journal Score of 8 and Above

Career Relevance & Impact

Working on this project has contributed to my professional development because it has helped make connections with colleagues interested in the art of critical thinking. The research has evolved to create a plethora of ideas and resources to strengthen the theory behind teaching about critical questioning and critical thinking.

This research has been of value to the students in the Social Studies curriculum course because they have become aware of critical questioning and a higher level of thinking it provoked in them. This enlightment will hopefully impact their questioning young children as they proceed to student teaching. The increased level of students awareness about thinking and the art of questioning has prompted them to ask more critical questions and to think and respond more critically in discussions within the Social Studies curriculum course.





(c) copyright

This electronic portfolio was created using the KEEP Toolkit™, developed at the
Knowledge Media Lab of The Carnegie Foundation for the Advancement of Teaching.
Terms of Use - Privacy Policy