[cssei-interest] CS Education Reading Group resuming this Friday, 3PM, ICCS 238

Hi folks,
The CS Education Reading Group will be resuming for the term this Friday
at 3PM in ICCS 238.
Papers have been posted here: http://people.cs.ubc.ca/~patitsas/rg/
I hope to see you all there, and happy new year!
Cheers,
Elizabeth

Are Clickers Really Effective in Improving Student Performance?

A survey of students in six biology courses showed that students not only have favorable opinions about the use of student response systems (or clickers), but clicker usages also increase student learning (Preszler et al, 2007). Prior to this, Judson and Sawada (2002) have shown that students consistently show positive evaluations of using clickers in class for three decades, but there has been no consistent demonstration of learning improvement until this study.

81% of the students in the study agreed that using clickers increased their interest in their course. 71% of students agreed that clickers made it more likely for them to attend class. 70% agreed that clickers improved their understanding of course material. Most important, there was a significant linear increase in exam scores across all three levels of clicker usage frequency per class (low - 0 to 2, medium - 2 to 3, high - 4 to 6). That is high clicker usage results in mean student grades greater than medium clicker usage, and medium clicker usage results in greater mean student grades than low clicker usage.

Methodology: The study by Preszler et al. first analyzed whether the course grade distribution is similar among the six courses (using stepwise chi-square analysis), and then the students' opinions of clicker usage is analyzed to see if they differ by course grades (again using stepwise chi-square analysis). Courses that are significantly different are not included in the analysis to preserve as much consistency among the courses as possible. Clicker usage frequency in the courses follows a Latin square design to maintain an overall similar equivalent number of clicker questions used over the testing period to avoid biasing variation.

Reference:

Judson, E., and Sawada, D. (2002). Learning from the past and present: electronic response systems in college lecture halls. Journal of Computers in Mathematics and Science Teaching. 21(2), pp 167 - 181.

Preszler, R., Dawe, A., Shuster, C., and Shuster, M. (Spring 2007). Assessment of the Effects of Student Response Systems on Student Learning and Attitudes over a Broad Range of Biology Courses. Life Sciences Education. Vol 6, pp 29 - 41.

Goal-Free Problems

Asking a student to find the solution to a problem creates high cognitive load that she may end up making a lot of mistakes. This is especially true when the problem requires many sub problems to be solved, and students tend to make many more errors in these sub goal stages than at the final goal stage. This effect is called the stage effect.

An alternative to help students learn problem solving is to ask them to find the value of as many unknowns as possible, rather than finding a value for a specific goal. As an example, given a programming assignment, students are asked what are the unknowns rather than asking them to create a final program. This can be open ended and the students may go off on a tangent if not properly guided. Of course, no one would hire these students if all they can do is to come up with unknowns(!), but this strategy can be used as a scaffolding device to help students connect what they already know and what we want them to know. New knowledge can be built up as the unknowns are identified, and how these unknowns are related to what has already been learned.

Reference:

Ayres, P. (1993). Why Goal-Free Problems Can Facilitate Learning. Contemporary Educational Psychology. 18, pp 376 - 381.

Value of Praising Your Students / Kids

We all love praises ... for a job well done, for academic achievements, for beauty, .. but what do they do to us? Well we all know that they inflate our ego's, but unknowingly, they may have more damaging effects than we think!

Research has shown that people tend to give up if they realize that their lack of performance is due to a lack of ability, whereas people tend to continue trying if they realize that it is due to a lack of effort. It should be clarified here that "ability" refers to something that is fixed, whether it is true or not. Some people believe that intelligence is fixed and cannot be changed. Others may believe that playing a musical instrument is an innate ability rather than learned. These are often referred to as fixed or growth mindset. Students with fixed mind set are concerned about looking smart with little regard of learning. Students with a growth mind set are more concerned about learning than getting good grades.

Dweck (2007) found out that praising someone's intelligence encourages a fixed mind set more often than praising them for their effort. The underlying belief system is that we tend to think that intelligence is fixed. Research has also shown that those who were praised for their intelligence tend to shy away from challenging assignments, and this is far more often than those who were applauded for their effort.

Children who are praised for their intelligence also tend to pursue performance goal which means that their primary motivation is to continue to prove that they are intelligent by the rewards or recognition they can get. This can have negative consequences in that they are likely to sacrifice potential learning opportunities if these opportunities have an element of risk of making errors and do not ensure immediate good performance. Children who are praised for their effort prefer a learning goal that emphasizes the mastery of new and challenging material.

Children praised for intelligence were less likely to want to persist on problems than children praised for effort (Mueller and Dweck, 1998). It has also been shown that children praised for intelligence also enjoyed the tasks assigned to them less than children praised for effort. In another experiment, children praised for intelligence perform worse than children praised for effort after encountering failures and setbacks.

In yet another study, Nussbaum and Dweck (2008) show that people who have a fixed mind set of intelligence (also called entity condition) tend to repair their self esteem "defensively" by comparing themselves with competitors of equal or lower abilities after they encounter failures, whereas people who have a growth mind set of intelligence (also called incremental condition) tend to repair their self esteem by trying to engage in remedial learning and comparing themselves with competitors of higher abilities.

References:

Dweck, Carol. (November 28, 2007). The Secret to Raising Smart Kids. Scientific American Mind.

Mueller, C. and Dweck, C. (1998). Praise for Intelligence Can Undermine Children's Motivation and Performance. Journal of Personality and Social Psychology. 75(1), pp 33 - 52.

Nussbaum, D. and Dweck, C. (May 2008). Defensiveness Versus Remediation: Self-Theories and Modes of Self-Esteem Maintenance. Personality and Social Psychology Bulletin. 34(5), pp 599 - 612.

Framing

Framing is a construct developed in anthropology and linguistics to describe how an individual or group forms a sense of "what is it that's going on here?". We frame an event, utterance, or situation by interpreting it based on previous experience. E.g. when we see someone running like a madman on the street, we may interpret that as a fugitive on the run, and may expect someone else is chasing after him. Students may look at an exam question and quickly associate the same question with a previous exercise problem she has seen before.

Epistemological framing refers to the way learners form a sense of what is taking place with respect to knowledge, e.g. what past experience or knowledge is relevant to complete an assignment. Social framing refers to the way people form a sense of what to expect of each other, and of themselves in a social setting, e.g. what students expect from each other in a group project. Social framing can be observed through people's behaviors. Epistemological framing can be deduced through student learning assessments and their problem solving skills.

Based on the idea of social and epistemological framing, Scherr and Hammer (2009) studied how student interact with each other in physics tutorials. They coded student behaviors based on whether they work alone, discuss with each other, discuss with the TA, or just social, and correlate with student thinking, and their epistemological framing. They show that the behavioral cluster are evidence of student epistemologies. In particular, sitting up, speaking clearly, and gesturing frequently are evidence of novel reasoning and mutually constructed understanding.

Reference:

Scherr, R. and Hammer, D. (2009). Student Behavior and Epistemological Framing: Examples From Collaborative Active-Learning Activities in Physics. Cognition and Instruction, 27(2), pp 147 - 174.

Designing Effective Questions

Good questions that engage students in discussions are essential in peer instruction, whether these questions are posed after a mini lecture (Mazur, 1997) or as the core of in-class instruction (Beatty et al, 2005). Every good question should try to achieve three goals: content goal (deals with the subject material that you want to illuminate, or the what's), process goal (deals with the cognitive skills you want students to exercise, or the how's), and metacognitive goal (deals with the beliefs about learning, thinking, the subject area, etc.).

Beatty et al. propose four tactics in designing good questions. They are listed here in the order that may be appropriate for an one hour lecture where usually four questions can be quite easily incorporated into the lesson:

1. Tactics for directing attention and raising awareness. Focusing student attention and increasing student motivation in learning are important aspects at the beginning of each lesson. Some of the ways to achieve this are to ensure the questions (or invention activities) have all nonessential material removed, provide opportunities for students to compare and contrast different cases, extending a familiar case to something different, setting a trap to show student misconceptions.
   
2. Tactics for promoting articulation discussion. Using unstated assumptions, deliberate ambiguity, questions with multiple possible answers, students can be challenged to discuss and articulate their thoughts, ideas, and to clarify the topic to be further presented.
   
3. Tactics for stimulating cognitive processes. The fundamental rule here is to ask questions that cannot be answered without exercising the desired habits of mind. Some of the methods include asking questions that require students to interpret representations, understand a process or algorithm (rather than just memorizing a formula), having students describe the meaning and to choose from a set of possible ways of solving a problem, comparing and making contrast of different cases, and having students identify the necessary information to continue in their learning.
   
4. Tactics for formative use of response data. By revealing other students' response to a question posed before via a response histogram, a follow up question can be used to drill further down into common student misconceptions and clarify the differences among them. Having students to explain their choice of answers also promote learning and discussion in the classroom.
   

References:

Beatty, I., Gerace, W., Leonard, W., Dufresne, R. (2005). Designing Effective Questions for Classroom Response System Teaching. American Association of Physics Teachers, American Journal of Physics. 74(1), pp 31 - 39.

Mazur, E. (1997). Peer Instruction: A User's Manual. Upper Saddle River, NJ: Prentice-Hall.

Prospective Adaptation

One of the many goals of an educator is to prepare their students to adapt what they have learned in new situations. There can be two types of adaptation: fault-driven adaptation (which are reactions to a difficult situation), and prospective adaptation (which are proactive reformulations of one's knowledge or environment prior to encountering a new problem of situation). Martin and Schwartz (2009) show that graduate students uniformly make prospective adaptations to create meaningful representations of available information much more often than undergraduate students before diagnosing a problem, even though this may cost them some start up time. Undergraduate students who do not have continuous access to reference material tend to create more meaningful representations than students who have continuous access. The long term benefit for the graduate students in creating meaningful representation through prospective adaptation is that they complete a new diagnostic task much quicker than others with no meaningful representation.

In Computer Science, as in many other Science disciplines, students are not usually given the time or opportunity to step back, reflect, and retool one's knowledge. In first year programming, students are taught to solve computing problems through a systematic and methodical way while they may not understand why "hacking" is not suitable. But rather than short circuiting this process of prospective adaptation which the students should work through themselves, perhaps the students should be allowed to hack their code, and then asked to step back to rethink what other ways of problem solving is more appropriate when given a more complex problem. Instead, most often, algorithmic or procedural formulations are provided and students often try to memorize these solutions, hoping that these are sufficient for any new problems they will encounter. A learning goal should be included in each course where students are expected to engage in prospective adaptation to create new representations and ways of integrating new ideas with their knowledge base. This can be in the form of creating concept maps, writing up summaries of new knowledge and its connections to other areas (e.g. through a blog), designing new procedures or methods in solving problems, or engaging in invention activities (which encourage risk taking at little cost).

Reference:

Martin, L. and Schwartz, D. (2009). Prospective Adaptation in the Use of External Representations. Cognition and Instruction, 27(04), pp 370 - 400.