CS Science Education Initiative

[cssei-interest] CS Education Reading Group resuming this Friday, 3PM, ICCS 238

2011-01-07T15:19:00.000-08:00

Hi folks,
The CS Education Reading Group will be resuming for the term this Friday
at 3PM in ICCS 238.
Papers have been posted here: http://people.cs.ubc.ca/~patitsas/rg/
I hope to see you all there, and happy new year!
Cheers,
Elizabeth

Are Clickers Really Effective in Improving Student Performance?

2010-03-08T12:40:00.000-08:00

A survey of students in six biology courses showed that students not only have favorable opinions about the use of student response systems (or clickers), but clicker usages also increase student learning (Preszler et al, 2007). Prior to this, Judson and Sawada (2002) have shown that students consistently show positive evaluations of using clickers in class for three decades, but there has been no consistent demonstration of learning improvement until this study.

81% of the students in the study agreed that using clickers increased their interest in their course. 71% of students agreed that clickers made it more likely for them to attend class. 70% agreed that clickers improved their understanding of course material. Most important, there was a significant linear increase in exam scores across all three levels of clicker usage frequency per class (low - 0 to 2, medium - 2 to 3, high - 4 to 6). That is high clicker usage results in mean student grades greater than medium clicker usage, and medium clicker usage results in greater mean student grades than low clicker usage.

Methodology: The study by Preszler et al. first analyzed whether the course grade distribution is similar among the six courses (using stepwise chi-square analysis), and then the students' opinions of clicker usage is analyzed to see if they differ by course grades (again using stepwise chi-square analysis). Courses that are significantly different are not included in the analysis to preserve as much consistency among the courses as possible. Clicker usage frequency in the courses follows a Latin square design to maintain an overall similar equivalent number of clicker questions used over the testing period to avoid biasing variation.

Reference:

Judson, E., and Sawada, D. (2002). Learning from the past and present: electronic response systems in college lecture halls. Journal of Computers in Mathematics and Science Teaching. 21(2), pp 167 - 181.

Preszler, R., Dawe, A., Shuster, C., and Shuster, M. (Spring 2007). Assessment of the Effects of Student Response Systems on Student Learning and Attitudes over a Broad Range of Biology Courses. Life Sciences Education. Vol 6, pp 29 - 41.

Goal-Free Problems

2010-03-04T14:01:00.000-08:00

Asking a student to find the solution to a problem creates high cognitive load that she may end up making a lot of mistakes. This is especially true when the problem requires many sub problems to be solved, and students tend to make many more errors in these sub goal stages than at the final goal stage. This effect is called the stage effect.

An alternative to help students learn problem solving is to ask them to find the value of as many unknowns as possible, rather than finding a value for a specific goal. As an example, given a programming assignment, students are asked what are the unknowns rather than asking them to create a final program. This can be open ended and the students may go off on a tangent if not properly guided. Of course, no one would hire these students if all they can do is to come up with unknowns(!), but this strategy can be used as a scaffolding device to help students connect what they already know and what we want them to know. New knowledge can be built up as the unknowns are identified, and how these unknowns are related to what has already been learned.

Reference:

Ayres, P. (1993). Why Goal-Free Problems Can Facilitate Learning. Contemporary Educational Psychology. 18, pp 376 - 381.

Value of Praising Your Students / Kids

2010-03-01T12:18:00.000-08:00

We all love praises ... for a job well done, for academic achievements, for beauty, .. but what do they do to us? Well we all know that they inflate our ego's, but unknowingly, they may have more damaging effects than we think!

Research has shown that people tend to give up if they realize that their lack of performance is due to a lack of ability, whereas people tend to continue trying if they realize that it is due to a lack of effort. It should be clarified here that "ability" refers to something that is fixed, whether it is true or not. Some people believe that intelligence is fixed and cannot be changed. Others may believe that playing a musical instrument is an innate ability rather than learned. These are often referred to as fixed or growth mindset. Students with fixed mind set are concerned about looking smart with little regard of learning. Students with a growth mind set are more concerned about learning than getting good grades.

Dweck (2007) found out that praising someone's intelligence encourages a fixed mind set more often than praising them for their effort. The underlying belief system is that we tend to think that intelligence is fixed. Research has also shown that those who were praised for their intelligence tend to shy away from challenging assignments, and this is far more often than those who were applauded for their effort.

Children who are praised for their intelligence also tend to pursue performance goal which means that their primary motivation is to continue to prove that they are intelligent by the rewards or recognition they can get. This can have negative consequences in that they are likely to sacrifice potential learning opportunities if these opportunities have an element of risk of making errors and do not ensure immediate good performance. Children who are praised for their effort prefer a learning goal that emphasizes the mastery of new and challenging material.

Children praised for intelligence were less likely to want to persist on problems than children praised for effort (Mueller and Dweck, 1998). It has also been shown that children praised for intelligence also enjoyed the tasks assigned to them less than children praised for effort. In another experiment, children praised for intelligence perform worse than children praised for effort after encountering failures and setbacks.

In yet another study, Nussbaum and Dweck (2008) show that people who have a fixed mind set of intelligence (also called entity condition) tend to repair their self esteem "defensively" by comparing themselves with competitors of equal or lower abilities after they encounter failures, whereas people who have a growth mind set of intelligence (also called incremental condition) tend to repair their self esteem by trying to engage in remedial learning and comparing themselves with competitors of higher abilities.

References:

Dweck, Carol. (November 28, 2007). The Secret to Raising Smart Kids. Scientific American Mind.

Mueller, C. and Dweck, C. (1998). Praise for Intelligence Can Undermine Children's Motivation and Performance. Journal of Personality and Social Psychology. 75(1), pp 33 - 52.

Nussbaum, D. and Dweck, C. (May 2008). Defensiveness Versus Remediation: Self-Theories and Modes of Self-Esteem Maintenance. Personality and Social Psychology Bulletin. 34(5), pp 599 - 612.

Framing

2010-02-12T10:35:00.000-08:00

Framing is a construct developed in anthropology and linguistics to describe how an individual or group forms a sense of "what is it that's going on here?". We frame an event, utterance, or situation by interpreting it based on previous experience. E.g. when we see someone running like a madman on the street, we may interpret that as a fugitive on the run, and may expect someone else is chasing after him. Students may look at an exam question and quickly associate the same question with a previous exercise problem she has seen before.

Epistemological framing refers to the way learners form a sense of what is taking place with respect to knowledge, e.g. what past experience or knowledge is relevant to complete an assignment. Social framing refers to the way people form a sense of what to expect of each other, and of themselves in a social setting, e.g. what students expect from each other in a group project. Social framing can be observed through people's behaviors. Epistemological framing can be deduced through student learning assessments and their problem solving skills.

Based on the idea of social and epistemological framing, Scherr and Hammer (2009) studied how student interact with each other in physics tutorials. They coded student behaviors based on whether they work alone, discuss with each other, discuss with the TA, or just social, and correlate with student thinking, and their epistemological framing. They show that the behavioral cluster are evidence of student epistemologies. In particular, sitting up, speaking clearly, and gesturing frequently are evidence of novel reasoning and mutually constructed understanding.

Reference:

Scherr, R. and Hammer, D. (2009). Student Behavior and Epistemological Framing: Examples From Collaborative Active-Learning Activities in Physics. Cognition and Instruction, 27(2), pp 147 - 174.

Designing Effective Questions

2010-02-08T13:06:00.000-08:00

Good questions that engage students in discussions are essential in peer instruction, whether these questions are posed after a mini lecture (Mazur, 1997) or as the core of in-class instruction (Beatty et al, 2005). Every good question should try to achieve three goals: content goal (deals with the subject material that you want to illuminate, or the what's), process goal (deals with the cognitive skills you want students to exercise, or the how's), and metacognitive goal (deals with the beliefs about learning, thinking, the subject area, etc.).

Beatty et al. propose four tactics in designing good questions. They are listed here in the order that may be appropriate for an one hour lecture where usually four questions can be quite easily incorporated into the lesson:

Tactics for directing attention and raising awareness. Focusing student attention and increasing student motivation in learning are important aspects at the beginning of each lesson. Some of the ways to achieve this are to ensure the questions (or invention activities) have all nonessential material removed, provide opportunities for students to compare and contrast different cases, extending a familiar case to something different, setting a trap to show student misconceptions.
Tactics for promoting articulation discussion. Using unstated assumptions, deliberate ambiguity, questions with multiple possible answers, students can be challenged to discuss and articulate their thoughts, ideas, and to clarify the topic to be further presented.
Tactics for stimulating cognitive processes. The fundamental rule here is to ask questions that cannot be answered without exercising the desired habits of mind. Some of the methods include asking questions that require students to interpret representations, understand a process or algorithm (rather than just memorizing a formula), having students describe the meaning and to choose from a set of possible ways of solving a problem, comparing and making contrast of different cases, and having students identify the necessary information to continue in their learning.
Tactics for formative use of response data. By revealing other students' response to a question posed before via a response histogram, a follow up question can be used to drill further down into common student misconceptions and clarify the differences among them. Having students to explain their choice of answers also promote learning and discussion in the classroom.

References:

Beatty, I., Gerace, W., Leonard, W., Dufresne, R. (2005). Designing Effective Questions for Classroom Response System Teaching. American Association of Physics Teachers, American Journal of Physics. 74(1), pp 31 - 39.

Mazur, E. (1997). Peer Instruction: A User's Manual. Upper Saddle River, NJ: Prentice-Hall.

Prospective Adaptation

2010-02-01T15:19:00.000-08:00

One of the many goals of an educator is to prepare their students to adapt what they have learned in new situations. There can be two types of adaptation: fault-driven adaptation (which are reactions to a difficult situation), and prospective adaptation (which are proactive reformulations of one's knowledge or environment prior to encountering a new problem of situation). Martin and Schwartz (2009) show that graduate students uniformly make prospective adaptations to create meaningful representations of available information much more often than undergraduate students before diagnosing a problem, even though this may cost them some start up time. Undergraduate students who do not have continuous access to reference material tend to create more meaningful representations than students who have continuous access. The long term benefit for the graduate students in creating meaningful representation through prospective adaptation is that they complete a new diagnostic task much quicker than others with no meaningful representation.

In Computer Science, as in many other Science disciplines, students are not usually given the time or opportunity to step back, reflect, and retool one's knowledge. In first year programming, students are taught to solve computing problems through a systematic and methodical way while they may not understand why "hacking" is not suitable. But rather than short circuiting this process of prospective adaptation which the students should work through themselves, perhaps the students should be allowed to hack their code, and then asked to step back to rethink what other ways of problem solving is more appropriate when given a more complex problem. Instead, most often, algorithmic or procedural formulations are provided and students often try to memorize these solutions, hoping that these are sufficient for any new problems they will encounter. A learning goal should be included in each course where students are expected to engage in prospective adaptation to create new representations and ways of integrating new ideas with their knowledge base. This can be in the form of creating concept maps, writing up summaries of new knowledge and its connections to other areas (e.g. through a blog), designing new procedures or methods in solving problems, or engaging in invention activities (which encourage risk taking at little cost).

Reference:

Martin, L. and Schwartz, D. (2009). Prospective Adaptation in the Use of External Representations. Cognition and Instruction, 27(04), pp 370 - 400.

Peerwise

2010-01-26T09:34:00.000-08:00

Peerwise is a collaborative web-based system that allows students to create and evaluate a test bank of multiple choice questions. The pedagogical motivation behind this system is that students can learn better if they go through a process of self-reflection (meta-cognition), identify / synthesize / evaluate (higher levels of Bloom taxonomy), and articulate the subtleties in a concise format. Denny et al. (2010) show that students who were most active using the system improved their rank in the class relative to their peers who were less active. This is measured by using the students' final course grade from the previous course as a baseline for their initial class rank, and comparing with their final course grade in the course that involves the use of Peerwise.

Reference:

Denny, P., Hanks, B., Simon, B. (2010). PeerWise: Replication Study of a Student-Collaborative Self-Testing Web Service in a U.S. Setting. SIGCSE 2010, March 10-13.

Learner's Styles, Aptitudes, Personalities .. do they make a difference?

2010-01-18T13:29:00.000-08:00

Learning styles refer to the different ways different people learn information (e.g. visual / audio learners). Learning aptitudes refer to how different people learn in different learning environment structure (e.g. how students learn in highly structured or less structured learning environments). Learner personalities refer to the learner's belief whether his or her successes or failures are a consequence of internal or external factors (e.g. whether students believe their success and failures are a consequence of internal or external factors). Pashler et al. (2009) report that there are inconsistent and insufficient evidences that learning will be effective if instructions are provided in the mode that match learner's styles / attributes / personalities. This does not mean that learners do not have preferences, but in the particular type of evidence that Pashler et al. are looking for, that according to them would be "credible validation of learning-styles-based instruction", such evidence is missing.

The lack of evidence also does not mean that instructors should just stick to one mode of teaching. Students benefit from different representations of information, whether it be verbal, visual, analytical, lecture-based, inductive / deductive reasoning, etc., and that students should not pigeon-holed themselves in learning from any one or two particular styles.

Reference:

Pashler, H., McDaniel, M., Rohrer, D., and Bjork, R. (2009). Learning Styles, Concepts and Evidence. Psychological Science in The Public Interest. 9(3), pp 105- 119.

Student Self-Explanation

2010-01-11T08:19:00.000-08:00

Student self-explanation of material they just read has been shown to be effective in producing robust learning gains in a number of disciplines. However, past research results have not been clear whether performance gain is due to student simply paying attention to explanation generated by the instructors, or explanation generated by the students themselves. One research has shown that explanation is more effective when the students generate it rather than simply paying attention to instructor generated explanations (Brown and Kane, 1988), while in another case, the reverse is true (Lovett, 1992). Most recently, Hausmann and Vanlehn (2007) show that generating self-explanation while students attempted solving problems and studying examples is more effective in normal as well as robust learning (which means knowledge is retained over a significant period of time and demonstrated in far transfer of problem solving) than students who comprehended and paraphrased explanations generated by the instructors.

Self-explanation, coupled with learning by examples, can be very effective in student learning. Learning by examples has a lower cognitive load than learning by doing or solving problems, based on cognitive load theory. Thus comparing students who learn by doing a number of questions with those who learn by working through a number of examples, the cognitive load in the latter is much lower, and this affords the students the capacity to come up with general solution principles through self-explanation to improve their effectiveness in learning.

A related theme is that students who self-monitor their learning and comprehension in addition to self-explain the material they learned are better problem solvers than those who don't. By self-monitoring, this means that the students keep track of what they know and what they don't know, what are the parameters and data provided by the problems they are trying to solve, what needs to be solved, how the problems relate to the examples they have worked through having specific goals such as looking for solution methods rather than equations, formulas, similar contexts, etc.

References:

Brown, A.L. and Kane, M.J. (1988). Preschool Children Can Learn to Transfer: Learning to Learn and Learning from example. Cognitive Psychology. 20(4), pp 493 - 523.

Hausmann, R.G.M. and Vanlehn, K. (2007). Explaining Self-Explaining: A Contrast Between Content and Generation. In R. Luckin, K.R. Koedinger, and J. Greer (Eds). Proceedings of Artificial Intelligence in Education (2007). Amsterdam, The Netherlands: IOS Press.

Lovett, M.C. (1992). Learning by Problem Solving versus by Examples: The Benefits of Generating and Receiving Information. Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, Hillsdale, NJ: Erlbaum, pp 956 - 961.

Optimized University

2010-01-02T16:18:00.000-08:00

Carl Wieman put together a "think piece" on a new model for post-secondary education, which he called the Optimized University. Here are some of the highlights:

The Optimized University will focus on the desired student education outcomes rather than number of courses / credits students need to graduate with. There will be a switch in focus from processes to outcomes.
The instructor's role will primarily be an educational designer who continually assesses student's development with the assistance of technology and provides targeted feedback and challenges to the students to optimize their learning rather than simply a one-way transference of knowledge to students.
Clearly delineated educational goals will be created by relevant faculty in consultation with other stakeholders such as industry, educational systems, and government.
IT will be used to accurately diagnose student preparation, conceptual knowledge, beliefs, and epistemologies. IT will also be used for new teaching methods (interactive simulations, intelligent tutors, sophisticated diagnostic capabilities, clickers), improved class organization and management systems, archiving systems for educational materials and data, deployment of new modes of presenting material and enhanced communication by linking students with each other and faulty.
The Optimized University will have sophisticated pedagogical content knowledge - knowledge on how the content and skills are best learned, common student difficulties, approaches most effective in helping students overcome those difficulties, and how to motivate students to master the subject.
Validated assessments of desired deep understanding of material rather than a simple memorization of facts and problem solving recipes will be in place.
Technology will be used to make classes more intellectually engaging and educationally effective. Research has shown that there have been demonstrations of classes of 200 or more achieving very good learning gains using clickers and peer instruction in the lectures, computer graded homework systems, student-student collaboration (on / off line), extensive course webpages, and survey systems.
Carefully constructed diagnostic exams will be used to assess student preparedness and to reduce large hidden cost in instructor's time to provide the unprepared students with extra assistance and in dealing with the repercussions of failing students.
Student support will range from peer support and intelligent tutoring system, to trained undergraduate and graduate TA, to the expertise available from the faculty.
Students will have authentic research experience upon graduation.

Reference:

Wieman, Carl. (n.d.) A New Model for Post-Secondary Education, the Optimized University. Retrieived on January 2, 2010, from here.

Think Aloud Protocols

2009-12-09T13:19:00.000-08:00

It is possible to study objectively the form of thinking that occurs covertly in many types of typical tasks and activities in everyday life through think aloud or talk aloud protocols (or protocol analysis) (Ericsson and Simon, 1980). One of the biggest obstacles in this type of study is to find nonreactive settings to reproduce the thinking process without altering how the subjects would normally think. The single most important precondition for successful direct expression of thinking is that the participants are allowed to maintain undisrupted focus on the completion of the presented tasks. They should not describe nor explain their thoughts to anyone during the process. Interviewers should also limit their interactions with the subjects as much as possible during the sessions. Subjects can also be given a series of simple warm-up exercises (such as mental multiplication of two numbers) that will provide them with the practice of directing their full attention to the presented task while verbalizing their thoughts.

If participants are asked to describe or explain their thinking, it is found that such verbalizations present "a genuine educational opportunity to make students' reasoning more coherent and reflective" (Ericsson and Simon, 1998). These subjects are more successful in mastering the material and generate more self-explanations and monitor their learning better. Writing is found to be the most effective (as well as demanding) activity to improve and develop student's thinking.

References:

Ericsson, K.A. and Simon, H.A. (1980). Verbal Reports as Data. Psychological Review. 87(3), pp 215 - 251.

Ericsson, K.A. and Simon, H.A. (1998). How to Study Thinking in Everyday Life: Contrasting Think-Aloud Protocols with Descriptions and Explanations of Thinking. Mind, Culture, and Activity. 5(3), pp 178 - 186.

EnGauging Students

2009-12-03T14:35:00.000-08:00

EnGauging students is the process of engaging students in learning and gauging what they are learning simultaneously. Engaged students are more motivated to learn, and gauging students in the process provides students with feedback so they know what they need to change in their study habits. Some tools to enGauge students include:

Brainstorming - list as many answers as possible to a question
Case studies - solve a problem or situation in a real-world context
"Clicker" questions - answer questions electronically in class
Decision making - work together to recommend solutions to a problem
Group exams - work together to discuss exam questions but writes answers individually
One-minute papers - write a short answer about a topic or question
Pre / Post questions - answer questions before and after a topic is taught
Strip sequence - arrange a series of events into the correct order (e.g. Parson's puzzles)
Think-pair-share - think about possible answers to a question individually, and discuss with partners to come to a consensus
Reading assessment - enlisting groups of students to design the activities and teach each other
99 words / seconds - summarize a topic / lecture in 99 words or in 99 seconds (see example here)
KWL - have students answer 3 questions, individually or in a group, each class: "what we Know", what we Want to know, and "what we Learned".

Reference:

Handelsman, J., Miller, S., Pfund, C. (2007) Scientific teaching. W.H. Freeman & Company.

Desirable Difficulties

2009-12-03T13:36:00.000-08:00

Given that the fundamental goal of education is to make changes in the learner's long term memory, Bjork et al. have shown that learning conditions that introduce difficulties for the learners are potent in enhancing long-term retention and transfer. Humans do not simply "store" information in long term memory but rather, we relate new information to what is already known. Our long term memory is not a playback device. It is primarily semantic in nature. "Desirable difficulties" have been shown to be effective in making changes in the long term memory. This includes: spacing rather than massing study sessions, interleaving rather than blocking practice on separate topics or tasks; varying how instructional materials are presented or illustrated; reducing feedback; and using tests rather than presentations as learning events. See Bjork's Seven Study Tips and his slide presentation.

Reference:

Bjork, R. and Linn, M. (ND). Introducing Desirable Difficulties for Educational Applications in Science (IDDEAS). Retrieved on December 3, 2009 from http://iddeas.psych.ucla.edu/IDDEASproposal.pdf.

Cognitive Load Theory (CLT)

2009-12-03T11:46:00.000-08:00

Cognitive Load Theory is all about efficiency where efficiency is defined in terms of learner performance and learner mental effort. CLT suggests that we have only a limited amount of cognitive capacity for solving problems in our short term working memory (as opposed to long term memory for information storage). The higher the learner performance and the lower the learner mental effort (which occurs in the short term working memory), the better! According to CLT, there are three main types of cognitive load when one tries to learn something: intrinsic load (due to the complexity of the content to be learned), germane load (due to the instructional activities), and extraneous load (due to wasted mental resources on irrelevant material). Thus, in a first year computer programming course, learning to program in Java imposes the intrinsic load, providing worked examples on a variety of programming tasks contribute to the germane load, and requiring students to work within a complex integrated development environment (IDE) impose extraneous load on the students. Efficient instruction maximize germane load and minimize extraneous load.

Cognitive load depends on the interaction of three components: the learning goal and its associated content, learner's prior knowledge, and the instructional environment.

Reference:

Clark, R.C., Nguyen, and F., Sweller, J. (2006). Efficiency in Learning. San Francisco: Pfeiffer.

Worked Examples

2009-11-25T12:28:00.000-08:00

An important discovery of Cognitive Load Theory (CLT) (Sweller, 1988) is that studying partially worked examples provide better learning results for novices in computing than working through problems from scratch or studying completely worked examples. Gray et al. (2007) suggested the use of fading worked example as an effective strategy for lowering cognitive load in the novice phase of skill acquisition in programming education.

The idea of a fading worked example (FWE) is a sequence of partially worked examples in which each problem in the sequence contains one fewer worked step than its predecessor so that, in the end, the learner is given a problem to solve with no worked steps provided. Thus in systems programming, instructors may start with a fully worked example (Clark et al., 2006) from a problem statement, to analysis, design, coding and testing. Then the next example may involve all steps except coding. The next example may remove design, etc, until the students are required to solve a problem given just a problem statement.

The key to creating FWE is decomposition of each learning goal into smaller steps. As an example of using FWE for learning programming, each aspect of a programming language is identified. This includes variable, expression, assignment, iteration, subroutine call, etc. Next the use of each of these aspects in a program is related to the dimensions of problem solving, namely design, implementation and semantics.

How does studying worked examples compared to actual practice? Actively solving practice problems imposes much more mental work than reviewing worked examples. However, skipping study of worked examples may impose too much cognitive load on the learners when they try to jump into practice assignments right away. (See Guzdial blog entry.) Studies have shown that students who learned by doing took twice as much time to learn as students who learned from worked examples (Mayer, 2008, chapter 9). Students also benefit more with worked examples if they generate explanations as they study the worked examples (meta-cognitive skill development).

A compromise between worked examples and actual practice is a completion example where some of the steps are demonstrated in a worked example and the other steps are completed by the learner as in a practice problem.

It should be noted that as learners gain expertise, worked examples actually become detrimental and they are better off working all the problems. The worked examples can become redundant. This is where FWE will be most useful.

Reference:

Clark, R.C., Nguyen, and F., Sweller, J. (2006). Efficiency in Learning. San Francisco: Pfeiffer. (Chapter 8).

Gray, S., Clair, C., James, R., Mead, J. (2007). Suggestions for Graduated Exposure to Programming Concepts Using Fading Worked Examples. International Computing Education Research Workshop, Proceedings of the third international workshop on Computing education research. pp 99-110.

Mayer, R. E. (2008). Learning and Instruction (2nd ed). Upper Saddle River, NJ: Merrill Prentice-Hall.

Sweller, J. (1988). Cognitive Load During Problem Solving: Effects on Learning. Cognitive Science. 12(2).

Video Lectures

2009-11-24T16:06:00.000-08:00

Internet delivered video lectures have been found to prepare students for exams as effectively as live in-class lectures in a biology course (Lents and Cifuentes, 2009) although students were not enthused with the concept of video lectures initially. Another experiment with video podcasts for Java CS1 course resulted in less than expected participation (Murphy and Wolff, 2009). However, in yet another study, students in a first semester calculus-based mechanics course using multimedia modules not only learned more than students using traditional textbook presentation, but also retained information better (Stelzer et al. 2009).

Much effort has gone into research on the design of multimedia materials to improve learning. This includes designing materials to help students stay focused of the learning goals, use of different input channels (visual and auditory) to help students build meaning and understanding, offloading (presenting words as narration rather than on-screen text), weeding (eliminating interesting but extraneous material), signaling (adding arrows or highlighting for emphasis), and aligning words and pictures (Mayer, 2001) (Mayer, 2003).

References:

Lents, N., and Cifuentes, O. (November / December 2009). Web-Based Learning Enhancements: Video Lectures Through Voice-Over PowerPoint in a Majors-Level Biology Course. Journal of College Science Teaching. 39(2), pp 38 - 46.

Mayer, R.E. (2001). The Cambridge Handbook of Multimedia Learning. Cambridge U.P., Cambrdige.

Mayer, R.E. and Moreno, R. (2003). Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educational Psychologist. 38(1), pp 43 - 52.

Murphy, L. and Wolff, D. (2009). Creating Video Podcasts for CS1: Lessons Learned. NorthWest Academic Computing Consortium (NWACC), Journal of Computing Sciences in Colleges, 25(1). pp 152 - 158.

Stelzer, T., Gladding, G., Mestre, J., Brookes, D. (February 2009). Comparing the Efficacy of Multimedia Modules with Traditional Textbooks for Learning Introductory Physics Content. American Association of Physics Teachers. 77(2), pp 184 - 190.

Good Problems and Effective Structures for Groups

2009-11-20T13:11:00.000-08:00

Context-rich group problems help students to focus on the concepts and principles that are needed to solve them. They have the following general characteristics:

Problem statement does not always specify the unknown to be computed.
More information may be available than is needed to solve the problem.
Some of the information needed to solve the problem may be missing from the question. Students need to determine what the missing information is and how to come up with it.
Reasonable assumptions may need to be made to simplify the problem and allow for a meaningful solution.

Groups of three and four members are found to generate better plans for solving problems and a solution with fewer conceptual mistakes than pairs. Pairs usually have no mechanism for deciding between two strongly held viewpoints. In groups of four, one student was invariably left out of the problem-solving process. That person is usually the most timid or the most knowledgeable.

Homogeneous gender groups and mixed gender groups of two females and one male performed better than groups with two males and one female.

Groups with mixed ability performed as well as groups consisting of only high-ability students (who tend to make problems more complicated than necessary or overlook the obvious), and better than groups with students of only low or medium ability. Low ability students contribute by keeping the groups on track by pointing out the obvious and simple ideas, and requesting for clarification of the concepts and procedures that are needed to solve the problems (which the higher ability students sometimes realize their wrong assumptions and mistakes when they justify their solutions to them).

To avoid dominance of any student in a group, or to avoid a group from jumping at the first possible solution to avoid conflict in the group, two strategies can be used:

have students take on special roles. In a three member group, the roles of Manager (who designs plans for action and suggests solutions), Skeptic (who questions premises and plans), and Checker / Recorder (who organizes and keeps track of the discussions) can be assigned.
have the students reflect on how well their groups have worked and suggest ways of improvement at the end of each activity.

Reference:

Heller, P., Hollabaugh, M. (July 1992). Teaching Problem Solving Through Cooperative Group. Part 2. Designing Problems and Structuring Groups. American Association of Physics Teachers. 60(7). pp 637 - 644.

Is Collaborative Group Learning Useful?

2009-11-20T10:40:00.000-08:00

A study on the effectiveness of group problem solving was conducted by Heller et al. (1992). The instructional approach was as follows:

students were taught general problem-solving strategies
a set of context-rich practice and test problems were given to help students focus their attention on the need to use conceptual knowledge to analyze a problem
students worked in carefully managed groups to practice solving context-rich problems

Students' work were judged based on "expert" level of problem solving which is characterized by the following:

evidence of conceptual understanding
usefulness of information identified to solve the problems
match of equations with information identified
reasonable plan
logical progression
appropriate mathematics

Results: Group problem solutions were significantly better than those produced by the best problem solvers from each group on matched problems. Individual problem solving performance also improved over time. The key seems to be explicit problem solving strategy instruction and having the students practice using the strategy in groups.

Reference:

Heller, P., Keith, R., Anderson, S. (July 1992). Teaching Problem Solving Through Cooperative Grouping. Part 1: Group versus Individual Problem Solving. American Association of Physics Teachers. 60(7). pp 627 - 636.

Student Cheating in CMS

2009-11-15T00:31:00.000-08:00

Do students tend to cheat more when they write exams or quizzes using online course management systems (CMS) like WebCT or Blackboard? Not according to Charlesworth et al. (2006). They did a survey on 178 students and asked them first their definitions of cheating, and their main reasons to cheat in a typical classroom. Most students define cheating as copying or taking answers from others, and their major reasons for cheating include, in order of importance: 1) laziness, 2) grades, 3) pressure to do well and not fail, 4) lack of knowledge, and lastly, 5) opportunity. Given that students are not the best in making proper assessment of themselves, I am not sure if this list accurately ordered. Here is how I would re-interpret this list. As noted in the paper, "[m]any students report lengthy study sessions yet realize incomplete understanding due to factors such as poor study skills and lack of knowledge. As a result, students may feel unprepared for quizzes and examinations, and seek alternative methods to ensure success." If students do not grasp the material (i.e. lack of knowledge (4)), they feel pressured to succeed (3) to obtain good grades (2), but since hard work is difficult, some may give up, and blame it on their laziness (1), and given the right opportunity (5) to cheat, they would do so.

It is also interesting to note from the paper that students whose GPA is between 2.4 - 3.0 are more likely to cheat on written assignments. However, the study does not show that a web-enhanced course automatically increase the amount of cheating.

Reference:

Charlesworth, P., Charlesworth, D., Vician, C. (September 2006). Students' Perspectives of the Influence of Web-Enhanced Coursework on Incidences of Cheating. Journal of Chemical Education. 83(9), pp 1368 - 1375.

Problem Based Learning

2009-11-14T07:22:00.000-08:00

Problem based learning (PBL) is not simply throwing a problem to the students and let them figure out the solutions all by themselves. There are significant support elements to guide the students in the learning. It is actually a well defined, structured instructional method that students work through in seven steps with appropriate scaffolding support, learning resources, instructional support, tutor support, group discussions, etc.:

students clarify any terms and concepts in the problem text
generate a definition of the problem (or what is really the problem to be solved)
students brainstorm ideas, hypothesize, question about the problem
systematize and scrutinize the ideas
produce a list of issues for individual learning (the learning goals / contents behind the problem)
the learning issues are used to guide student study activities where students study the available resources
students share findings, review and discuss literature, solve other problems, and synthesize what is learned.

PBL is therefore not equated to minimally guided instruction (Schmidt et al. 2007) (Hmelo-Silver et al, 2007).

References:

Hmelo-Silver, C., Duncan, R.G., Chinn, C.A. (2007). Scaffolding and Achievement in Problem-Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist. 42(2), pp 99 - 107.

Schmidt, H.G., Loyens, S.M., van Gog, T., Paas, F. (2007). Problem-Based Learning is Compatible with Human Cognitive Architecture: Commentary on Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), pp 91 - 97.

Minimal Guided Learning

2009-11-14T06:43:00.000-08:00

Are there really any benefits to minimal guided learning, as practiced in a number of classroom activities in the form of inquiry learning, problem based learning, invention activities, etc.? According to Kirschner et al. (2006), not much. Their argument is that problem solving takes place in the working memory, which is severely limited in capacity when dealing with novel information, and since learning is to ultimately alter long term memory, they conclude that 1) the changes in the short term memory will likely not cause any changes in the long term memory since all information is lost within 30 seconds if the information is not rehearsed, 2) the heavy cognitive load is detrimental to learning.

Instead, a worked example with strongly guided instruction, process worksheets where descriptions on how to solve problems with specific hints and rules of thumb are more effective for student learning. Kyllonen and Lajoie (2003) found that highly structured instructional presentations benefit less able learners and unstructured instructional presentations benefit more able learners. Clark (1982) also noted that less able learners tend to choose less guided approaches to learning and they learn less. Higher aptitude students tend to choose more guided approaches to learning but they could have learned even more if they have chosen less guided instruction.

Is it possible then that CS education tends to create such a heavy cognitive load on our students, especially first year students, that result in such high attrition rate? Would providing students with detailed worked programming examples, strategies to solve programming problems, use of worksheets to allow students engage in deliberate practice help transition students to become more skilled programmers a better approach?

References:

Clark, R.E. (1982). Antagonism between Achievement and Enjoyment in ATI Studies. Educational Psychologist, 17, pp 92 - 101.

Kyllonen, P.C., and Lajoie, S.P. (2003). Reassessing aptitude: Introduction to a Special Issue in honor of Richard E. Snow. Educational Psychologist, 38, pp 79 - 83.

Kirschner, P.A., Sweller, J., Clark, R.E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 41(2), pp 75 - 86.

Sweller, J., Kirschner, P., Clark, R.E. (2007). Why Minimally Guided Teaching Techniques Do Not Work: A Reply to Commentaries. Educational Psychologist. 42(2), pp 115 - 121.

Tutorials and the Significant Role of the TA's

2009-11-06T09:37:00.000-08:00

It is unfortunate that many TA's are so busy with their research and course work that they often have minimal time to devote to tutorial preparation, whether in the material to be covered or teaching methods. A study by Koenig et al (2007) found that student performance gain in learning drastically improved in tutorials where students work in groups with TA's interaction using Socratic dialogue over tutorials where they work alone, or where they learn in a traditional lecture setting, or even in groups by themselves without other inputs from TA's. Student satisfaction of tutorials is clearly linked to the teaching performance of the TA's.

Interestingly though, when students were asked which style of tutorial did they prefer, more students indicate a traditional lecture style than Socratic group discussion with a TA, even though it is less effective. Perhaps the latter style moves the students out of their comfort zone a tad more than what they are used to and may seem to demand more work from them? In any case, this latter style seems to be more successful in moving students away from their initial misconceptions in the tutorials.

In order to implement such learning / teaching style in the tutorials, TA's will require weekly training to prepare for the tutorials. They have to work through the material and they need guidance on how to use Socratic dialogue with each tutorial topic.

Reference:

Koenig, K., Endorf, R., Braun, G. (15 May 2007). Effectiveness of Different Tutorial Recitation Teaching Methods and Its Implications for TA Training. The American Physical Society. Physical Review Special Topics - Physics Education Research. 3, 010104-1 to 010104-9.

Multiple Choice Questions

2009-11-05T08:18:00.000-08:00

Here are some suggestions and results on studies of multiple choice questions (Haladyna and Downing, 1989).

Three-option questions are optimal for most examinees. Three-option questions provides the most information at the mid range of the score scale, two-option questions provides the most information for high-scoring examinees, and the four- and five-option questions provide the most information for low-scoring examinees.
Use question format rather than sentence completion format.
Use as many functional distractors as are feasible. Eliminate dysfunctional distractors.
Type K questions (i.e. where each option includes combination of answers such as A) 1, 2, and 3, B) 2 or 3, etc.) are more inefficient to construct, more laborious to read, make a heavier cognitive demand on the students. They can be used to measure complex, higher level thinking skills.
Place the keys to the questions equally in different positions throughout the exam.
Avoid incorrect grammar that may clue the examinees to the correct option.
Humor in the options lowers test anxiety.
Word the question positively and avoid negative phrasing.
Common student errors can be used to make up distractors.
"All of the above" option makes the questions more difficult and less discriminating.
Avoid, or use sparingly, the option "None of the above". Similar to "All of the above", the questions are more difficult, less discriminating, and test scores are less reliable.

References:

Haladyn, T. and Downing S. (1989). Validity of a Taxonomy of Multiple-Choice Item-Writing Rules. Applied Measurement in Education. 2(1), pp 51- 78.

Haladyn T. and Downing S. (1989). A Taxonomy of Multiple-Choice Item-Writing Rules. Applied Measurement in Education. 2(1), pp 37 - 50.

Promising Practices in Undergraduate STEM Education

2009-10-30T10:29:00.000-07:00

In STEM education transformation, it is important to evaluate changes in light of implementation and student performance standards. Froyd puts together eight promising practices in STEM transformation, and how these practices can be evaluated.

Use of learning outcomes
Organize students in small groups
Organize students in learning communities to promote integrated and interdisciplinary learning
Organize content based on problem or scenario
Provide students feedback through systematic formative assessment
Design in-class activities to actively engage students
Provide students with the opportunities to engage in undergraduate research
Have faculty initiate student – faculty interactions

Implementation standards and student performance standards can be used to evaluate each of these practices. Implementation standards include:

whether the practice is relevant for the course
whether sufficient resource is available
the amount of effort required for the implementation

Student performance standards include:

whether there is any performance gain with the new practice compared to other students
comparison of different approaches to the implementation of the same practice, different class settings, students, etc.

Reference:

Froyd, J. (2008). White Paper on Promising Practices in Undergraduate STEM Education. Retrieved on October 30, 2009 from here.

Innovative Ways of Teaching Computer Science (Part 2)

2009-10-23T17:39:00.000-07:00

Please add to this list if you have any good ideas on innovative ways of teaching Computer Science:

Team teaching
Turning lectures into labs (given that many students bring their laptops to lectures, why not form groups of students with at least one laptop in the group for some hands on activities?)
Treating programming assignments like math homework problems (why do we give only big assignments most of the time?)
Play games (use games to engage students, see Thiagi web site)
Invention activities
Use humor, group activities, field trips
Get rid of textbooks or let students learn as much as they can on their own and share (these are two ends of the spectrum)
Let students decide what practical problems they are interested in solving with guidance from faculty (e.g. program iPhone, web app, robotics, etc.) and structure the course around them.

Problem Solving

2009-10-22T09:16:00.000-07:00

Definition: Problem solving is cognitive processing directed at achieving a goal when no solution method is obvious to the problem solver. (Meyer, 1992)

Proposition #1: Problem solving abilities do not transfer between disciplines. (Maloney, 1993)

Proposition #2: A student's strengths and weaknesses in problem solving are the same regardless of the environment. As an example, a student's strengths and weaknesses in solving a complicated trip planning problem are the same in solving a physics problem or performing in a work place. (Adams and Wieman, 2007)

Implications: if #1 is true, the argument that math and logic help students in Computer Science is no longer valid?

If #2 is true, all Computer Science students should play a lot more video games?

References:

Adams, W. and Wieman, C. (2007). Problem Solving Skill Evaluation Instrument - Validation Studies. Retrieved on October 22, 2009 from here.

Maloney, D.P. (1993). Research on Problem Solving: Physics, in Handbook of Research on Science Teaching and Learning edited by D.L. Gabel. Toronto: Macmillan. pp 327 - 354.

Meyer, R.E. (1992). Thinking, problem solving, cognition (2nd ed). New York: Freeman.

Innovative Approaches to Teaching Computer Science (Part 1)

2009-10-22T07:43:00.000-07:00

What we teach in Computer Science depends a lot on how we think of Computer Science as a discipline. According to Lewis and Smith (2005), the segregationists think that it is mainly problem solving, algorithmic analysis, theory building, and not an art. The integrationists think that it should be driven by what is needed in other computing fields and majors, such as applied computing in bioinformatics, engineering, and only partly by industries. The synergists think it should transcend any discipline where computing concepts can be applied in much broader terms in non-computing specific areas. As an example, the computing concept of pattern matching may be applied to DNA sequence matching initially (synergistic model), but now it is core to DNA analysis in bioinformatics (integration model), and complexity theories may come out of special algorithms in this area (segregation model).

How we teach Computer Science can also be influenced by these three models. One of the synergistic ways of teaching Computer Science is to consider the approaches to teaching in fine arts and how these can be applied to our discipline. Computer Science is traditionally taught in a format that is instructor centered (instructor is the expert, students are the novices), where the subject matter is abstracted from its practical use (toy programs vs. real life applications), and taught in individualized, non-collaborative (to avoid cheating) environment. In contrast, the fine art approach to teaching has a lot more student - student collaboration, student - instructor engagement, etc. (Barker et al. 2005). This is starting to change as we see more Just in Time teaching, use of clicker questions during lectures, pair programming, peer instruction, peer evaluations, in class activities, group projects, two-stage exams, media programming, etc. to increase enrollment and reduce attrition especially for female students in Computer Science. The paper by Barker et al. also has a good background summary on the attrition of women in Computer Science, and how fine arts approach to teaching may help in Computer Science teaching.

An integrative approach to teaching Computer Science can be seen in the paper by Cushing et al. (2009) where he reported an entry level Computer Science course integrated with Computational Linguistics that included case studies, term project, lecture series and seminars. Out of 70 students that completed the course, 24 students went on to the next quarter of Computer Science with several of them not originally intended to. It is not clear how this compares to other years.

References:

Lewis, T. and Smith, W. (June 2005). The Computer Science Debate: It's a Matter of Perspective. The SIGCSE Bulletin. 37(2), pp 80 - 84.

Barker, L. Garvin-Doxas, K., Roberts, E. (February, 2005). What Can Computer Science Learn From a Fine Arts Approach to Teaching? SIGCSE 2005, pp 421 - 425.

Cushing, J., Hastings, R., Walter, B. (2009). CS0++ Broadening Computer Science At The Entry Level: Linguistics, Computer Science, And The Semantic Web. The Journal of Computing Sciences in Colleges, Papers of the Sixteenth Annual CCSC Midwestern Conference, October 9 - 10, 2009. pp 135 - 142.

Curriculum Change

2009-10-17T09:37:00.000-07:00

What / who drives curriculum change? Some claim that it should be the academic faculty, others claim the industry, employers, or best practices, while others claim the students. Gruba et al (2004)'s extensive survey finds that computer education curriculum changes are driven by individuals, politics, and fashion (what is attractive to students) more than they are driven by academic merit and external curricula. So how can curriculum changes be made more objectively?

Peter Wolf, Associate Director of Teaching Support Services at the University of Guelph, co-edited the New Directions for Teaching and Learning publication, “Curriculum Development in Higher Education: Faculty-Driven Processes and Practices”. He is also the first author of the Handbook for Curri culum Assessment. In the handbook, he suggests a curriculum development process that combines Donald Kirkpatrick's four level training assessment model during curriculum development. It is evidence based that informs and guides the entire process. Here is a synopsis of the individual processes:

Curriculum Development

Peter Wolf's model of curriculum development process is a top-down model which starts with the learning goals and expected outcomes that should be acquired by an ideal graduate and then further refine this to how these goals can be implemented within a program / course structure and specific learning activities.

Training / Learning Assessment

Donald Kirkpatrick (1994) proposed a four l evel model to assess effectiveness of training:

Reaction - Did the learners like the program? Was the material relevant to their work? This type of evaluation is often called a “smilesheet.” According to Kirkpatrick, every program should at least be evaluated at this level to provide for the improvement of a training program.
Learning - Did the learners learn anything? Have the students advanced in skills, knowledge, or attitude? Pre-tests and post-tests are often administered to assess student learning.
Transfer - Are the newly acquired skills, knowledge, or attitude being used in the everyday environment of the learner? For many trainers this level represents the truest assessment of a program's effectiveness. It is also most difficult to test at this stage.
Results - Is there any increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment from the training?

Integrated Development and Assessment Model

By combing Wolf's and Kirkpatrick's models, each stage of Wolf's development process can be accessed by various levels of Kirkpatrick's assessment model, thus each is informed by the other.

References:

Gruba, P., Moffat, A., Søndergaard, H., and Zobel, J. 2004. What drives curriculum change?. In Proceedings of the Sixth Conference on Australasian Computing Education - Volume 30 (Dunedin, New Zealand). R. Lister and A. Young, Eds. ACM International Conference Proceeding Series, vol. 57. Australian Computer Society, Darlinghurst, Australia, 109-117.

Kirkpatrick, D.L. (1994). Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler.

Wolf, P., A. Hill, and F. Evers, The Handbook for Curriculum Assessment, 2006, Guelph University, obtained February 2007 from here.

Student Sharing (legitimately)

2009-10-12T09:19:00.001-07:00

While students are warned repeatedly against plagiarism, are there any advantages to have them share their work with each other after submission? One of the possible benefits is that students get to see how their peers have completed their assignments. This is particular useful if the assignment is open-ended where students are free to choose the problems they like to solve, the essays they like to write, projects they like to work on, or any areas of interest related to the course subject they may want to pursue. This in turn creates a multitude of contexts of learning that promotes knowledge transfer. According to Bransford et al. (2000), knowledge transfer is influenced by a number of factors. Some of these are:

degree of mastery of original subject (without a good understanding of the original material, transfer cannot be expected)
degree of understanding rather than just memorizing facts
amount of time to learn, and more specifically the time on task (or deliberate practice)
motivation (whether students are motivated by performance or learning)
exposure to different contexts
problem representations and relationships between what is learned and what is tested
student metacognition .. whether learners actively choose and evaluate strategies, consider resources, and receive feedback (active transfer), or depend on external prompting (passive transfer)

Open-ended assignments where students are encouraged to pursue problems that they are interested in and to share their work with one another and even critique each other's work touch upon many of these factors. Poogle (Head and Wolfman, 2008) is a framework for students to submit, share, and assess open-ended, interactive "unknown-answer" computer science assignments. The SWoRD system (Cho et al, 2007) allows students to review each other's writing and studies have shown that peer reviewing can empower learning to write from many angles. Both have been successful in promoting student learning through the process of student sharing.

References:

Head, C and Wolfman, S. (2008). Poogle and the Unknown-Answer Assignment: Open-Ended, Sharable CS1 Assignments. SIGCSE 2008. pp 133 - 137.

Cho, K., Schunn C., Kwon, K. (2007). Learning Writing by Reviewing. Retrieved on October 13, 2009 from here.

Bransford, J., Brown, A., Cocking, R. (eds). (2000). How People Learn. Washington: National Academy Press.

Case-Based Teaching and Data Analysis

2009-10-09T12:09:00.001-07:00

Case-Based Teaching and Learning Gains

Case based teaching that emphasizes problem solving and discussion improve student performance significantly on exams throughout the semester. It also enhances students' abilities to correctly answer application and analysis type questions.
While case based teaching improves student exam performance overall, lecture-based teaching results in more top performing students (90% or higher exam score) than case-based teaching. I wonder the "top" students that we traditionally think of are so well trained in learning under the didactic teaching method, that when they are exposed to other learning styles, they just become lost!

Data Analysis on Changes in Course Delivery

Here are the different data analysis that can be done to determine the effects of changes made in a course:

Use prerequisite course final exam scores or entrance exam scores to determine variation of student academic ability when comparing students from different terms.
Compare first test score with the final test score in a course to see how students improve in their different levels of learning (which can either follow Bloom's categories, or simply two levels: knowledge-comprehension / application-analysis).
Compare total exam points earned by students under different grade band (90% or higher, 80% - 90%, 70% - 80%, etc.)
Bloom course material / homework / etc. and correlate with test scores in 2.

Reference:

Chaplin, Susan. (September / October 2009). Assessment of the Impact of Case Studies on Student Learning Gains in an Introductory Biology Course. Journal of College Science Teaching. pp 72 - 79.

Case Studies Resources:

National Center for Case Study Teaching in Science:
http://ublib.buffalo.edu/libraries/projects/cases/case.html

The case page (with cases for many different science areas):
http://ublib.buffalo.edu/libraries/projects/cases/ubcase.htm

Student Cheating

2009-10-04T15:27:00.000-07:00

In a recent student survey conducted in one of the Computer Science courses at UBC, we asked the following question with the preamble: Just like all your other responses in this survey, no instructor will have access to your identity. In particular, your responses to the following two questions will not be used in any way as evidence of violation of academic misconduct.

Do you believe you may have ever violated the academic conduct guidelines of a UBC course and, if so, what activities were you engaged in?

Out of 81 responses we received, no student admitted to having violated the academic conduct guidelines. Of course it is quite probable that UBC students are highly ethical in nature, or the question may not be clear enough on what constitutes "academic conduct guidelines". In any case, even with the preamble, the students may not feel comfortable in revealing the truth because the survey did ask for their student number in the beginning! In a study of student cheating by Sheard et al. (2002), students self-report of their cheating activities ranges from around 10% to 47%. In general, there are internal and external factors that cause students to cheat, but the three most common reasons are: time pressure, possible failure of the course, and difficulty of work.

One of the more publicized cases of student cheating in Computer Science is reported by Zobel (2004). In that case, students cheated by purchasing assignments and even have someone write the exams for them. As the faculty tried to investigate on the case, there were met with violent threats and even office break in's. It all sounded like a soap opera, but it is understandable that many faculty members or administrators do not want to deal with cheating cases. After all, it is costly on every one's part.

Greening et al. (2004) and Joyce (2007) examine ways of integrating ethical content into computer curricula. A student survey that involves a number of scenario's that involve cheating seems to challenge the students' thinking on critical ethical issues of a number of issues. It is also critical that faculty needs to have a good background of philosophical frameworks to guide the students. Some of these include utilitarian, deontological, virtuous, and relativist frameworks.

The prevalence of cheating cases, especially in assignments, works against student learning in that properly designed assignments are effective ways to help students construct their knowledge. If instructors knew that students mostly cheat on the assignments, they tend to place less emphasis (and hence, marks) on assignments, and students are further unmotivated to do the assignments. Why is it so difficult to make up Computer Science assignments that are fun and are made up of small incremental tasks to engage the students?

References:

Zobel, Justin. (2004). Uni Cheats Racket: A Case Study in Plagiarism Investigation. Retrieved on October 4, 2009 from http://crpit.com/confpapers/CRPITV30Zobel.pdf.

Greening, T., Kay, J., and Kummerfeld, B. (2004). Integrating Ethical Content Into Computing Curricula. Sixth Australasian Computing Education Conference, Dunedin, NZ. Retrieved on October 4, 2009 from http://crpit.com/confpapers/CRPITV30Greening.pdf.

Sheard, J., Carbone, A., and Dick, M. (2002). Determination of Factors which Impact on IT Students' Propensity to Cheat. Australasian Computing Education Conference (ACE2003), Adelaide, Australia. Retrieved on October 4, 2009 from http://crpit.com/confpapers/CRPITV20Sheard.pdf.

Joyce, D. (2007). Academic Integrity and Plagiarism: Australasian perspectives. Computer Science Education. 17(3), pp 187 - 200.

Asking Questions

2009-10-04T14:44:00.000-07:00

When we pose questions to our students, they sequentially and iteratively go through four stages: comprehension, memory retrieval, judgment, and mapping (Conrad and Blair, 1996) (Tourangeau, 1984) (Oksenberg and Cannell, 1977). At any one of these stages, students may find it difficult to answer the questions due to the choice of words and the way the questions are asked. This may not because of their misconceptions of the subject matter but may indicate the questions need to be revised. Ding et al. summarized their results of validating clicker questions using interviews (2009).

In the comprehension stage, we want to make sure the students understand the problem accurately. In a think-aloud session, we may be able to see whether the students have misinterpreted the questions. Otherwise, this can be easily dismissed as a misconception that the students have.

In the memory retrieval stage, we want to make sure the students are accessing the relevant information to solve the problem. If any part of the question triggers the students that lead them in the wrong path, these questions can be seen as "trick" questions and are not testing the student learning.

In judgment, students need to perform the appropriate task to solve the problem given a correct retrieval of relevant information. If the questions are not clear about the context / conditions, the students may not be able reach a definite conclusion. In those cases, the questions need to be clarified.

In mapping, students need to correctly map the right answer to the right choice. Here, the choices provided must be clear and the students can make a definite choice.

Validating questions take time, and student interviews seem to be an effective way of helping instructors refine their questions. Teachers can also find out something about the student responses to the questions and see if there is a majority of them getting the questions wrong by examining the exam sores and their correlation with other data. Such forensic study may reveal how students interpret and think through the questions.

References:

Ding, L, Reay, N.W., Lee, A., Bao, L. (2009). Are We Asking the Right Questions? Validating Clicker Question Sequences by Student Interviews. American Journal of Physics. 77(7), pp 643 - 650.

Conrad F. and Blair, J. (1996). From Impressions to Data: Increasing the Objectivity of Cognitive Interviews. Proceedings of the Section on Survey Research Methods, American Statistical Association. (ASA, Alexandria, VA). p 1.

Tourangeau. R. (1984). Cognitive Science and Survey Methods. Cognitive Aspects of Survey Design: Building a Bridge Between Disciplines. Edited by T. Jabine, M. Straf, J. Tanur, and R. Tourangeau. (National Academics Press, Washington, DC). p 73.

Oksenberg, L. and Cannell, C. (1977). Some Factors Underlying the Validity of Response in Self-Report. Bull. I'Institut Int. Stati. 48, pp 325 - 346.

7 Techniques of Teaching / Learning

2009-10-01T10:06:00.000-07:00

deWinstanley summarizes Bjork's seven studying techniques in the reference below. These seven learning techniques have corresponding implications for teachers. Here is the list for teachers:

Allocate your attention efficiently. Anything that does not help your students bridge what you want them to learn with what you want to tell / show them is a distraction. If you tell a story, make sure there is a connection with what you want them to learn. Use questions to help your students to focus.
Interpret and elaborate on what you are trying to teach. Students need context to apply what they learn so they can have better recall and retention.
Make your teaching variable (e.g. location, interpretation, example). Use a variety of contexts to illustrate what you want to teach (see points 1 and 2). Try contrasting cases.
Space your teaching of a topic or area and repeat your teaching several times. Instead of blocking or massing what you want to teach on XXX in one big chunk of time, try to space it out in a number of sessions.
Organize and structure the information you are trying to teach. Provide skeleton outline rather than a full outline so students can pay more attention. Provide or have the students produce (see point 7) a concept map that captures the concepts and their relationships with one another.
Help students to visualize the information. Reinstate the context during a test. Use mnemonics, graphs, props, etc., but make sure they are helpful for the student to build bridges to the learning content (see point 1).
Generate Generate Generate ... Retrieve Retrieve Retrieve. Give students lots of tests and opportunities to construct their knowledge. Feedback is good but even if they don't get immediate feedback, have them generate their knowledge over and over again.

References:

deWinstanley, Patricia. (1999). The Science of Studying Effectively. Bjork's Seven Studying Techniques. Retrieved on September 4, 2009 from http://www.oberlin.edu/psych/studytech/.

Misconceptions

2009-09-24T11:06:00.000-07:00

Constructivistic learning claims that "all learning involves the interpretation of phenomena, situations, and events, including classroom instruction, through the perspective of the learner's existing knowledge" (Smith et al, 1993). As such, with the prior knowledge students bring into the classroom, learning involves confrontation and replacement of misconceptions that students have. (Otherwise, there is no need for them for any formal training.) But how do students recognize these misconceptions, and how can they correct these misconceptions given that misconceptions are hard to change? Traditional strategies include lectures, assignments, exams, etc. Well constructed clicker questions can be particularly effective in exposing misconceptions. We can also learn a lot from video games. Good video games (Gee, 2005) can expose players' misconceptions of the game by slowly guiding the players to gain proficiency in the game play, whether it may be motor skills required to use the controls, or mental skills to solve the problems, or awareness of hidden story lines, etc. Rewards have been used effectively to grab the player's attention. How we can turn our classroom experience into a well constructed video game remains a mystery and challenge for all instructors!

According to Smith, diSessa and Roschelle, instruction is supposed to replace misconceptions by confronting the students with their misconceptions. Instead of replacement, perhaps learners are integrating what they know and trying to resolve the conflicts they encounter when new information is presented. There may be knowledge replacement but I suspect it is more integration or resolution of these conflicts, than replacement that is going on.

McCartney et al's paper shows some of the misconceptions on how CS students determine algorithm efficiency. Given two algorithms, the students were asked which one is more efficient to solve a certain problem. The goal is to see whether they consider how the algorithms behave in the worst case - which most experts would do. Although the majority of the students pick the right algorithm, some focus on one part of an algorithm to determine the "worst" case, while others focus on another part of the same algorithm. One of the misconceptions then is that students do not really know what the worst case was. They also do not seem to think tracing through an algorithm on concrete data is important in the analysis.

What are other CS misconceptions? Computer will do what I mean. Command line is not as powerful / efficient as graphic interface. The scenes in a computer game are stored in the program rather than dynamically generated. Playing with / debugging / changing code in the process of writing a program are not expert behaviors. Doing rough work is not cool when solving problems. Web design is programming. Spreadsheet is a database. Design is useless. Testing is not valuable. Any others, there must be a whole lot more ...

References:

Smith, J., diSessa, A., Roschelle, J., (1993), Misconceptions Reconceived: A Constructivist Analysis of Knowledge in Transition, retrieved on September 24, 2009 from http://ctl.sri.com/publications/downloads/MisconceptionsReconceived.pdf.

McCartney et al., (2009), "Commonsense computing (episode 5): Algorithm Efficiency and Balloon Testing", retrieved on September 24, 2009 from http://portal.acm.org/citation.cfm?id=1584322.1584330.

Gee, J. (2005). "Learning by Design: good video games as learning machines." E-Learning, 2(1). pp 5 - 16.

Lectures

2009-09-17T11:24:00.000-07:00

Presenting information in lectures require careful planning so that the precious class time will not be wasted. Since learning is an interpretative process, new information needs to be integrated with what is already known. deWinstanley and Bjork suggested 5 processes that affect much on how students learn. Attention - divided attention is most detrimental during encoding of new information. What is worse is that "divided attention during a lecture may leave students with a subsequent sense of familiarity ... without the concomitant ability to recall or recognize the material on a direct test of memory". Interpretation and Elaboration - learning requires accurate interpretation and thorough elaboration. Students need to know the "story" behind the new information. Simply presenting a graph or a formula does not help the students to learn why and how the new information can be used. Generation and Retrieval Practice - students learn better if they generate the information rather than just passively absorb information. If students are asked to retrieve information, it is more likely they will recall the information later. Students can create concept maps / reflective blogs / contribute to discussion forums as means of generating the information they have learned.

Other techniques that can promote long term retention of information in the lectures include: spacing - distributing rather than massing the presentations of information at the same time, (an example of spacing is the spiral curriculum, i.e. start with an introduction, then drill down into the topics in the next interaction, and then focusing more details in further iterations), presenting material from more than one standpoint, providing outline (but not too much detail), having students to generate their outline, using visual images and other mnemonic devices, analogies, humor, having the students to make predictions and elaborate interrogation.

To keep student attention, one can also use appropriate games, toys, simulators, play, etc. Interactivity is important to engage students. Pollard and Duvall suggested also using prizes, games, good competition, creating artwork, media, acting out (algorithms), and even rewarding students with stickers and smileys on their papers.

Getting students to generate / reproduce information is a powerful tool. Invention activities are one way to get students attempt the solution and then apply the concept to another area.

Hichens and Lister noted that students expect the teachers to go beyond what is written in the lecture notes, and the teachers to assess student learning during the lectures and adjust the teaching accordingly. Reading straight out from the lecture slides, or making them feel bad / lazy, and going over material too fast / too slow are absolute no no's!

References:

deWinstanley, Patricia Ann and Bjork, Robert, A. (2002). Successful Lecturing: Presenting Information in Ways That Engage Effective Processing. New Directions for Teaching and Learning. No. 89, pp 19- 31.

Hitchens, Michael and Lister, Raymond. (January 2009). A Focus Group Study of Student Attitudes to Lectures. Eleventh Australian Computing Education Conference.

Pollard, Shannon and Duvall, Robert. (2006). Everything I Needed to Know About Teaching I Learned in Kindergarten: Bringing Elementary Education Techniques to Undergraduate Computer Science Classes. SIGCSE 2006. Pp 224 - 228.

Knowledge Transfer

2009-09-03T11:36:00.000-07:00

Most educators are hopeful that their students are able to apply what they have learned in different settings, "from one problem to another within a course, from one course to another, from one school year to the next, and from their years in school to their years in the workplace." (Bransford and Schwartz, 1999). However, researchers have found that people seem to learn things that are very specific (Thorndike and Woodworth, 1901). Further studies have shown that sufficient initial learning is critical in effective transfer, and that concrete examples can enhance initial learning because students see the relevance of new information. But overly contextualized information can impede transfer because information is too tied to the context.

People also forget information easily ("replicative knowing") and people have difficulty applying their knowledge to solve new problems ("applicative knowing") (Broudy, 1977). That is people have difficulty knowing "that" (replicative), and knowing "how" (applicative). What seems to help is people know "with" other concepts / experiences. This is related to Piaget's learning theory of assimilation and accommodation.

Contrasting cases are especially useful for people to "learn with" their experiences. The differences among the contrasting cases help people to notice the pattern that persist among the cases. After the students have a chance to work through some contrasting cases, a lecture that follows results in much greater retention than simply working through the contrasting cases only or have the students summarize what they learned after a lecture.

In order for students to transfer their knowledge from one area to another, they need to "let go" of previously held ideas and behaviors. It is not the same as repeating the same idea / behavior in a new situation. The word "insight", coined by Land, inventor of the Polaroid Land camera, highlights the importance of "letting go" of previous assumptions and strategies rather than simply repeating them (Land, 1982). For Land, insight is "the sudden cessation of stupidity". It is not enough to try to adapt old ideas to new situations. Thus, effective learners revise and actively control their learning when things do not work.

Knowledge transfer also benefits from actively seeking others' ideas and perspectives. Other essential ingredients include: tolerance for ambiguity, courage spans, persistence in the face of difficulty, willingness to learn from others, and sensitivity to the expectations of others. All these help people to be life long learners.

How can students learn to develop these characteristics? Bransford Schwartz suggested lived experiences (spending time in a different country), learning to play a musical instrument, learning to perform on stage, learning to participate in organized sports activities. Learners need to self evaluate in areas such as their commitment to excellence, their need to be in the limelight, their respect for others, their own fears and strategies that may be hampering their progress. Such meta-cognitive reflection are part of "knowing with" new information.

Having students evaluate their own confidence level and then realizing whether their confidence level matches their competence helps them realize whether they are ready to move on to more challenging problems or new problems, or whether they should seek help before they can attempt the problems. Some learners need to know the dangers of confidence when there is little competence. All these prepare the learners to transfer their knowledge to new situations and domain areas.

Reference:

Bransford, J. and Schwartz, D. (1999). Rethinking Transfer: A Simple Proposal with Multiple Implications. Review of Research in Education, Vol 24, pp 61-100.

Broudy, H.S. (1977). Types of knowledge and purposes of education. In R.C. Anderson, R.J. Spiro and W.E. Montague (Eds.), Schooling and the acquisition of knowledge (pp. 1-17), Hillsdale, NJ: Erlbaum.

Land, E.H. (1982). Creativity and the ideal framework. in G.I. Nierenberg (Ed.), The Art of Creative Thining. New York: Simon & Schuster.

Thorndike, E. L., and Woodworth, R.S. (1901). The infludence of improvement in one mental function upon the efficacy of other functions. Psychological Review, 8, 247-261.

Expert Tutors

2009-08-27T13:36:00.000-07:00

One of the most distinguishing characteristics of an expert tutor is their considerable attention given to motivating students as well as providing cognitive information to them. They seem to have a working model of each tutee on when they need more emotional support, and when they need to be challenged in their state of knowledge construction. Lepper and Woolverton proposes the INSPIRE model which highlights seven critical characteristics of expert tutors:

I - intelligent. Expert tutors know their subject well and are able to guide their tutees in knowledge construction.
N - nurturant. Expert tutors are highly supportive and nurturing of their students.
S - Socratic. Expert tutors engage their students using questions rather than directions or assertions, they provide hints and not answers.
P - progressive. Expert tutors carefully plan their tutoring sessions of increasing difficulty and complexity, but they are also flexible in adjusting their session in response to the student's learning.
I - indirect. Expert tutors deliberately avoid overt criticism of their students' mistakes but rather often pose questions that imply the existence of the errors.
R - reflective. Expert tutors ask their students to reflect aloud on what they have just done immediately after a successful attempt in their problem solving.
E - encouraging. Expert tutors keep their students' interest / attention / involvement high by instilling confidence and a sense of curiosity.

Reference:

Lepper, M. and Woolverton, M. (2002). "The Wisdom of Practice: Lessons Learned from the Study of Highly Effective Tutors" in Improving Academic Achievement. Elsevier Science, pp 135 - 158.

Framing

2009-08-22T11:42:00.000-07:00

Solving problems usually involve a variety of concepts and skills. Some problems can be approached from a number of angles but usually, when one goes down a "wrong track", it may take sometime to recover unless one is aware of the backtracking points and be able to try alternate paths. The perception / judgment that is used in problem solving is called epistemological framing. It refers to the class of tools and skills that one would bring to a particular situation or context for problem solving. As a simple example, some students may rely on memorized facts to solve a problem, while others may rely on logical reasoning, etc.

Bing and Redish (2009) identify four common framing clusters that students commonly use of mathematics to solve physics problems: calculation, physical mapping, invoking authority, and math consistency. Calculation refers to the algorithmic use of established computational steps to derive a solution, e.g. calculus rules, geometry rules, algebraic rules, etc. Physical mapping refers to the mapping between mathematics with the student's intuition of the physical or geometrical situation at hand to support their arguments and reasoning. Invoking authority points to the resource / book / journal / quote / person / etc. to support a claim. Math consistency appeals to the other math ideas and concepts that are demonstrably consistent to offer validation of an argument.

As I reflect on these four framing clusters, I wonder how these clusters can be detrimental for beginning computer science students. Take for example, calculation, the meaning of "=" in math as equating two entities, like x = y, is so different from computer science use of assigning one value to a variable. Similarly, there is hardly any connection between how computer science models physical objects, like tree, or student, and how we intuitively understand and interact with them. Students are also often surprised at what they can do and cannot do with a programming language. They lack the source(s) of "authority" to guide them in their learning. One comment that I often hear from students when they are learning a new programming language is "I didn't know you can do that!". Finally, although computer science students know that computers are consistent and logical, the subtleties of programming language syntax and the precision of logic that is also highly dependent on the sequence of execution in a program can be frustrating and overwhelming to them. Identifying some of these framing clusters that students bring into the classroom may help in their learning process.

Reference:

Bing, T. J. and E. F. Redish (2009, Jul). Analyzing problem solving using math in physics: Epistemological framing via warrants. Available at http://arxiv.org/pdf/0908.0028.

What can we learn from Video Games?

2009-08-06T15:15:00.000-07:00

How do we motivate people to learn? Well, Gee (2005) notes that "[u]nder the right conditions, learning, like sex, is biologically motivating and pleasurable for humans (and other primates)." It is the same hook that game designers use to attract gamers (see link), so we can learn a great deal about learning from video games. Gee organized these attributes in video games in 3 categories and for each category a number of principles:

Empowered Learners - gamers / learners need to have some sense of control

They feel that they are co-designers of the game or learning, they can customize their game play or learning experience, they can take on a new identity (and for learners to adopt the culture and role of a biologist / computer scientist / etc.), and be able to manipulation and distributed knowledge in the game virtual world or in the real world.

Problem Solving - gamers / learners need to be exposed to appropriate information and problems

They need to be exposed to well-organized problems that are not too complex nor too trivial, and problems should be pleasantly frustrating and there is payoff. There should be cycles of practice to help them develop their expertise, information is given 'on demand' and 'just in time' so they don't feel overwhelmed, they are exposed to fish tanks and sandboxes (simplified versions of the game / learning content) so they can understand a simple system or try out things without any risk first, and they see their practice of skills as strategies to accomplish their goals.

Understanding - gamers / learners make sense of their world

They want to look at the big picture and be able to think of the system at large, they can attach meanings to their past experiences.

Reference:

Gee, J. (2005). "Learning by Design: good video games as learning machines." E-Learning, 2(1). pp 5 - 16.

Situated Learning

2009-08-06T13:26:00.000-07:00

Much learning is done within contexts. An average 17 year old would have learned her vocabulary at a rate of 5000 words per year for over 16 years by listening, talking, reading, and interactions. In contrast, if vocabulary were taught simply by abstract definitions and sentences taken out of context, it is hardly even possible to learn 100 to 200 words per year.

Students should be exposed to and then adopt the culture of which the tools they are taught to use. This requires the support of a community, and learning is a process of enculturation. The activities that the students will be exposed to will be authentic (i.e. ordinary practices of the culture, and not just classroom or toy problems), and these activities are framed by its culture.

While we want our students to have practical knowledge on how to use the tools and develop practical skills, we also want them to develop deep thinking and cognitive sills. Within the context of situated learning, this is called cognitive apprenticeship. It begins with problems and practice in situ, and moves them beyond the traditional practices by emphasizing that practices are not absolute, and students are encouraged to generate their own solutions with other members of the culture, which we sometimes call a community of practice.

Reference:

Brown, J., Collins, A., Dugid, P. (1989). "Situated Cognition and the Culture of Learning". Educational Researcher. 18(32). pp 32 - 42.

Interactive Engagement vs. Traditional Methods

2009-08-05T13:40:00.000-07:00

A study of 6,542 students (Hake, 1998) who took introductory physics courses in high schools, colleges and universities was conducted to compare the effectiveness of interactive engagement in the classroom as compared to traditional lecture style presentations. Not surprisingly, the average gain (measured as per Halloun-Hestenes Mechanics Diagnostic test, Force Concept Inventory, and Mechanics Baseline test) due to interactive engagement delivery is significantly higher than traditional courses.

In another paper by Hake (1997), he lists several interactive engagement methods that have been used successfully for teaching physics. These include collaborative Peer Instruction, microcomputer-based labs, concept tests, modeling, active learning problem sets or overview case studies, physics-education-research based text or no text, and socratic dialogue inducing labs.

It should be noted that interactive engagement is "necessary but not sufficient for marked improvement over traditional methods" (Hake, 1997) since there are a number of colleges which have marginal gain even when interactive engagement activities were used.

I like the Epilogue that Hake included in his 1997 article:

I am deeply convinced that a statistically significant improvement would occur if more of us learned to listen to our students....By listening to what they say in answer to carefully phrased, leading questions, we can begin to understand what does and does not happen in their minds, anticipate the hurdles they encounter, and provide the kind of help needed to master a concept or line of reasoning without simply "telling them the answer."....Nothing is more ineffectually arrogant than the widely found teacher attitude that ’all you have to do is say it my way, and no one within hearing can fail to understand it.’....Were more of us willing to relearn our physics by the dialog and listening process I have described, we would see a discontinuous upward shift in the quality of physics teaching. I am satisfied that this is fully within the competence of our colleagues; the question is one of humility and desire.
Arnold Arons, Am. J. Phys. 42, 157 (1974)

I often wonder whether this applies to Computer Science. Afterall, don't we know pretty well how our students think? ... or do we?

Reference:

Hake, Richard. (1997). "Interactive engagement methods in introductory mechanics courses". Retrieved on August 6, 2009 from http://www.physics.indiana.edu/~sdi/IEM-2b.pdf.

Hake, Richard. (1998). "Interactive-engagement versus Traditional Methods: A six-thousand-student survey of mechanics test data for introductory physics courses". American Association of Physics Teacher. 66(1), pp64-74.

Deliberate Practice, Part 2

2009-07-16T14:07:00.000-07:00

We tried to dissect the elements of "deliberate practice" [Ericsson, et al., 2006] during today's CWSEI Reading Group meeting. Not all aspects of "practice" are the same. We recognize that some students insist on multitasking while doing homework (e.g., watch TV or listen to iPod while engaging in practice activities). Perhaps the term "deliberate practice" should be reserved for those tasks that do not readily permit TV or other multitasking interferences. Ray Lister suggested another paper related to practice quality [Plant, et al., 2005] that may be of interest.

Some of the elements of practice include foundations that students often do not enjoy, but are recognized as skill development techniques. In music, this includes practice on scales, repertoire, technical exercises, etc. [Sloboda, et al., 1996]. In computer science, this may involve practice with "boring" parts of CS, like math skills, analyzing sort routines, fixing badly designed or poorly documented code, or coding non-interactive applications.

The studies of Sloboda, et al., showed that no matter what skill level, there is a common trend that performers that are better at that skill/age level have spent more hours in deliberate practice. The highest achievers in each level are those individuals that have practiced the most. Performers in the highest level have accumulated a considerably larger number of hours than in the next highest level, and so on. This reinforces the results of other papers (e.g., [Ericsson, 1996; Colvin, 2008]).

There is some debate about whether a long programming assignment is better than a shorter programming assignment. Historically, to convey a CS learnaing objective, programming assignments tend to be longer than necessary (perhaps because "I had to do it that way when I was an undergrad"). But, if a student cannot get the long program to work at all, does this mean the student has failed? What if the student is really close to getting it working, but just can't get it to work, or simply doesn't understand a small component of it? Might it be better to have many shorter programs/exercises and more manageable or self-contained milestones, thus building confidence for the student?

References:

Colvin, Geoff. Talent is Overrated. Portfolio (Penguin), 2008.

Ericsson, K. A. The influence of experience and deliberate practice on the development of superior expert performance. In K. A. Ericsson, N. Charness, P. Feltovich, and R. R. Hoffman, R. R. (Eds.). Cambridge handbook of expertise and expert performance (pp. 685-706). Cambridge, UK: CambridgeUniversityPress, 2006.

Plant, E. Ashby; Ericsson, K. Anders; Hill, Len; Asberg, Kia (2005). "Why Study Time Does Not Predict Grade Point Average across College Students: Implications of Deliberate Practice for Academic Performance". Contemporary Educational Psychology, v30 n1 p96-116 Jan 2005.

Sloboda, John A.; Davidson, Jane W.; Howe, Michael J.A.; Moore, Derek G. "The role of practice in the development of performing musicians". British Journal of Psychology (1996), 87, pp. 287-309.

Deliberate Practice

2009-07-12T21:54:00.000-07:00

According to extensive studies on how experts develop their specialized knowledge, one of the primary factors is deliberate practice (Ericsson, Krampe, Tesch-Romer, 1993). This means that it is through prolonged efforts to improve performance skills or understanding, whether in chess, sports, music, science, etc., that result in expert performance. Such effortful activities (deliberate practice) need to be carefully designed and administered to optimize improvement with the help of coaches, mentors, teachers, often parents, etc. Many expert characteristics that were once believed to reflect innate talents are actually the result of intense practice extended for a minimum of 10 years.

For computer science, most of the current computing education I have been exposed to do not include significant amount of "practice". There may be some reading assignments, a few programming assignments, but the amount of actual practice is not significant. If expert knowledge does require significant amount of time and effort, we should explore 1) how to deconstruct learning of computer science concepts into sequences of practice activities, and 2) how these activities can be incorporated in the lectures / labs and perhaps even other available technologies, such as online and mobile learning, to promote deliberate practice beyond class time.

Reference:

Ericsson, K.A., Krampe, R.T., Tesch-Romer, C. (1993). The Role of Deliberate Practice in the Acquisition of Expert Performance. Psychological Review. 100(3), 363 - 406.

Student Overconfidence

2009-06-19T08:43:00.000-07:00

People tend to be overconfident in their answers to a wide variety of general knowledge questions, and in particular when the questions are difficult (Plous, 1993). How do researchers study overconfidence? One approach is to ask participants to estimate the probability that their judgment is correct. These estimates are then used to calibrate between confidence and accuracy. A person is perfectly calibrated when his proportion of judgment at a given level of confidence is identical to his expected probability of being correct. Another approach is to ask participants to give a "confidence intervals" that have a specific probability (usually .9 or .98) of containing an unknown quantity. In one study, participants were 98% sure that an interval contained the correct answer but they were right only 68% of the time.

In one of the summer sessions of an introductory CS courses, 16 students out of a class of 68 students overestimated their final course grade after they have received feedback from their first midterm, and 3 students underestimated their final course grade.

Overconfidence can be relearned, just like any belief system. People who were initially overconfident could learn to make better judgments after 200 tries with intensive performance feedback (Lichtenstein and Fischhoff, 1980). Arkes et al. (1987) found that overconfidence could be eliminated by giving participants feedback after five "deceptively difficult problems". Yet another study by Lichtenstein and Fischhoff shows that by having the participants generate opposing reasons alone was sufficient to reduce accuracy overconfidence, but this has not been confirmed in subsequent studies.

References:

Arkes, H.R., Christensen, C., Lai, C., and Blumer, C. (1987). Two methods of reducing overconfidence. Organizational Behavior and Human Decision Processes. 39, 133-144.

Lichtenstein, S., Fischhoff, B., Phillips, L. 1980. Training for calibration. Organizational Behavior and Human Performance, 26, 149-171.

Lichtenstein, S., Fischhoff, B., Phillips, L. 1982. Calibration of Probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic, and A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases (pp 306-334). Cambridge, England: Cambridge University Press.

Plous, S. 1993. The Psychology of judgment and decision making. New York: McGraw-Hill.

Plous, S. 1995. A Comparison of Strategies for Reducing Interval Overconfidence in Group Judgments. The American Psychological Association Inc. 80:4 p 443-454

Some other resurces I found: papers, chapter on overconfidence, presentation on certainty

Why Don't Students Attend Class?

2009-06-18T13:45:00.000-07:00

According to Friedman, Rodriguez, and McComb, who did a study on 350 undergraduate students on their reasons for attendance and nonattendance in class, they conclude that "males and females, older and younger students, students who live on and off campus, student who do and do not have jobs, students have light and heavy course loads, and students who do and do not pay their own way in school attend classes with equal frequency." The only difference is that students with better academic records attend classes more regularly.

As to the differences in course characteristics, "students attended faculty taught courses less often than GTA [graduate TA] taught classes, larger classes less often than smaller classes, and natural science classes less often than others." However, courses that penalize absences encourage student attendance in any of the above course settings.

The primary reason why students attend class is internal. They feel they have the responsibility to attend class, their interest in the subject matter, and also getting the material first hand rather than from other sources. Another study has also shown that better attendance is associated with higher grades (Wyatt 1992).

In another article (Jensen and Moore 2009), students who attend help sessions are mostly A and B students and virtually no D and F students. Results also show that students get better grades if they attend these help sessions, and they also attend class more often.

The bottom line is that attendance seems to have a correlation with higher grades. The question is do students really want higher grades, or they are just satisfied with a pass? It will be interesting to survey students on what grades do they realistically expect to get given the effort they are willing to put into the course.

References:

Friedman, P., Rodriguez, F., McComb, J. 2001. Why Students Do and Do Not Attend Classes, Myths and Realities. College Teaching. 49:4, p124-133.

Jensen, P., Moore, R. 2009. What Do Help Sessions Accomplish in Introductory Science Courses? Journal of College Science Teaching. May/June 2009. p60-64.

Wyatt, G. 1992. Skipping class: An analysis of absenteeism among first-year college students. Teaching Sociology 20:201-7.

Invention Activities

2009-06-05T22:31:00.000-07:00

Knowledge transfer from one context to another depends on student learning at least two things: 1) the relevant concepts or skills, and 2) the situations to which they apply. Students are more likely to transfer knowledge from one context to another when instructional examples are abstract and relatively free of surface details. Instead of "tell-and-practice" where instructors often tell the students about the formula they need to use, and then practice using the formulas, it is much better to allow students to develop their "solutions" to a number of contrasting cases before they are told the formula through a mini lecture. Contrasting cases force the students to see beyond the surface differences and explore the underlying deep structure. These contrasting cases constitute what is called an invention activity where students undertake productive activities to note these differences and produce a general solution for all these cases. Such productive activities help students to let go of old interpretations and develop new ones.

Schwartz particularly advocates the use of mathematical tools or procedures in solving invention activities to encourage preciseness and yet general in the solution presentation. They also allow reflection on how the structure of the mathematical tools accomplish their work in the solution of the problems. However, this does not have to be the case. Invention activities can prime students in areas that do not involve quantitative analysis (Yu and Gilley, 2009).

In Schwartz's case, the combination of using visual (problem presentation), numeric (expressing solutions in quantitative mathematical terms), and verbal (student presentation of their solutions) helps to reinforce learning.

In computer science, when we ask our students to create "invent" a solution to a programming assignment, this is an invention activity. The difference with other cases is that invention activities are used as scaffolding for further learning, whereas, in this case, the programming assignment is used to learn the material. In other cases, the students usually don't "invent" the final solution. In the case of computing, the students must get to the final solution themselves. Is that why so many students get frustrated with computer programming? After all, Schwartz did note that students can get tired of repeatedly adapting their inventions.

References:

Schwartz, D., and Martin, T. 2004. Inventing to Prepare for Future Learning: The Hidden Efficiency of Encouraging Original Student Production in Statistics Instruction. Cognition and Instruction. 22(2) 129 - 184.

Yu, B., Gilley, B. 2009. Benefits of Invention Activities Especially for Cross-Cultural Education. Retrieved on October 16, 2009 from http://www.iated.org/concrete2/view_abstract.php?paper_id=8166.

Blooming in Teaching and Learning

2009-06-05T13:03:00.000-07:00

It is important to align appropriate teaching activities with learning outcomes, and students need to know at what level of cognitive engagement they are expected to demonstrate. If only facts are presented in lectures, but students are expected to provide an analysis in their assignment but are never taught how, this may not be effective in assessing student's capabilities. Bloom's Taxonomy provides a common language to coordinate what is taught and what is being assessed. The six different levels of Bloom's Taxonomy are: knowledge, comprehension, application, analysis, synthesis, and evaluation. These are further revised with a set of verb counterparts: remember, understand, apply, analyze, create, and evaluate. Here are three ways of using "Blooming" to enhance learning:

Instructor assigns Bloom level to grading rubric, and provide additional learning activities to improve the levels where students have low scores.
Introduce Bloom levels to students and students are asked to "bloom" questions asked in class (i.e. rank the questions according to Bloom's levels). This helps students to develop meta-cognitive skills and reflection on their learning. Students are also shown the class average at each Bloom level after a test, and evaluate their score at each level.
Students are taught the Bloom levels and write questions at each level in small groups. The groups exchange the questions and rank them to see if they correspond to the levels intended.

Most of computer science education will likely require higher level in Bloom's taxonomy. However, students may operate at lower level in learning the material. Recognition of this discrepancy and the use of specific activities in achieving higher Bloom's level may help students to become better computer scientists. Instructors will also benefit in articulating at which level they expect their students to demonstrate by comparing what is stated in the course outline and what is being taught and tested.

Reference:

Crowe, A., Dirks, C., Wenderoth, M., Biology in Bloom: Implementing Bloom's Taxonomy to Enhance Student Learning in Biology, CBE - Life Sciences Education, Vol. 7, 368-381, 2009

Item Response Theory

2009-06-04T10:08:00.000-07:00

How do we (as instructors) decide whether a test is "hard" or "easy"? Most of us will answer something along the line .. "it all depends". I find this observation which Hambleton et al. make of the common responses to this question interesting: "Whether an item [or test] is hard or easy depends on the ability of the examinees being measured, and the ability of the examinees depends on whether the test items are hard or easy!" Not very helpful, isn't it? Item Response Theory is a body of theory which applies mathematical models to analyze student scores of individual questions from a test to facilitate comparison of the difficulty level of the questions and their capabilities to differentiate student abilities. It is based on two basic postulates: 1) the performance of an examinee can be predicted by a set of factors called traits (or abilities), 2) the relationship between examinees' item performance and the set of traits can be described by an item characteristic function or item characteristic curve (ICC) like the one in the graph above. The x axis is the trait or ability score, and the y axis is the probability of the examinee with certain trait or ability score to obtain the correct answer. As the ability of an examinee increases, so does the probability of a correct response to an item.

Each item in a test has its own ICC and the ICC is the basic building block of IRT. The steepness of the graph shows how well the item can differentiate examinees with low and high abilities. A flat curve is a poor indicator while a steep curve, like the one shown above, is a good indicator. If several ICC's are plotted in the same graph for the corresponding test items with the same shape, the curves on the left (or top) correspond to the easier items than those on the right (or bottom).

By analyzing the examinees' scores of each item from an exam using IRT software, one can have an idea 1) which questions are good indicators of assessing student abilities or not, and 2) objectively respond to which questions are "easy" or "hard".

Reference:

Baker, F. The Basics of Item Response Theory. Available online here.

Graph is taken from http://echo.edres.org:8080/irt/ where one can also find a great deal of information on IRT.

Hambleton, R., Swaminathan, H., Rogers, H. Fundamentals of Item Response Theory. Newbury Park: Sage Publications. 1991.

Why and how do students take notes in class?

2009-05-24T23:09:00.000-07:00

Most students take notes in class so that they can review and memorize the information just before the final exam. Students take notes so they do not forget what they believe is important for later "cramming". While most instructors would hope that they reformulate and interpret what has been taught, rather than just copy and regurgitate later, most students tend to do the latter. Students also pick up certain cues from the instructors when they should take notes. These include when instructors:

write on the board
dictate the information slowly
write a title of a section or a list of information
write out the definitions or catch phrases
write or draw macro-textual planning indicators that organize and structure the classes

There are also inhibiting indicators when students do not take notes. These occur:

during discussions of material that do not contribute to the organization of what has been said
when the instructors interact with the students such as during responses by the instructors to students' questions
when there are hesitations in instruction, which the students take as signs that what is being said has not been planned
when the instructors put aside his or her notes, or walk around the classroom

In general, if students perceive that the information is not planned or not written, they do not think that the information is important. Students tend to put more effort in learning (also in notes taking) when the information learning process involves understanding and transformation operations. A matrix structure or concept map structure seems to be more beneficial to the students than an outline structure, or a linear structure. Students also benefit most in notes taking if they reflect and rework the notes to reinforce the structuring of knowledge after the lectures.

Reference:

Boch, F. and Piolat A. (September, 2005). Note Taking and Learning: A Summary of Research. The WAC Journal. Volume 16.

What makes video gaming fun and computer science education not (at least for some)?

2009-05-13T12:58:00.000-07:00

Neuroscientists suggest that human brains are drawn to systems where reward is clearly defined and achieved. The human brain has this dopamine system that keeps track of expected rewards and sends out alerts when those rewards don't arrive as expected. Good video games have rewards everywhere, and feed the brain with this craving of rewards. Good games also force the players to make decisions. Players may make good or bad decisions, but players learn from these decisions and adjust their game play accordingly. If they weigh the evidences, analyze the situations, consult their long term goals, and then decide wisely, they receive good rewards. Otherwise, their desire to attain rewards will drive them to continue in the game. This is the "flow experience" (Kiili and Lainema, 2008) that every game designer wants their players to experience. It is a a state of complete absorption or engagement in an activity and refers to the optimal experience that nothing else matters (Csikszentmihalyi, 1991).

How can teaching computer science, or actually any subject, be as exciting as playing video games? For one, if grades in a course are the equivalence of rewards in a video game, then the frequency and the number of occasions where students can earn course grades need to be drastically increased. Instead of grades being awarded only through the few assignments, the midterms (possibly at most two in a course), and a final exam, perhaps a grading system where students accumulate grades more often, such as in every lecture and every lab, may be more effective. Rather than big assignments, midterms or final that may make up 50% of the final course grades, students may be more motivated in learning, and willing to learn through their mistakes, if the stake is lower, and they are given more chances to learn from their mistakes. Instead of coming to the lectures and expecting that the material not to be tested until the next assignment or test, each student would come to the lecture with an attitude and expectation of increasing their "score". Such constant feedback and rewarding may simulate the effect of the dopamine system. But then, is this really learning, or manipulating our students like Pavlovian subjects?

References:

Csikszentmihalyi, M. (1991). Flow: The Psychology of Optimal Experience. New York: Harper
Perennial.

Kiili, K., Lainema, T. (2008). Foundation for Measuring Engagement in Educational Games. Journal of Interactive Learning Research, v19 n3 p469-488 Jul 2008.

Johnson, S. (2005). Everything Bad is Good for You. New York: Riverhead Books.

The Good and Bad of Multiple-Choice Testing

2009-04-24T14:14:00.000-07:00

The Testing Effect refers to the improvement of students' performance when students are tested repeatedly of their knowledge. See previous blog entry. Frequent testing using multiple choice questions has also been shown to be effective not just for recall but for higher Bloom level of learning. So the testing effect is not limited to memory recall of facts only, but also to application type of learning. However, the presence of incorrect answers (also known as "lures") in multiple choice questions may cause the students to acquire incorrect concepts via faulty reasoning. Even then, repeated testings produce more positive benefits than negative side-effects.

One way to compensate the negatives of multiple-choice testing is to provide immediate feedback to correct learner's misconceptions and avoid their construction of incorrect knowledge. Another way is to offer a "don't know" option or a penalty for selecting a wrong answer. This can also reduce the amount of guessing. Lastly, a different way of testing may be used, such as short answer questions, which seem to have even more positive benefits than multiple choice questions.

Reference:

Marsh, E., Roediger III, H., Bjork, R., Bjork, E. (2007). The Memorial Consequences of Multiple-Choice Testing. Psychonomic Bulletin & Review. 14(2), 194-199.

Advice to Students: do more testing and less studying!!

2009-04-09T09:44:00.000-07:00

At least for memory recall, taking a memory test repeatedly rather than studying repeatedly results in much better long term retention. The abstract from the first reference below says it all:

Taking a memory test not only assesses what one knows, but also enhances later retention, a phenomenon known as the testing effect. We studied this effect with educationally relevant materials and investigated whether testing facilitates learning only because tests offer an opportunity to restudy material. In two experiments, students studied prose passages and took one or three immediate free-recall tests, without feedback, or restudied the material the same number of times as the students who received tests. Students then took a final retention test 5 min, 2 days, or 1 week later. When the final test was given after 5 min, repeated studying improved recall relative to repeated testing. However, on the delayed tests, prior testing produced substantially greater retention than studying, even though repeated studying increased students’ confidence in their ability to remember the material.Testing is a powerful means of improving learning, not just assessing it.

In other words, if S stands for study, and T stands for testing, a final recall test after the sequence STTT results in much higher retention than SSST or SSSS. In computer science, most of the learning requires reasoning rather than memory recall, although a good repository of learned concepts is definitely an asset to being a good programmer. However, from a number of interviews with students enrolled in a first year programming course, when asked how they prepared for exams, 90% of the students would say reading from lecture notes, textbooks, and only about 10% would mention about doing some coding and testing. The learning-by-experimentation concept seems to be foreign to many students.

It won't be surprising that the result from Roediger and Karpicke applies just as well to reasoning skills as memory recall. What will be interesting for CS is to identify the set of core skills and concepts that expert programmers need to have and apply this strategy of studying and testing (mostly) throughout a program of study rather than just a course, and conduct longitudinal study of their retention and programming skills beyond graduation. Also, how can repeat testing be made "fun" for learners? Is there an "optimal" study and test sequence for CS courses?

References:

Roediger III, H., Karpicke, J. (2006). Test-Enhanced Learning. Psychological Science. 17(3), pp 249 - 255.

Karpicke, J., Roediger III, H. (2008). The Critical Importance of Retrieval for Learning. Science. Vol 319, pp 966 - 968. Link.

Two-stage Cooperative Exams

2009-04-09T08:13:00.000-07:00

The idea of a two-staged cooperative exam is that students take the same exam repeatedly during an extended period of time but in different settings. These settings can be individual in the beginning, then working in pairs, or collaboratively in a larger group. The goal is to turn these testing sessions into a learning experience.

Here is an example of how this is implemented in a large class for midterm or final exams: during the first 30 minutes of the class period, the students take a multiple-choice exam with about 20 - 25 questions in it individually. They hand in the answer sheets at the end of the exam. Then right away, they are given the same multiple-choice exam but with added questions in it, and are asked to work on it collaboratively with someone close by for 45 minutes. They can use books, notes, and other resources. The grade of the exam is calculated based on a weighted average (75%) of the first submission and 25% of the second submission of the exam. However, if this grade is less than the grade in the first submission (i.e. from the solo effort alone), then the final score of this exam is based solely on the first submission.

With this simple change in exam format throughout the term, there has been large improvement in the final exam scores from a mean of 74% to 80%, based only on the solo part of the exam. Although it seems that the collaborative component of the exam may have boosted the final score, a statistical comparison with grades from previous years with no collaborative component in the exams shows that there is no dramatic change in grade distribution. The number of students at the bottom rungs of the ladder are fewer with the two-stage cooperative exam strategy, but there is no increase in the upper rungs.

References:

Yuretich, R., Khan, S., Leckie, R., Clement. (March 2001). Active-Learning Methods to Improve Student Performance and Scientific Interest in a Large Introductory Oceanography Course. Journal of Geoscience Education. 49(2), p 111- 119.

Yuretich, R. Accessing Higher-Order Thinking in Large Introductory Science Classes.

"Well Polished" Lectures May Not be Good for Learners

2009-03-31T17:01:00.000-07:00

Most trainers are "conditioned" to expect improvements in the performance and also the "happiness" of their trainees. Instructors don't see how their students perform after a course, but during the course, it is important for them, as well as for the students, to see that progress is being made with measurable improvements. However, most training methods produce impressive short term improvements with no long term benefits. As an example, practice drilling exercises may give the impression that the students have actually acquired a set of knowledge and skills through increasing familiarity of the material. However, it has been shown that the more familiar the learners believe they are with a subject, their actual level of comprehension is actually inversely related (as least in certain domains like physics or music). As another example, if answers are given readily, learners adopt a mentality that they "knew it all along". Hence a well polished lecture where the listeners can follow easily may give the illusion that the learners have already learned or known the material, which in fact, may not be the case when they are called upon the task to actually solving a problem based on the material presented.

Bjork gives five examples of training that may produce durable transfer of knowledge and skills in post-training environments. This implies long term retention and transfer of knowledge and skills to new situations. The key is to introduce meaningful and desirable difficulties in the training. Here are the five examples:

Varying the conditions of practice such as the (un)predictability of the training environment, scheduling practice exercises in variety and in random fashion rather than a block of training on one specific task, etc.
Providing contextual interference such as designing and interleaving materials to be learned, rearranging the material presentation that is inconsistent with an outline, or adding to the complexity of the tasks to be performed.
Distributing practice on a given task over time rather than "cramming" the material in a short session.
Reducing feedback to the learner (mainly applicable to motorized skill).
Using tests as learning events rather than providing more study opportunities.

In Computer Science, the lab exercises and problem sets do help students in their learning, especially when the complexity and context of the problems are varied. By mixing the type of problems to be solved, changing the duration between similar types problems to be solved, and the use of tests to continue monitor student progress, not just within a course, but over several courses, we may begin to get a clearer picture of our student learning.

Reference:

Bjork, R. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing About Knowing (pp. 185-205). Cambridge, MA: MIT Press.

Demos .. learning or entertainment?

2009-03-23T11:39:00.000-07:00

Doing live code demos in class can help in engaging students in their learning. Students get to see the effects of code changes right away rather than just hearing about the concepts through a lecture presentation. However, at least for physics, students who observe demonstrations perform only slightly better, but not statistically significant, than those who do not. Without active participation in the demonstration, students are not engaging with their learning, just like listening passively in a lecture. The only difference is that the demonstration may be a bit more entertaining and may have some affective effects on the students.

To make the most out of classroom demonstrations, one simple strategy is to simply ask the students to predict what will happen before doing the demonstration. Adding one or two minutes in having the students think about the topic under discussion, and predict what would happen if there is a change, turns out to have dramatic effects on student learning. If students are also required to discuss their predictions with their peers, their learning can be improved even more.

Reference:

Crouch, C., Fagen, A., Callan, J., Mazur, E. (June 2004). Classroom demonstration: Learning tools or entertainment?. American Association of Physics Teachers. 72(6), p 835 - 838.

What students need to know to solve Math / CS problems?

2009-03-23T11:06:00.000-07:00

According to Richard Mayer, there are four essential stages one needs to go through to solve a typical mathematical problem like the following:

Floor tiles are sold in squares 30 centimeters on each side. How much world it cost to tile a rectangular room 7.2 meters long and 5.4 meters wide if the tiles cost $.72 each?

The four stages are: problem translation which converts each sentence into an internal representation, problem integration which involves putting the different pieces of information into a coherent whole, solution planning which involves the selection of the most appropriate strategy to solve the problem, and solution execution which involves carrying out the procedures to derive the solution.

The same four stages of problem solving can be applied to solving Computer Science problems. Problem translation involves a good data representation, usually in the form of a data structure, database design, or file structure. Once we have a good data representation, the flow of data and its transformation need to be analyzed. A good strategy usually in the form of a good algorithm needs to be selected next, and finally, the problem can be coded in a programming language for execution.

In CS, we have at least a course for each of these steps: Data Structure, Systems Analysis and Design / Software Engineering, Algorithm Design, and Programming Languages. The process of solving problems in computer science is indeed non-trivial, but yet, in many CS1 courses, we expect our students to be able to do all these while the primary focus of the course is probably just getting students to learn a programming language. Since the first course in CS usually determines whether a student will continue into the discipline, care must be taken not to overwhelm the students with too much material and frustrate them in their learning.

Reference:

Mayer, R. (2007) Learning and Instruction. Prentice Hall.

Learning Focused Course Transformation

2009-03-16T09:09:00.000-07:00

At the United States Air Force Academy, a learning-focused transformation of Biology and Physics core courses was made to support deep student learning. This involves first transforming the learning goals from using terms like "list, find, calculate, describe, use, what, and when" to "explain, analyze, apply, create, predict, and evaluate". Students are also exposed to familiar and concrete settings where the knowledge can be applied. E.g. students are asked to serve as "expert witness" in a trial which requires their knowledge on gene expression, or they have to explain how spies can tap phone lines during the Cold War using Faraday's law. Class lessons include mini-lectures (about 10 minutes) and the rest of the time is mostly spent on learning experiences through activities and exploration. Instructors become learning facilitators rather than just lecturers.

The ultimate question is whether students learn just as much from activity based lessons as from traditional lecture style of delivery of content. In Computer Science education, students are invariably exposed to learning through activity based programming assignments since the nature of the discipline is mostly practical and applied. However, programming is not the only activities that students can be involved in, even though it is the most natural one. Especially if one of the learning goals is the development of abstract thinking, care must be taken not to over-emphasize the programming aspect that may monopolize the time and attention the students should spend. Programming projects can take up a lot of time and the students may end up gaining programming skills and not other skills. Here is where concise learning goals need to be articulated and the proportion of time for each learning goal is matched appropriately with the learning activities.

Reference:

Sagendorf, K., Noyd, R., Morris, D. (2009). The Learning-Focused Transformation of Biology and Physics Core Courses at the U.S. Air Force Academy. Journal of College Science Teaching. January / February 2009.

SIGCSE Presentations on Making CS More Relevant & Engaging

2009-03-12T15:15:00.000-07:00

Here are a few highlights from SIGCSE 2009, Chattanooga, TN. These highlights are about making computer science "interesting" and "more relevant" to students that normally bypass traditional CS courses:

1. "Rediscovering the Passion, Beauty, Joy, and Awe"--panel presentation:

Dan Garcia (Berkeley): CHANGE (Obama style?) has come to computing. Let's avoid the old style of syntax-driven CS 0/1 curriculum. Let's let students choose their own projects, mix of CS courses, partners, etc. In elementary schools, parents help out after school; so, why can't we get [CS-type] moms & dads to help out in the computing club? "But, we want a winning basketball team, so not just any mom or dad will do." When constructing assignments and putting together lecture material, think relevance! We need to motivate students so that they will want to spend hours of their own free time on programming ... just for fun.

Eric Roberts (Stanford): CS enrollment at Stanford is "skyrocketing", wiping out previous CS1 losses post-dot-com. Programming continues to be a very important skill that needs to be emphasized and taught more effectively. "The best programmers are several orders of magnitude better than the average programmer." At Google, for example, they represent one-tenth of 1% of the applicant pool. The best programmers are 300 times as good/productive as Google's typical programmer.

Zachary Dodds (Harvey Mudd College): suggests a breadth-first approach to CS, including functional programming. Right now, "What is learned is the square root of what is taught."--implying not much is learned/remembered from a typical CS course.

2. Microsoft's exhibit on computational thinking. An excellent book, edited by Yan Xu (former MSc student at UBC) is: Transform Science: Computational Education for Scientists (CEfS), Special Edition. Microsoft Research was giving away free copies at the conference. Many authors, including former UBC STLF Beth Simon, contributed 1-2 page positions and short research papers on educational themes in computer science, focusing on the current mismatch between typical CS programs and typical science programs. For example, many authors claim that CS courses are not serving biology, chemistry, physics, earth & ocean science, biological engineering, etc., students very well. We need to develop new courses that take the highlights of numerous first, second, and third year CS courses, and condense them into digestible units for such non-CS students. Such highlights include topics in programming principles, an easy-to-learn powerful language (say Python), discrete math, algorithms, complexity, scripting, database topics, etc., should be made available to biology students using examples (taken from biology, etc.) that are relevant and interesting to the students. Forget about command-line driven programs that compute interest, etc.; instead, stick with real bioinformatics examples. Incorporate visualization techniques, problem-driven applications, modelling, simulation, etc.

3. Owen Astrachan created a new CS course at Duke University for arts students, theater students, varsity athletes, would-be lawyers, etc. Owen created a very interesting, non-programming, non-math, CS course that would appeal to non-CS/non-science types. And it did! 250 students enrolled in his experimental and engaging course. He got several guest speakers (some of whom were Duke alumni) to speak on their areas of expertise. The topics included case studies of network protocols, privacy issues, social issues, copyright issues, high-profile/controversial law cases, etc. Owen gave us take-home copies of his midterm and final exam. The exams included questions on IETF standards, Internet voting, DNS servers, security, Skype and security, worms, Flickr, BitTorrent, Richard Stallman's Free Software Foundation, copyright act and RIAA, IPv6, spam, cookies, iTunes, iPhone, P2P, etc.

Knowledge Transfer Assessment

2009-03-06T12:37:00.000-08:00

There are four ways to test whether students have acquired knowledge transfer skills according to Mayer (2001):

Troubleshooting - by asking students why a system does not work.
Redesign - by asking students for a redesign of a system for a different purpose.
What-if - by asking students what would happen under other conditions.
Principle - by asking students the function of a component in the system, or why a component behaves the way it does.

In Computer Science, this is pretty easy to do. Here are some examples:

Troubleshooting - have students debug a program, or discover and debug another student's program.
Redesign - have students build another version of an application.
What-if - have students compare and contrast the use of different algorithms, database design, logic design, infrastructure, etc.
Principle - have students construct context diagram, use case diagrams, etc. and explain how one component functions within the entire system.

References:

Mayer, R. (2001). Multimedia Learning. New York: Cambridge University Press.

Too Much "Seductive Details" In Lectures

2009-03-06T12:18:00.000-08:00

Bet that title caught your attention ... sex and death are inherently interesting (Kintsch, 1980). So if you want to spice up your lectures, inject some sex and death. But make sure they are relevant. Seductive details are high interest to students but if they are irrelevant to what the students are learning, they are considered as extraneous details, and they actually have a negative effect on student learning.

Learners have only a limited amount of processing capacity available to them for learning. Like a battery, if the energy is wasted on irrelevant material, there is just not enough energy left for the relevant material. What's more, high interest details take up more energy than low interest details, so if you add more "seductive" material in your presentation, you are leaving your students with less energy for the more important material you want to present.

Mayer et al's article (2008) concluded that increasing irrelevant details even though they are of high interest does not appear to affect learners in their understanding of material (as measured by their retention of material), but they do disrupt their construction of a coherent mental model of the to-be-learned system (as measured by their transfer ability performance).

References:

Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why would anyone read a story anyways? Poetics, 9, 87-98.

Mayer, R., Griffith, E., Jurkowitz, I., Rothman, D., (2008). Increased Interestingness of Extraneous Details iin a Multimedia Science Presentation Leads to Decreased Learning. Journal of Experimental Psychology: Applied. 14(4), 239-339.

Affect and Cognition

2009-02-16T09:01:00.000-08:00

Learning objectives can be classified into three domains: cognitive, affective, and psychomotor (Bloom Taxonomy). Affective domain refers to learning outcomes in areas such as emotions, moods, attitudes, and feelings, which has been shown to strongly influence cognitive outcomes. As an example, the production of adrenaline and dopamine during emotional experiences affects the transfer of information from short term to long term memory, as well as the level of motivation and cognitive engagement.

Computer Science education is heavy on the cognitive domain but not so for the affective or psychomotor domains. If according to Bloom that holistic education should include all three domains, then how can this be incorporated in CS education? Other disciplines of study have used fieldwork to promote development in these domains. Students go on fieldtrips and work in groups in their learning, as well as developing friendships during the off hours social functions and activities. Engineering students have their share of pranks and parties, and for computer science students, one of the most popular forms of team activity is online video gaming. But these may not be enough to attract the students to these programs or even promote these program to the extent we like to see especially for female students. Perhaps we need to revisit the "art" of computer programming as Knuth has proposed. There is the construction of a beautiful program which seems to be missing in our current CS education, where programming can give our students both intellectual and emotional satisfaction (Ershov, 1972).

Reference:

Ershov, A. P. Aesthetics and the human factor in programming. Comm. ACM 15 (July 1972), 501-505.

Student Perceptions of Their Grades

2009-02-09T11:58:00.000-08:00

In a recent study of 278 students from an introductory biology course at the University of Minnesota where students were asked to predict their final course grades after each of the four exams given throughout the term, the following are some of the findings:

On the first day of class, more than 90% of the students believed they would earn an A or B in the course.
Students who earned A's and B's in the course at the end predicted they would earn lower grades than they actually earned. Students who earned C's, D's, and E's predicted they would earn higher grades than they actually earned.
Students who earned high grades were more likely to attend class, submit extra assignments for credit, and attend help-sessions than students who earned low grades.
On average, it is unlikely that students will significantly improve their grades deep into a semester.

Students are poor predictors of their grades and thus often fail to regulate their learning. First year students are usually not used to the expectations and demands of college level courses, which may explain the reason for their high expectations at the beginning of the course. The students need to be made aware of this and appropriate support put in place to encourage them to participate in order for them to achieve their goals. Perhaps the extra assignments / help-sessions / etc. should be made mandatory to assist first year students to transition into college life.

Reference:

Jensen, P., Moore, R. (2008). Students' Behaviors, Grades & Perceptions in an Introductory Biology Course. The American Biology Teacher. 70(8), pp483-487.

Self-Regulated Learner

2009-02-03T21:39:00.001-08:00

A self regulated learner (SRL) is one who knows what she wants to learn and be able to monitor and adjusting her learning in achieving her goals. This involves metacognition, motivation, and strategic actions. The diagram on the right is a metacognition model by Winne and Hadwin. "Metacognition is the awareness learners have about their general academic strengths and weaknesses, cognitive resources they can apply to meet the demands of particular tasks, and their knowledge about how to regulate engagement in tasks to optimize learning processes and outcomes."

The diagram looks too simple and cryptic. Starting with the 4 phases in the bottom right, it shows that the learner goes through these 4 phases in self regulated learning. These phases are not linear, that is, the learner may switch from phase 3 back to phase 1 and then to phase 4, etc. In any case, here is a simplistic explanation of the diagram. After realizing what task is to be learned (phase 1), the learner checks with the external and internal conditions (external conditions are conditions external to the learner, like the time / resource, etc. available, and internal conditions are the motivational factors, beliefs of the learners, etc.), the learner engages in some operations to learn what needs to be learned. In the process of learning, she compares the expected Standards she have constructed (e.g. hitting the golf ball straight, or solving a differential equation), and the Products (or results) she is experiencing (e.g. the golf ball went to the left, or the answer to the solution of the differential equation is different from the answer in the textbook), and this results in the Cognitive Evaluations. The learner then may need to revise the goals and plans (Phase 2), repeat the process, or study and find other tactics (Phase 3), and through further adaptions of these learning processes, continue to evaluate the Standards (which may also be revised), and further compare with the Products of her learning.

In computing, the task that students usually encounter is in the from of creating a computer solution for a problem. The usual tactics we provide the students in their learning include: lecture / lab materials, textbook, previously solved problems, google, etc., and students may explore all these in their learning. Learning goals are useful especially if learning goals are constructed in the form of a semantic map so students can refer back the supporting learning goals so they can reassess whether they have learned these to continue. This is also part of self-regulated learning. In any case, students often find themselves "stuck" in their assignments. What other ways can we help them get "unstuck" so they can continue in the process of self-regulated learning?

Reference:

Winne, P., Perry, N. (2000). Handbook of Self-Regulation. Edited by M. Boekaerts, P.R. Pintrich, M. Zeidner. Academic Press.

Moving Students from Rule Based to Creative Problem Solving Skills

2009-01-27T10:08:00.000-08:00

Lubben et al. wrote an article on the change of students' perception of "preciseness" under different contexts. In laboratory / pharmacy settings, measurements are expected to be more precise than in a kitchen setting. Deviations are not acceptable in the laboratory / pharmacy settings, but ok in a kitchen setting. The interesting thing is that students based their judgement on the perceived effects of the result mostly rather than on the instructions given or the process to be used. That is, whether deviations are ok or not, (or what precision really means), depends largely on whether there will be any effects on the results. In the kitchen setting, deviations are ok because the measurements are perceived not to have significant impact on the results, whereas such is not the case in laboratory and pharmacy settings. From this, the authors conclude that context makes a difference in the students' choice of a point-paradigm (drawing conclusions from individual data points) in the laboratory / pharmacy settings as opposed to the set-paradigm (drawing conclusions from the ensemble of all data) used in the kitchen. One of the goal of teaching is to move students from a point-paradigm to a set-paradigm.

In computing, context does not play such a significant role in student's perception of preciseness. Whether the students are writing a program for data analysis in a laboratory or for a game program, preciseness and accuracy are needed. But a similar transformation of students' perception of what is essential in programming needs to take place for computer science students. Novice programmers stick to a "formulaic" strategy in solving problems. To them, there is one solution they need to come up with in solving a problem. Whereas seasoned programmers are free to explore different ways of thinking about the problems, modeling, and solving them. Students eventually learn that the result is what really matters and they realize they can be free to be creative, and innovate and construct their programs.

I started programming with BASIC, and was it fun to create programs with GOTO's! I could create the most convoluted programs and few people would have understood them, but it was fun. Those programs would probably fail under many conditions and any half decent test plan, but it was fun. I wonder whether our computer science education may be prescribing too many rules in programming and rob the students from experiencing the fun and creativity in computer science.

Reference:

Lubben, F., Campbell, B., Buffler, A., Allie, S. (2004). The Influence of Context on Judgements of the Quality of Experimental Measurements. Proceedings of the 12th Annual Conference of the Southern African Association for Research in Mathematics, Science and Technology Education. Pages 569 - 577.

STROBE

2009-01-18T09:59:00.000-08:00

STROBE is a classroom observation tool used by trained observers on learners without interfering with their activities. It yields quantitative data from brief observations of individual learners from around the classroom. The observation occur over 5-min "STROBE cycle" that is typically repeated from 8 to 10 times depending on the length of the class session. Each STROBE cycle proceeds as follows:

First, the observer writes down the following:

the start time of the cycle,
the subjects to be observed, whether it be "entire class", "subgroups", or any specific group,
the major activity, which can be "instructional", "procedural", or other,
the estimated portion of learners on task, which can be "all", "almost all", "half or less", etc.,

Next, the observer selects a learner from the class and observes the selected learner for 10 to 20 seconds, marking the type of engagement the learner exhibits, such as "talking", "listening", "reading", "writing", etc., and the object at whom the learner's engagement is directed ("other learners", "instructor", "self", etc.). This is repeated 4 times.

The observer also observes the instructor and marks the instructor's type and object of engagement. Finally, the observer also notes the number of questions students ask in the cycle.

What STROBE provides is a simple and effective way of gauging the level of engagement of students in the classroom, not necessary learning. It can also be skewed by untrained observers like me who did it for the first time in one of the CS classes recently. As a newbie to this, I found myself picking on the students who were not "norms" to be my targets, those who were working on their computers, those who were talking to other people, etc. I had to keep reminding to randomly pick students and not just the ones that catch my attention!

Reference:

Kelly, P. , Haidet, P., Schneider, V., Searle, N., Seidel, C., Richards, B. (2005). A Comparison of In-Class Learner Engagement Across Lecture, Problem-Based Learning, and Team Learning Using the STROBE Classroom Observation Tool, Teaching and Learning in Medicine, Volume 17, Issue 2 April 2005 , pages 112 - 118

Outliers

2009-01-12T13:23:00.000-08:00

Having heard of the latest book by Malcolm Gladwell, Outliers, and read some of the raving reviews about the book, I was naturally drawn to it while my family roamed the malls during the Christmas holidays. Little did I anticipate that as soon as I started the first page of the book, it was not until three chapters later when I finally left the store with a copy in hand. The condensed message behind the book is simple: outliers are not born, they are made. They are shaped by culture, tradition, communities, and they do have breaks that they seize and take advantage of .. but most of all, they work hard. This reminds me of one of my professors in my undergraduate years who told me that if one wants to pursue a PhD, all one needs is patience, persistence, and money! According to Malcolm, there is this magic number of 10,000 hours of practice and hard work which outliers usually spend to get to where they are at. In a culture where many believe that success comes only to the selected few with special genetic makeup, or by pure luck, the book contains a number of evidences to dispel these perceptions. Also, Malcolm suggests that our culture and tradition may either make or break us. He traces the cause of a number of plane crashes to the cultural influence on the pilots, and the difference in aptitude towards mathematics between Asian and Western children also to their different cultural upbringing.

What does this have to do with computer science education? I have heard so many students who claim that they “are just not made to program”, or they “just don’t have the aptitude” for computer programming. What Malcolm has shown, even though mostly via anecdotal accounts, that success depends largely on repetitive practice and hard work. In computing, it has also been demonstrated that highly intensive training programs have been successful in converting students with no programming background to proficient software developers. The problem that face every computer science educator is how to make this repetitive practice and seemingly hard work that require long hours of engagement to be perceived as challenging, rewarding, and, at the same time, providing the students with a sense of autonomy in their learning – the three essential ingredients, according to Malcolm, that make any work satisfying.

Designing Assessment To Support Students' Learning

2008-12-13T15:19:00.000-08:00

It has been reported that the single most power influence for student achievement is feedback (Hattie, 1987; Black & William, 1998). But it is not just any plain ol' feedback. Informative, timely, concise feedback from the instructors, and feedback which the students actually read and follow up on, are what really count. As such, conducting assessment and providing feedback is an art. Assessment can be enormously expensive, and can be perceived as ineffective and a poor representation of student learning. Gibbs and Simpson (2005) come up with a list of 10 plausible conditions under which assessment supports learning.

Sufficient assessed tasks are provided for students to capture sufficient study time.
These tasks are engaged with by students, orienting them to allocate appropriate amounts of time and effort to the most important aspects of the course.
Tackling the assessed task engages students in productive learning activity of an appropriate kind.
Sufficient feedback is provided, both often enough and in enough detail.
The feedback focuses on students' performance, on their learning and on actions under the students' control, rather than on the students themselves and on their characteristics.
The feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance.
Feedback is appropriate to the purposes of the assignment and to its criteria for success.
Feedback is appropriate, in relation to students' understanding of what they are supposed to be doing.
Feedback is received and attended to.
Feedback is acted upon by the student.

Reference:

Hatte, J.A. (1987). Identifying the salient facets of a model of student learning: a synthesis of meta-analyses, International Journal of Educational Research, vol. 11, pp. 187-212.

Black, P. & William, D. (1998). Assessment and classroom learning, Assessment in Education, vol. 5, no. 1, pp 7-74.

Gibbs, G. & Simpson, C. (2005). Conditions Under Which Assessment Supports Students' Learning, Learning and Teaching in Higher Education, Issue 1, pp. 3 - 31.

Class Observations

2008-12-08T15:46:00.000-08:00

I recently had the opportunities to visit a number of CS classes both domestically and internationally. Here are some observations from these visits that I really like. I will keep adding to this list as I come across other gems!

Use analogies to explain difficult concepts ... e.g. passing of a hockey puck as an analogy to passing parameters, recipe as an analogy to algorithm, etc.
Even in a big lecture, try to learn the names of at least some students. One easy way is to get to know those who ask questions in class. All of us like to be known!
When students ask what-if questions about programming languages, (e.g. what if you divide a integer number with a real number), instead of just giving them the answer, do a quick demo on the computer (if you have one set up). This instills a culture of learning via experimentation.
Repeatedly ask the students if there are any questions throughout the entire lecture, and PAUSE. Students may not ask questions right away, but they know that the instructor is encouraging any questions they may have.
Instead of asking “Any Questions”, try “Who is comfortable with the material presented so far?” and take a poll. The poll can be done via either clicker or raising of hands.
Show Learning Goals at the beginning of class, show Learning Goals before each learning unit, show Learning Goals for each learning activity, show Learning Goals at the end of the class. The Learning Goals should reference back to the Learning Goals as stated on the course outline.
Great teachers are usually experts in what they teach. They know the subject well ... very well!
Students like a variety of presentation styles .. try video, simulation, debate, demonstration (such as having the instructor develop a piece of code live, or work through a problem after a number of unsuccessful attempts.) Students like to see the process of solving a problem rather than just the solution.

Clicker Use in Computer Science Education

2008-12-07T08:55:00.000-08:00

There have been a number of reports on the effective use of Clickers (or Classroom Response System, or Student Response System, or Classroom Communication System, or many other names) across different disciplines. The general comments have consistently been:

increased attendance
greater involvement (especially when good questions are asked)
more interactions with instructor
more interactions among students (especially when peer discussions are used before or after each clicker question)
anonymity increases participation rate
students like immediate feedback on their learning

While there are a number of websites related to clicker questions in physics, mathematics, biology, etc., there does not seem to be any for computer science. In any case, I found this website at Vanderbilt particular useful. The Bibliography section contains a number of links to the use of clickers in a number of disciplines, including Computer Science.

Getting students to ask good questions

2008-11-28T12:09:00.000-08:00

On the rare occasions I bothered reading the textbook when I was a student, "reading" meant looking at all the assigned pages. As faculty, I've finally realized that textbooks are invaluable as a jumping-off point for my own thoughts, ideas, questions, and problems. In CPSC 111, we experimented with "weekly reading questions" (questions marked on completeness, inspired by students' assigned readings) to help students transition from passive reading habits to this type of "interrogation" of the textbook.

Marbach-Ad and Sokolove probe the issue of improving students' reading questions deeply in their 2000 paper (cited below). Their most successful method involves several parts: Ask students regularly for their "best question" after a reading. Give students a clearly defined rubric with real examples for what good questions are. Make many opportunities for students to practice asking questions, evaluating questions, and answering questions. Give student questions pride of place in the classroom, including using wireless mics so that other students can hear students asking the question.

The paper is somewhat interesting but not especially strong from an experimental standpoint. (Of the four techniques I mention above, they provide some quantitative evidence for the combined value of the last two.) However, the ideas may be worth trying out in our own classrooms.

I've appended their rubric for questions below. It's not directly adoptable for CS, but it's an interesting starting point.

Their most interesting mechanism for student practice with questions is to have stable student teams submit their questions as a stack. Before submitting, the students have a few minutes to decide which questions are the best and put those on top. This seems like a simple way to enforce student practice discussing and assessing questions.

Unfortunately, the paper does not address what to do with the questions the instructor received. In CPSC 111, we chose 10 questions at random to answer every week and sometimes answered additional questions that were common or interesting but didn't show up on the random list. This was somewhat satisfying to students. A complementary currency system (where students can purchase answers or invest in questions?) or a voting system (like ActiveClass's) might be more successful.

Marbach-Ad, Gili and Sokolove, Philip (2000). Can Undergraduate Biology Students Learn to Ask Higher Level Questions? Journal of Research in Science Teaching 37(8): 854-870.

Marbach-Ad and Sokolove's taxonomy for student questions in Intro Biology (developed from sample student questions):

Category 0: Questions that do not make logical or grammatical sense, or are based on a basic misunderstanding or misconception, or do not fit in any other category. (This is a "catch all" category that instructors can readily subdivide for teaching purposes--for example, when grading written questions. In this case we chose not to subdivide the category in order to focus on the characteristics of desirable questions.)

Category 1a: Questions about a simple definition, concept, or fact that could be looked up in the textbook (i.e., "what is meant by the polarity of the membrane?").

Category 1b: Questions about a more complex definition, concept, or fact explained fully in the textbook (i.e., "what does it mean when it is says air moves through a bird's lungs?").

Category 2: Ethical, moral, philosophical, or sociopolitical questions (i.e., "carbon monoxide is a very deadly gas binding to hemoglobin much faster than oxygen. If it is so deadly, why are there no carbon monoxide detectors throughout the dorm halls?").

Category 3: Questions for which the answer is a functional or evolutionary explanation. (In this case students begin by asking a question that relates to function and could, in principle, be answered in functional terms--"Why do people have an appendix?"--however, the deeper answer is more often related to evolution than to function (the human appendix is a vestigial organ)).

Category 4: Questions in which the student seeks more information than is available in the textbook (i.e., "what causes the 'rumbling' in your stomach when you are hungry?").

Category 5: Questions resulting from extended thought and synthesis of prior knowledge and information, often preceded by a summary, a paradox, or something puzzling. (i.e., "In chapter 35 it says that caffeine, if taken excessively, can disrupt motor coordination and mental coherence which can cause depression. I known that Coca-Cola has some amount of caffeine in it. Does this mean that excessive consumption of it could lead to depression . . . ?")

Category 6: Questions that contain within them the kernel of a research hypothesis (i.e., "I have heard that some people snore so badly that they stop breathing during their sleep. What correlation is there, if any, between 'heavy snorers' and a higher instance of apnea during REM sleep. Can the attention their nervous system is devoting to a dream, interfere the regulation of respiration?").

Collaborative Groups Useful for Individual Student’s Problem-Solving Abilities?

2008-11-09T07:42:00.000-08:00

Do you wish that your students have better problem solving strategies and abilities to tackle those tricky questions that you give in assignments or exams, or be able to think “outside the box”? Well, apparently this can be a reality, at least according to a research project conducted in the Chemistry department at Clemson University. Students who were given the opportunity to work collaboratively in small groups are found to have better problem solving skills on their own afterwards. The effect of problem solving abilities extends beyond the group work afterwards when they are given problems to be solved on their own.

In computer science education, group work is quite common for programming assignments and projects. However, one key ingredient in improving student problem solving skills is not just to divide the tasks among them (i.e. simply project management), but to have each member discuss, analyze, debate, and articulate how to solve the problem. Especially when there is a mix of students with different problem solving abilities, the result of improving individual problem solving abilities can be significant.

What are your experiences of collaborative work in computer science education? Have you noticed similar improvement in individual problem solving abilities after a team works on a problem together? What kind of collaborative projects have been most useful in computer science education?

Reference:

Cooper, M., Cox. C., Nammouz, M., Case, E., Stevens, R. (June 6, 2008). An Assessment of the Effect of Collaborative Groups on Students’ Problem-Solving Strategies and Abilities. Journal of Chemical Education 85(6). Pages 866-872.

How-To Advice on Think-Alouds to Explore Students' Problem Solving

2008-11-05T14:16:00.001-08:00

The problem: From mathematical perspectives on assignment in CS1 to naive views of probability in AI, students' misconceptions can lead them astray in CS problem-solving. Identifying and addressing those misconceptions is an important step in helping them achieve expertise in the discipline.

Unfortunately, getting inside a student's head to understand how they perceive and address a problem can be tremendously difficult. Just seeing a student's solution to a problem gives scant hints on their thought process.

A solution: Think-aloud protocols (common in HCI) can help us to explore students' thought processes as they solve a problem.

The basic idea of a think-aloud is for you to quietly observe a student as the student solves a problem. The student, in turn, vocalizes (but does NOT explain) their thoughts as they work. To make this effective, have the student practice on a simple problem first, be sure they don't try to clarify or interpret their thoughts for you, prompt them with a simple "Please keep talking." if they fall silent, and sit out of the student's line-of-sight during the process (to reduce the feeling that they're talking to you). Ericsson and Simon suggest mental multiplication (e.g., "24 x 36") as a practice task, which should produce verbalizations like "'carry the 2,' 'fourteen,' 'one forty four,' 'let's see,' and 'seven twenty'" rather than vocalizations like "I'm going to start working on the problem now. I know that my algorithm for multiplication is...". Between the work you see the student performing and the verbalizations, you will hopefully be able to learn a bit more about what's going on inside the student's head.

This is a time-intensive process; so, you'll want to use this technique only for critical questions. You may also want to work with your friendly neighbourhood STLF (or HCI specialist!) either to help plan your think-alouds or to help execute them.

Read more about think-alouds for exploring student thinking in:

Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). Cambridge, MA: Bradford Books/ MIT Press.

Ericsson, K. Anders and Simon, Herbert A.(1998)'How to Study Thinking in Everyday Life: Contrasting Think-Aloud Protocols With Descriptions and Explanations of Thinking',Mind, Culture, and Activity,5:3,178--186.
http://www.informaworld.com/smpp/content~content=a785309769~db=all

Payne, J. W. (1994). Thinking aloud: Insights into information processing. Psychological Science, 5,241,245-248.

Welcome to CSSEI blog

2008-11-05T13:35:00.000-08:00

We'll be using this blog to post material relevant to CSSEI, including brief best practice reports about various teaching & learning techniques.