"Well Polished" Lectures May Not be Good for Learners

Most trainers are "conditioned" to expect improvements in the performance and also the "happiness" of their trainees.  Instructors don't see how their students perform after a course, but during the course, it is important for them, as well as for the students, to see that progress is being made with measurable improvements.  However, most training methods produce impressive short term improvements with no long term benefits.  As an example, practice drilling exercises may give the impression that the students have actually acquired a set of knowledge and skills through increasing familiarity of the material.  However, it has been shown that the more familiar the learners believe they are with a subject, their actual level of comprehension is actually inversely related (as least in certain domains like physics or music).  As another example, if answers are given readily, learners adopt a mentality that they "knew it all along".  Hence a well polished lecture where the listeners can follow easily may give the illusion that the learners have already learned or known the material, which in fact, may not be the case when they are called upon the task to actually solving a problem based on the material presented.

Bjork gives five examples of training that may produce durable transfer of knowledge and skills in post-training environments. This implies long term retention and transfer of knowledge and skills to new situations.  The key is to introduce meaningful and desirable difficulties in the training.  Here are the five examples:

• Varying the conditions of practice such as the (un)predictability of the training environment, scheduling practice exercises in variety and in random fashion rather than a block of training on one specific task, etc.
• Providing contextual interference such as designing and interleaving materials to be learned, rearranging the material presentation that is inconsistent with an outline, or adding to the complexity of the tasks to be performed.
• Distributing practice on a given task over time rather than "cramming" the material in a short session.
• Reducing feedback to the learner (mainly applicable to motorized skill).
• Using tests as learning events rather than providing more study opportunities.

In Computer Science, the lab exercises and problem sets do help students in their learning, especially when the complexity and context of the problems are varied.  By mixing the type of problems to be solved, changing the duration between similar types problems to be solved, and the use of tests to continue monitor student progress, not just within a course, but over several courses, we may begin to get a clearer picture of our student learning.

Reference:

Bjork, R. (1994). Memory and metamemory considerations in the training of human beings.  In J. Metcalfe and A. Shimamura (Eds.), Metacognition: Knowing About Knowing (pp. 185-205). Cambridge, MA: MIT Press.

Demos .. learning or entertainment?

Doing live code demos in class can help in engaging students in their learning. Students get to see the effects of code changes right away rather than just hearing about the concepts through a lecture presentation. However, at least for physics, students who observe demonstrations perform only slightly better, but not statistically significant, than those who do not. Without active participation in the demonstration, students are not engaging with their learning, just like listening passively in a lecture. The only difference is that the demonstration may be a bit more entertaining and may have some affective effects on the students.

To make the most out of classroom demonstrations, one simple strategy is to simply ask the students to predict what will happen before doing the demonstration. Adding one or two minutes in having the students think about the topic under discussion, and predict what would happen if there is a change, turns out to have dramatic effects on student learning. If students are also required to discuss their predictions with their peers, their learning can be improved even more.

Reference:

Crouch, C., Fagen, A., Callan, J., Mazur, E. (June 2004). Classroom demonstration: Learning tools or entertainment?. American Association of Physics Teachers. 72(6), p 835 - 838.

What students need to know to solve Math / CS problems?

According to Richard Mayer, there are four essential stages one needs to go through to solve a typical mathematical problem like the following:

Floor tiles are sold in squares 30 centimeters on each side. How much world it cost to tile a rectangular room 7.2 meters long and 5.4 meters wide if the tiles cost $.72 each?

The four stages are: problem translation which converts each sentence into an internal representation, problem integration which involves putting the different pieces of information into a coherent whole, solution planning which involves the selection of the most appropriate strategy to solve the problem, and solution execution which involves carrying out the procedures to derive the solution.

The same four stages of problem solving can be applied to solving Computer Science problems. Problem translation involves a good data representation, usually in the form of a data structure, database design, or file structure. Once we have a good data representation, the flow of data and its transformation need to be analyzed. A good strategy usually in the form of a good algorithm needs to be selected next, and finally, the problem can be coded in a programming language for execution.

In CS, we have at least a course for each of these steps: Data Structure, Systems Analysis and Design / Software Engineering, Algorithm Design, and Programming Languages. The process of solving problems in computer science is indeed non-trivial, but yet, in many CS1 courses, we expect our students to be able to do all these while the primary focus of the course is probably just getting students to learn a programming language. Since the first course in CS usually determines whether a student will continue into the discipline, care must be taken not to overwhelm the students with too much material and frustrate them in their learning.

Reference:

Mayer, R. (2007) Learning and Instruction. Prentice Hall.

Learning Focused Course Transformation

At the United States Air Force Academy, a learning-focused transformation of Biology and Physics core courses was made to support deep student learning. This involves first transforming the learning goals from using terms like "list, find, calculate, describe, use, what, and when" to "explain, analyze, apply, create, predict, and evaluate". Students are also exposed to familiar and concrete settings where the knowledge can be applied. E.g. students are asked to serve as "expert witness" in a trial which requires their knowledge on gene expression, or they have to explain how spies can tap phone lines during the Cold War using Faraday's law. Class lessons include mini-lectures (about 10 minutes) and the rest of the time is mostly spent on learning experiences through activities and exploration. Instructors become learning facilitators rather than just lecturers.

The ultimate question is whether students learn just as much from activity based lessons as from traditional lecture style of delivery of content. In Computer Science education, students are invariably exposed to learning through activity based programming assignments since the nature of the discipline is mostly practical and applied. However, programming is not the only activities that students can be involved in, even though it is the most natural one. Especially if one of the learning goals is the development of abstract thinking, care must be taken not to over-emphasize the programming aspect that may monopolize the time and attention the students should spend. Programming projects can take up a lot of time and the students may end up gaining programming skills and not other skills. Here is where concise learning goals need to be articulated and the proportion of time for each learning goal is matched appropriately with the learning activities.

Reference:

Sagendorf, K., Noyd, R., Morris, D. (2009). The Learning-Focused Transformation of Biology and Physics Core Courses at the U.S. Air Force Academy. Journal of College Science Teaching. January / February 2009.

SIGCSE Presentations on Making CS More Relevant & Engaging

Here are a few highlights from SIGCSE 2009, Chattanooga, TN. These highlights are about making computer science "interesting" and "more relevant" to students that normally bypass traditional CS courses:

1. "Rediscovering the Passion, Beauty, Joy, and Awe"--panel presentation:

Dan Garcia (Berkeley): CHANGE (Obama style?) has come to computing. Let's avoid the old style of syntax-driven CS 0/1 curriculum. Let's let students choose their own projects, mix of CS courses, partners, etc. In elementary schools, parents help out after school; so, why can't we get [CS-type] moms & dads to help out in the computing club? "But, we want a winning basketball team, so not just any mom or dad will do." When constructing assignments and putting together lecture material, think relevance! We need to motivate students so that they will want to spend hours of their own free time on programming ... just for fun.

Eric Roberts (Stanford): CS enrollment at Stanford is "skyrocketing", wiping out previous CS1 losses post-dot-com. Programming continues to be a very important skill that needs to be emphasized and taught more effectively. "The best programmers are several orders of magnitude better than the average programmer." At Google, for example, they represent one-tenth of 1% of the applicant pool. The best programmers are 300 times as good/productive as Google's typical programmer.

Zachary Dodds (Harvey Mudd College): suggests a breadth-first approach to CS, including functional programming. Right now, "What is learned is the square root of what is taught."--implying not much is learned/remembered from a typical CS course.

2. Microsoft's exhibit on computational thinking. An excellent book, edited by Yan Xu (former MSc student at UBC) is: Transform Science: Computational Education for Scientists (CEfS), Special Edition. Microsoft Research was giving away free copies at the conference. Many authors, including former UBC STLF Beth Simon, contributed 1-2 page positions and short research papers on educational themes in computer science, focusing on the current mismatch between typical CS programs and typical science programs. For example, many authors claim that CS courses are not serving biology, chemistry, physics, earth & ocean science, biological engineering, etc., students very well. We need to develop new courses that take the highlights of numerous first, second, and third year CS courses, and condense them into digestible units for such non-CS students. Such highlights include topics in programming principles, an easy-to-learn powerful language (say Python), discrete math, algorithms, complexity, scripting, database topics, etc., should be made available to biology students using examples (taken from biology, etc.) that are relevant and interesting to the students. Forget about command-line driven programs that compute interest, etc.; instead, stick with real bioinformatics examples. Incorporate visualization techniques, problem-driven applications, modelling, simulation, etc.

3. Owen Astrachan created a new CS course at Duke University for arts students, theater students, varsity athletes, would-be lawyers, etc. Owen created a very interesting, non-programming, non-math, CS course that would appeal to non-CS/non-science types. And it did! 250 students enrolled in his experimental and engaging course. He got several guest speakers (some of whom were Duke alumni) to speak on their areas of expertise. The topics included case studies of network protocols, privacy issues, social issues, copyright issues, high-profile/controversial law cases, etc. Owen gave us take-home copies of his midterm and final exam. The exams included questions on IETF standards, Internet voting, DNS servers, security, Skype and security, worms, Flickr, BitTorrent, Richard Stallman's Free Software Foundation, copyright act and RIAA, IPv6, spam, cookies, iTunes, iPhone, P2P, etc.

Knowledge Transfer Assessment

There are four ways to test whether students have acquired knowledge transfer skills according to Mayer (2001):

1. Troubleshooting - by asking students why a system does not work.
2. Redesign - by asking students for a redesign of a system for a different purpose.
3. What-if - by asking students what would happen under other conditions.
4. Principle - by asking students the function of a component in the system, or why a component behaves the way it does.

In Computer Science, this is pretty easy to do. Here are some examples:

1. Troubleshooting - have students debug a program, or discover and debug another student's program.
2. Redesign - have students build another version of an application.
3. What-if - have students compare and contrast the use of different algorithms, database design, logic design, infrastructure, etc.
4. Principle - have students construct context diagram, use case diagrams, etc. and explain how one component functions within the entire system.

References:

Mayer, R. (2001). Multimedia Learning. New York: Cambridge University Press.

Too Much "Seductive Details" In Lectures

Bet that title caught your attention ... sex and death are inherently interesting (Kintsch, 1980). So if you want to spice up your lectures, inject some sex and death. But make sure they are relevant. Seductive details are high interest to students but if they are irrelevant to what the students are learning, they are considered as extraneous details, and they actually have a negative effect on student learning.

Learners have only a limited amount of processing capacity available to them for learning. Like a battery, if the energy is wasted on irrelevant material, there is just not enough energy left for the relevant material. What's more, high interest details take up more energy than low interest details, so if you add more "seductive" material in your presentation, you are leaving your students with less energy for the more important material you want to present.

Mayer et al's article (2008) concluded that increasing irrelevant details even though they are of high interest does not appear to affect learners in their understanding of material (as measured by their retention of material), but they do disrupt their construction of a coherent mental model of the to-be-learned system (as measured by their transfer ability performance).

References:

Kintsch, W. (1980). Learning from text, levels of comprehension, or: Why would anyone read a story anyways? Poetics, 9, 87-98.

Mayer, R., Griffith, E., Jurkowitz, I., Rothman, D., (2008). Increased Interestingness of Extraneous Details iin a Multimedia Science Presentation Leads to Decreased Learning. Journal of Experimental Psychology: Applied. 14(4), 239-339.