Pushing the Right Buttons
Teachers have long struggled to understand the learning issues of each individual student. Now technology is helping them do the job. Who knew?
By Joe Levine
It’s 6:30 a.m., and over her morning coffee, Christine O’Connor, a middle-school math teacher in Shrewsbury, Massachusetts, is reading a computer-generated email that’s telling her how each of the 26 students in her algebra class fared on last night’s homework assignment, which involved a series of one- and two-step problems. Here is what she knows with one click of the mouse: Two students didn’t even attempt the assignment, the rest of the class spent between 11 and 35 minutes on the work, and 87 percent of the class got stuck on problems 10 and 11.
That is a lot of data to digest at this early hour, but knowing where every student stands gives O’Connor a jump on her day and the edge in class, where she will devote the first 10 minutes to focusing on the two problems that gave almost everyone trouble.
“This has changed how I teach,” O’Connor says. “It saves me at least a couple of hours a week.”
“This” is ASSISTments, a web-based application developed by Neil Heffernan, a computer scientist at Worcester Polytechnic Institute and former eighth-grade math teacher. Hosted at WPI but run in middle-school and high-school students’ browsers with no installation needed, ASSISTments is neither an online curriculum nor a substitute for direct teacher instruction. Rather, it is an intensive practice tool that reinforces and builds upon what teachers already do. The system performs many valuable tasks: generating sets of practice math problems geared to a range of skills; providing cues to students when they cannot answer problems correctly; sending teachers same-day reports on students’ progress, along with aggregated reports about how the entire class is doing; furnishing additional assignments in which students either practice specific sub-skills or move to the next level of difficulty; and enabling teachers to communicate weekly or even daily with parents about how their children are doing. There’s even an open-response function, which prompts students to describe the logic they employed in tackling specific problems.
O’Connor, who works primarily with English language learners, is a big fan of ASSISTments. “I can explain to them about ‘there,’ ‘their,’ and ‘they’re,’ assess their vocabulary, test their ability to restate problems or refer to diagrams,” she says. “It puts all their skills on display.”
ASSISTments is just one of an emerging new class of teaching tools known as intelligent tutoring systems or, more broadly, adaptive education technologies (AETs). Most intelligent tutors are instructional tools. But where some are designed to be “teacher proofed”—set up so students can work alone, with the computer in charge—ASSISTments empowers teachers to conduct formative assessment, so that they can alter their instruction from day to day based on the real-time insight the technology is providing them about students’ perceptions and misperceptions.
That kind of insight typically eludes even the most knowledgeable observers, including Heffernan, who likes to recall the moment a few years ago when a house-guest—a cognitive scientist visiting from Australia—picked up his son’s fourth-grade math homework and pointed out that it reflected a consistent subtraction error.
“He said, ‘Look, he’s confused about borrowing across the zero in the tens place,’ which the literature has identified as one of the most common misconceptions about subtraction in kids that age,” Heffernan recalls. “My wife and I were a bit mortified. And then, being busy people, we compounded the problem because we forgot to flag it for the teacher.”
Intelligent tutoring systems have emerged thanks to faster web connections, better servers and improved programming languages, such as Javascript and Ajax, that let users run complicated functions within a web browser at no cost. Many of the new tools are the product of open-content authoring, a Wiki-like environment in which designers can link to other programs and libraries of problems and assignments on the web.
Many systems are open-source and offered for free, reflecting the fervent egalitarian philosophy of the community that has produced them. Marcia Linn, a professor at the University of California, Berkeley, spent 20 years researching how young people learn science and how scientific visualizations can contribute to the process. She and a colleague, Jim Slotta, eventually built the Web-based Inquiry Science Environment, or WISE. The program includes access to a library of more than 50 weeklong inquiry assignments developed by researchers and teachers around the country, spanning a range of topics such as chemical reactions, global climate change, photosynthesis, recycling and plate tectonics.
Heffernan resolved to build ASSISTments six years ago after being told he had terminal cancer, a diagnosis that, to date, happily has failed to materialize. He has tapped at least one retired teacher to help create new problem sets and hints that he can incorporate into the system, and he hopes at some point to visit retirement communities to recruit others to lend their skills and expertise.
Intelligent tutoring systems vary in content and focus, but they seem to share the decidedly Deweyan philosophy that learning best occurs when learners draw on their own experiences in order to make sense of the world.
WISE, for example, employs a “knowledge integration approach,” which is premised on the idea that, whether they realize it or not, young people enter a classroom with their own repertoire of what Linn and Slotta call “rich, confusing and intriguing ideas” about how and why phenomena occur. Right or wrong, those ideas become the starting point for inquiry because they have direct relevance to students’ lives. Students often use scientific research skills such as observation and experimentation to develop incomplete ideas such as “Objects in motion come to rest.” The goal of each WISE project, then, is to get students to articulate their initial thinking, to test it to see where it falls short of reality, to refine it—and then to repeat the cycle, again and again, in order to arrive at a progressively fuller understanding.
The act of prediction is critically important in this iterative process because it forces students to make their ideas “visible” to themselves and to their teachers. Or, as Linn told listeners at a conference on educational data-mining convened last May by TC President Susan Fuhrman, “Predictions are the ideas students have already constructed, so constructivist learning needs to start with these ideas.”
Predicting is not sufficient for learning, though. Students also need to add new, accurate ideas. Yet phenomena such as global warming and albedo (the fraction of solar energy reflected from Earth back into space) are often too complex, or occur over too long a period, for students to make sense of or visualize. WISE offers a number of nifty features that get around these limitations. Students can experiment with an interactive global warming visualization to test their conjectures. They can use the WISE Data tool to draw several different kinds of graphs using their data points and then modify them based on new results. They can use WISE Draw to, say, sketch predictions of a temperature graph, map the location where they think an invasive species might appear, or construct a model of hydrogen combustion at the molecular level. With a tool called the Idea Manager, they can generate ideas and then visually categorize pieces of data and evidence according to the viewpoint or argument they buttress. Students can use MySystem to create a concept map with arrows that show the order of connection between different causal factors. And through a threaded Discussion Forum, they can talk to one another. These discussions accelerate learning in two ways: One, those who “get it” can often appreciate or more readily relate to a misconception that a peer holds; and two, they avoid the standard classroom format in which the so-called smart kids do all the talking.
Another feature of WISE is a “Show All Work” icon that enables the student or teacher to access, at any time, a page showing all of the student’s work, providing teachers with an invaluable window onto his or her thinking. Teachers can access student work in real time, flag ideas for class discussion, assign grades and send comments.
“The detail you get is so different from the knowledge you get from just teaching from a textbook or looking at a year-end test, which doesn’t tell you where students’ knowledge and understanding broke down,” Linn says.
The benefits of intelligent tutoring systems like ASSISTments and WISE to teachers and students are obvious. At the same time, researchers like Heffernan and Linn are equally excited about these systems as research tools that could improve instruction and school management on a system-wide or even nationwide basis. The systems record every keystroke that a student makes. In many instances they can recognize and tag certain sequences of keystrokes as successful or unsuccessful attempts to implement defined skills. And they can aggregate data for groups of almost any size.
These features, Heffernan says, are “ideal for enabling researchers to conduct randomized, controlled experiments of large numbers of students.” And as the field hammers out a common format for archiving data, it will be possible for researchers who have never entered the classroom or met any students to draw conclusions based on secondary analysis of information from these studies, or to construct meta-studies that pool information from many experiments.
For example, with Ryan Baker, another WPI faculty member, Heffernan has received a National Science Foundation grant to analyze whether young students’ level of engagement with math, as measured by certain ASSISTments-generated indicators, can be used to effectively predict whether these same students will go on to pursue careers in the so-called STEM fields —science, technology, engineering and math. The project will follow students into college and the workforce (and also look at their academic records going back as far as seven years), correlating predictions with survey measures of vocational interest, self-efficacy, math attitudes and choices of college majors, as well as job placement. The hope is to ultimately establish guidelines for identifying and intervening early on with “at risk” students in these subjects, as well as to recognize gifted students and accelerate their progress.
For Andrew Burnett, a middle-school math teacher in Millbury, Massachusetts, who uses ASSISTments, it all gets back to an issue he has struggled with throughout his career.
“As a teacher, I think I’m good at a lot of things, but my weakness has always been differentiating my instruction to help all my students in the areas they need. Now, with ASSISTments, I can have students practice math questions on the same topic but with different numbers. Plus, I don’t need to grade the practice work and pinpoint areas of weakness anymore, because ASSISTments does that for me. It’s honestly made me a better teacher.”
Published Wednesday, May. 2, 2012