Gold Rush! New Technology Mines Students' Minds
TC President Susan Fuhrman has convened an effort to bolster a formalized field of educational data-mining
by Joe Levine
It’s no secret that technology has arrived in education. from middle school on up, kids live on their laptops. Classroom work is conducted on smart boards that allow the user to turn a square into a rectangle with the tap of a finger. Thanks to the Internet, an eighth grader writing a research paper can access the most sophisticated scientific findings without ever leaving her room. With toddlers using tablets and textbook apps, we can expect that more and more content will be delivered through technological tools.
Now—with less fanfare, but with perhaps even greater potential benefit—comes the next wave of the revolution.
Over the past year, in her role as President of the National Academy of Education, TC President Susan Fuhrman has convened researchers and developers from around the world to bolster a new era of research based on the mining and analysis of data from adaptive education technologies (AETs).
These powerful new tools can perform many valuable functions, including assessing student learning and engaging students in exciting learning environments. Among the better-known AETs are:
‣ the Carnegie Learning Math Series, a core and supplemental middle-school curriculum that uses problem-centered activities and games that trade on sports, art, money, the environment and other topics of interest to motivate students to think about mathematical ideas;
‣ PhET Interactive Solutions, developed at the University of Colorado at Boulder, which provides simulations of physical phenomena to aid in visual comprehension of physics concepts;
‣ the Web-based Inquiry Science Environment (WISE), developed at the University of California, Berkeley, which provides access to a library of more than 50 weeklong inquiry assignments developed by partnerships of researchers and teachers around the country;
‣ ASSISTments, a web-based application developed at Worcester Polytechnic Institute that generates sets of practice math problems geared to a range of skills. (See story about WISE and ASSISTments, page 22.)
But the potentially bigger news is that AETs also record a gold mine of as yet mostly untapped data that could redirect teaching strategies on a broad scale and improve the management of schools and school systems. AETs record every keystroke a student makes. Analysts can link students’ performance in the games or simulations offered by AETs to specific teacher interventions. And AETs also aggregate data for whole classrooms and even entire school systems.
Prior to the meetings convened by Fuhrman, there had been few concerted efforts to establish a common framework for sharing and analyzing this information. Most of the data analytics work was being conducted by computer scientists, with little input from education researchers. With a veritable Who’s Who of experts culled from universities around the world and companies such as Cisco, Pearson PLC, and Wireless Generation, Fuhrman’s gatherings in Washington, D.C., took on historical significance.
“If we knew everything about students’ learning challenges, we wouldn’t need data. But there’s so much we don’t know, and the challenges that are directly visible to us are just the tip of the iceberg,” said Kenneth Koedinger, a cognitive psychologist and computer scientist at Carnegie Mellon University and Director of the Pittsburgh Science of Learning Center.
Just as the Human Genome Project has enabled scientists to begin developing treatments that target the underlying genes and proteins that govern our biology, data-mining from AETs could help education researchers design curricula and teaching strategies that are more directly calibrated to an understanding of human cognition.
Interactions between people and adaptive technologies generate valuable data about how individuals think, decide and learn. Hence, the ability of companies such as Amazon and Facebook to suggest new books and other products that you might be inclined to buy, or of your laptop to anticipate email addresses that you frequently use.
Indeed, the past decade has seen the emergence of an entirely new field of computer science called “knowledge discovery in databases,” or KDD. The focus, whether in business, science or other areas, is on the “secondary analysis” of vast amounts of data computers record on user behavior that may not previously have been directly targeted by a specific study. Perhaps the best recent example of commercial data-mining research was the Netflix Prize competition in 2009, in which independent researchers vied to create a better algorithm for predicting viewer preferences. The competition was managed and judged by Charles Elkan, a University of California, San Diego, computer science professor who runs an annual data-mining competition of his own for students and postdoctoral researchers.
Now, versions of the Netflix competition have begun to spring up in education. “Secondary analysis is critical for making education a more scientific field than it currently is,” says Allan Collins, Professor Emeritus of Education and Social Policy at Northwestern University. “In certain fields, like children’s language, developing archives available to lots of researchers was a critical step in getting more people talking to one another. That provides a grounding that lets a field advance in a way it can’t when people are simply doing their own things.”
Hence the excitement at the two meetings convened by Fuhrman in Washington, D.C.—a gathering in May 2011 of 40-odd leading researchers and a larger meeting this past December that included a wider array of representatives from gaming companies and software developers. Colorado Senator Michael Bennett, a noted technology enthusiast who recently introduced a plan to “turbocharge education R&D,” was also in attendance.
First and foremost on the agenda of both meetings was the “Tower of Babel” problem—the current lack of common standard formats for archiving research data, which prevents investigators from accessing one another’s study data for making comparisons, conducting meta-studies, or teasing out new correlations. With the advent of data repository centers, such as DataShop, at the Pittsburgh Science of Learning Center (see story on page 28), common formats are beginning to emerge—but the bigger challenge is getting researchers to think about data-sharing up front, so that archiving is built into the process.
The groups also discussed issues of study design, primarily the relative merits of a theory-driven experiment that seeks to test a specific hypothesis versus a large mining study that simply correlates performance with one factor or another. The consensus? Both approaches are necessary, and neither alone is sufficient.
Other discussions focused on a range of topics: whether and how to draw on digital gaming, an industry that has clearly learned how to engage users, but not always with educational results in mind; how to know whether teachers and students are implementing intelligent tutoring technologies in the ways originally envisioned; and how to identify “noisy data”—patterns that turn out to be artifacts created by the technology, rather than genuine representations of significant human behavior.
There was also considerable debate over how to secure the privacy of students and grapple with the proprietary interests of commercial technology companies. For example an idea that has been championed by a team led by Gary Natriello, TC’s Ruth L. Gottesman Professor of Educational Research, is to ensure that data includes identifying information about individual students only within a student’s school, while data fed into any central repository would be stripped of identifying information.
At the conclusion of a comprehensive background paper he wrote to anchor discussion at the meetings, Natriello presented a vision of the future in which intelligent tutoring systems come into widespread use, giving rise to “a new generation of researchers who blend the skills of educational research and systems development,” and who bridge the worlds of learning theory and technology. To that, the participants offered a heartfelt amen.
Published Wednesday, May. 2, 2012
