by Barbara Finkelstein

It should come as no surprise that Ryan Baker has turned his own work habits into a data set, including an itemized record of the 7,000-plus minutes he invested in constructing TC’s first Massive Open Online Course (MOOC).  

The focus of the course, like that of a proposed new TC master’s degree program also created by Baker and TC faculty colleagues, is on the exploding field of educational data mining (EDM), which uses advanced computer technologies to sift through huge amounts of data generated by intelligent tutoring systems and other online learning environments for information on how learners behave.  As Baker detailed in his MOOC, data mining – already a staple of the medical, financial services and retail worlds – can reveal why an individual student is getting the wrong answers to a subtraction problem; guide a teacher on how to make the best use of classroom time by pinpointing which homework problem stumped the most students the night before; and tell a superintendent which science curriculum is proving most effective with students across the district.    

These are issues that Baker, Associate Professor of Cognitive Studies, has won international recognition for probing.  At 36, he has published more than 150 peer-reviewed papers on the use of EDM that range from assessing boredom and cheating among students who use online tutoring systems to the mining of educational data to better understand metacognition, motivation and self-regulated learning. His ultimate focus is on creating computer-based environments in which users learn because they are genuinely engaged in their work. 

“Students make hundreds of meaningful actions per hour, from pausing and thinking after getting feedback on their answer, to running away from a skeleton in a game, to changing a setting in a simulation,” he says. Using that information, “we can answer more interesting questions than we could answer before. For example, we can ask what choices are associated with positive outcomes for students?”

Baker’s interest in student engagement dates back to his graduate student days at Carnegie Mellon University when, armed with a clipboard and a pen, he spent hundreds of hours in suburban Pennsylvania schools trying to identify key moments when students went “off task” and why.   

“After six weeks of this, I said, ‘Agh!’” Baker says. “There has to be a better way.”

Sensors and webcams, which were typically used to capture moment-to-moment student behavior, proved to be intrusive and financially unsustainable.  So Baker instead developed software to analyze student clicks and keystrokes in online math lessons.  By analyzing data housed in the computer log, Baker and his group could even determine when students had resorted to guesswork. The test was the first-ever automated detector of disengaged learning behavior based on moment-to-moment data.

At Worcester Polytechnic Institute, where he taught before coming to TC, Baker studied intelligent tutoring systems such as ASSISTments, a free Web-based program designed by a fellow faculty member, Neil Heffernan, which generates sets of practice math problems and provides students with immediate cues when they cannot answer problems correctly. Funded by the National Science Foundation, Heffernan and Baker are now conducting an eight-year tracking study to see how the choices they making in using ASSISTments affect students’ long-term outcomes, such as whether they go to college.

Most recently, Baker has turned his attention to MOOCs – online courses consisting of educational modules, benchmarks and homework assignments. Since their adoption in 2012 by universities such as Harvard, Stanford and MIT, MOOCs have become an increasingly popular feature on the education landscape around the world.

Because of their potential as a low-cost educator of millions of people –Baker’s course at TC alone enrolled more than 44,000-plus students, which is considered merely mid-sized – MOOCs have been hailed as a “particle accelerator” for learning.  On the opposite of the debate, a recent article in The Chronicle of Higher Education quoted one technology expert describing MOOCs as “a sideshow,” and reported that “the dream of a MOOC U. fades with each empirical study showing their ineffectiveness.”

For all the rhetoric, though, serious research is sparse in two important areas: the pedagogical methods and technologies most responsible for successful MOOC learning, and the challenges of keeping a large number of registrants committed from start to finish. (Course completion rates hover between just 5 and 13 percent.)

Baker intends to fill those gaps by mining data from his own MOOC. He will address questions that MOOC instructors everywhere are asking themselves: What is the goal of MOOC education? What draws such large numbers of potential students to a MOOC? How effective are MOOC feedback systems and student forums? Most important, what impact do MOOCs have on students’ futures, and does it even matter whether a course registrant stays on to receive a “statement of accomplishment?”

While he has only just begun to look at the data on his own MOOC, Baker is ready to share some anecdotal evidence about the efficacy of MOOC instruction.  Among his observations:

• MOOCs are an excellent vehicle for introductory material. Many introductory courses are given by teaching assistants relatively new to teaching. By contrast, MOOC lecturers have to be subject matter experts who carefully hone their lectures and continuously work to improve them.
• MOOCs fill an educational void when subjects are locally unavailable. Programs in educational data mining, for example, existed in about five cities worldwide before Baker made his course available to anybody with a computer and Internet connection.
• MOOCs can be a vessel to interest students in more formal education programs, such as TC’s Masters in Cognitive Studies in Education (Focus in Learning Analytics) in the Department of Human Development.

The big question, of course, concerns how MOOC instruction compares with in-classroom learning. Baker takes a measured approach. He thinks MOOCs in their current state will not dominate education partly because of enrollment fall-off rates, but also because MOOCs have been made into an educational hybrid. The lectures, Baker says, are “mostly pretty good,” while homework assignments are not yet as effective.  Baker cautions against making the MOOC all things to all people.

Published Tuesday, Jan. 28, 2014