You have /5 articles left.
Sign up for a free account or log in.
Davidson College’s first MOOC, a course on medicinal chemistry, wrapped up last month on the edX platform. Producing and delivering the MOOC required significant effort on the part of several Davidson staff and faculty members. With such an investment comes the natural question… Was it a success?
Evaluating success in education of any type is difficult, and MOOCs are no different. The mountains of (often messy) data from online student activity create assessment challenges for MOOCs. Details such as page views, time of engagement with videos, and discussion board interactions are all available. In fact, each time a student clicks within the platform that interaction is recorded. Despite the opportunities posed by ample data sets, MOOC assessments are often disappointingly reduced to reporting how many students finish a course.
Even the idea of “finishing” a MOOC is not a simple concept. How does one define finished―students who passed the course, took the last test, or viewed content during the end of course? In comparing the outcomes of our course to other schools’ MOOCs, we found our definition of finishing had to adapt to the definitions used by other institutions.
Harvard and MIT recently disclosed data on all their edX courses that had been offered from fall 2012 through summer 2013. Harvard and MIT had a collective MOOC enrollment of 841,687 students. Attrition was reportedly highest in the first week at 50 percent but then slowed to 16 percent in the second week. A total of 29,551 (5.1 percent) students merited certificates of achievement at the end of the course. For our first edX course (14,183 students), attrition was higher in the first two weeks at 61 and 32 percent, respectively. The percentage of students receiving a certificate, however, also was slightly higher at 5.8.
Another institution that has been active in the MOOC area is Duke University. Duke’s spring alumni magazine reported enrollment and outcome data for all of Duke’s 2013 MOOCs on the Coursera platform. Of the 876,354 enrolled students, 12.4 percent attempted to answer the first homework set of the course. Approximately 3.4 percent attempted the last assessment task in the course. In our course, 18.5 percent of all students attempted the first homework set, and 4.3 percent worked the last task. Fortunately, the description of Duke’s criteria was very clear and could be matched directly to accessible data for our course.The data in the table are not easily compared. The Harvard/MIT study bases percentages upon “student activity”, which is never explicitly defined. The activity is presumably related to video or page views in the various courses. For the Davidson data, attrition estimates are based upon student views of homework problems at the beginning of each week. Class-wide homework viewing information is readily available from within the edX system.
Based off of various categories of completion, I consider our course a success. Relative to MOOC offerings at Harvard, MIT and Duke, our course fared well in two metrics for course completion―both the percentage of student certification and percentage of students who attempt the final exercise.
Are these two metrics the best measures of course success? Probably not. Assessment should factor in student expectations and course goals, as well as involve rich student activity data. Ideally, institutions offering MOOCs should work with MOOC platforms toward the creation of measures for reporting course and student outcomes. Such endeavors would certainly facilitate the evaluation and comparison of MOOCs while perhaps also lending credibility to MOOCs as an emerging means of education. In other words, some standards in the Wild West world of MOOCs would benefit both the offering institutions as well as education consumers.
Although best practices for assessing MOOCs are still in their infancy, I am very comfortable with celebrating Davidson’s first MOOC as a success. Maybe someday I will be able to support my opinion with generally accepted data.
Professor Erland Stevens teaches organic and medicinal chemistry at Davidson College. His research interests include nitrogen heterocycles, medicinal chemistry, and bioorganic chemistry.
