The top of the annual performance review form at my university has a blank space for us to list any additional education we obtained during the previous year. I’ve never filled that space in before, but that will change in my review for 2012 because I spent part of my sabbatical last fall as a student in a massive open online course (or MOOC).
I'm an American historian by training, but ever since I left graduate school a global perspective has become increasingly important for historians of all kinds. That’s why I decided to get some free professional development in world history, courtesy of Coursera. I learned a lot of interesting and useful specific factual information from the MOOC instructor (or superprofessor, as the lingo goes) that has already helped me become a better teacher and scholar.
But I didn’t just listen to the lectures. Like any other student (since that’s what I was), I also wrote out all the assignments and helped grade papers written by my peers in class. This peer grading process differs from peer evaluation (which I use in class all the time) since students not only read each other’s work, they assign grades that the course professor never sees. Professors in the trenches tend to hold their monopoly on evaluating their students’ work dearly, since it helps them control the classroom better by reinforcing their power and expertise. On the other hand, superprofessors (and the MOOC providers that teach for them) have begun to experiment with having students grade other students out of necessity since no single instructor could ever hope to grade assignments from tens of thousands of students by him or herself.
With MOOCs in their infancy, few precedents exist for designing online peer grading arrangements for humanities courses. For this reason, I don’t intend to criticize my superprofessor’s choices here. However, I do have to describe some of the peer grading process from my class in order for my critique of peer grading in general to make sense. All students in the MOOC were supposed to write six essays between the start of the course and its end. For each assignment, we could choose one of three single-sentence questions to answer in 750 to 1,000 words. The week after we submitted those essays, we were supposed to grade the essays of five of our peers with respect to their argument, evidence and exposition, and leave comments. If you didn’t grade the essays your peers wrote, you didn’t get to see the grade you earned.
With respect to the grades I earned, I think my peers graded my essays just right. The grading scale in our MOOC went from zero to three. When I already knew a fair bit about the topic of the question that I answered or I tried very hard to write the best essay I could, I earned mostly threes from my peers. When I didn’t try very hard, I tended to get twos. While I listened to all my superprofessor’s lectures fairly closely, I never read the recommended textbook, which also undoubtedly hurt my scores.
For me at least, the primary problem with peer grading lay in the comments. While I received five comments on my first essay, for every subsequent essay I received number grades with no comments from a minimum of two peers and as many as four. In one case, I got no peer grades whatsoever. That meant that the only student who evaluated my essay was me. Every time I did get a comment, no peer ever wrote more than three sentences. And why should they? Comments were anonymous so the hardest part of the evaluative obligation lacked adequate incentive and accountability.
I read in The New York Times a few weeks ago that a study had begun to examine whether peer grades would match the grades assigned by professors and teaching assistants in one sociology MOOC. While that would prove an impressive feat if true, it would in no way validate the process of peer grading. Learning, as any humanities professor knows, comes not through the process of grades but through the process of students reading comments about why they got the grades they got. That’s how students find out how to do better next time.
To be fair, the course included a good set of instructions about how to grade a history essay linked from the course homepage. Unfortunately, there was no way for the superprofessor to force students to read those instructions, and due to the inevitable pressure to cover as much world history as possible, he never discussed how to grade in any of the class lectures. How could he? Good grading technique is difficult enough for graduate students to learn. Because of the size of the course I think I can safely assume that many of my fellow MOOC students inevitably had no history background at all, yet the peer grading structure forced them to evaluate whether other students were actually doing history right.
The implicit assumption of any peer grading arrangement is that students with minimal direction can do what humanities professors get paid to do and I think that’s the fatal flaw of these arrangements. This assumption not only undermines the authority of professors everywhere; it suggests that the only important part of college instruction is the content that professors transmit to their students. How many of the books you read in college can you even name, let alone describe? It’s the skills you learn in college that matter, not the specific details in any particular class, particularly those outside the major.
Over the course of my career, I have increasingly begun to spend much more time in class teaching skills than I do content. Some of this has been a reaction to encountering students who do not seem as prepared for reading or writing college-level material as the students I had back when I started teaching. However, I have also come to believe that teaching these skills is much more important than teaching any particular historical fact. After all, it really is possible to Google nearly anything these days.
Certainly good students can do a good job grading peer essays and I got a few short but insightful comments on the papers I wrote for my MOOC. Even if all of my comments had been less than helpful, I didn’t come into the MOOC process seeking to improve my writing skills. I wanted to learn new information, and many other students who engaged the material the same way that I did probably felt the same way.
Students like me won’t be the ones who’ll suffer because of peer grading. Its victims will be the future students who take MOOCs to earn college credit at increasingly cash-strapped universities. Who will teach them how to write well? Who will monitor their progress through the peer grading assignments? Who will help them understand that history is as much about argument as it is about facts or that literature can be appreciated on multiple levels? While other students can certainly teach other students some things, they can never teach students everything that a living breathing professor can.
Education startups like Coursera are experimenting with peer grading not because it is the best way for students to learn history or English, but because it is the only way that the MOOC machine can ever run itself in a humanities course. If MOOCs incurred high labor costs the same way that colleges do, those startups would never be able to extract a profit from those classes. While that’s a legitimate concern for Coursera’s venture capital investors, everyone else in academia – even the superprofessors – should give more weight to purely educational concerns.
“The fruit ripens slowly,” the Guru Nisargadatta Maharaj once observed, “but it drops suddenly.”
In a similar fashion, MOOCs (or massive open online courses) seem to have arrived almost out of nowhere, in quick succession – first Udacity in February of last year, followed by Coursera in April, then edX in May. Remarkable as it may seem, MOOCs as we know them today have been with us only for as long as it has taken the Earth to make one orbit around the sun.
“I like to call the last year ‘the decade of online learning,’ ” joked Anant Agarwal, president of edX, during my recent visit to the offices of his bustling startup in the Kendall Square area of Cambridge, Mass.
As accelerated as the progression of MOOCs has been from curious acronym to household name, and as much as it may seem that MOOCs themselves have fallen from the sky, in truth MOOCs have been ripening for some time.
Consider the free “courses” delivered through iTunes U for the last several years, or TED Talks, and Khan Academy, not to mention some of the early progenitors of MOOCs themselves, including Dave Cormier, credited with coining the phrase in 2008, as well as George Siemens, Stephen Downes, Alec Couros, David Wiley, and others.
Recall Carnegie Mellon’s Open Learning Initiative, the “open educational resources” movement, and MIT’s OpenCourseware, launched all the way back in 2002. And let’s not forget Fathom.com, an initiative out of Columbia University launched at the turn of the millennium, or even the early days of America Online and Compuserve, both of which offered educational content through their services as early as the 1990s.
MOOCs, then, are not as new as they seem – though the world today appears to be more ready for them than it was in decades past. Indeed, it isn’t hard to see how forces as diverse as Clayton Christensen’s theory of “disruptive innovation” from the late 1990s, the expansion of online enrollments over the last decade, the reformist intentions of the Spellings Commission on the Future of Higher Education from 2005-2006, the great recession of 2007-2009, or the completion agenda supported by the Lumina and Gates Foundations over the last few years have all contributed to a public thirst for what look like very high-quality educational offerings at very low – or even zero – cost.
“I also call the last year,” Agarwal added, “ ‘the decade of innovation.’ ”
And like many innovations before them, MOOCs have been received with the usual contradictory apocalyptic fervor – where some believers foresee the arrival an educational golden age and others see the eventual destruction of our institutions, our faculty, and the intangible value of face-to-face learning.
Writing in The American Interest this month, for example, Nathan Harden claimed that “ten years from now Harvard will enroll ten million students." He went on to argue that as a result of the MOOC movement, “the changes ahead will ultimately bring about the most beneficial, most efficient and most equitable access to education that the world has ever seen."
At the other end of the apocalyptic continuum, Gregory Ferenstein, writing for TechCrunch last month, foresaw a future in which MOOCs wreaked a terrible devastation on the land, as “part-time faculty get laid off, more community colleges are shuttered, extracurricular college services are closed, and humanities and arts departments are dissolved for lack of enrollment.”
The real significance of MOOCs lies, however, not in their being a harbinger of our educational salvation or demolition. Nor does their real significance lie principally in their potential to increase access or reduce costs – at least not for Agarwal and edX.
“We are about two things,” Agarwal told me. “We are about dramatically increasing quality and impacting campus learning. We are being very deliberate. This is not a numbers game – this is not a game at all. This is a quality quest.”
Funded with $60 million in seed capital from MIT and Harvard, edX can make a claim to being the first MOOC platform to market, inasmuch as its predecessor, MITx, was launched in December 2011. Until this week, the edX consortium featured five independent member institutions (MIT, Harvard, the University of California at Berkeley, Georgetown University, and Wellesley College) and one state university system comprising 15 colleges and universities (the University of Texas System). Thursday, it added six more, including several outside the United States.
In less than a year, edX’s 25 courses have enrolled close to 700,000 people. “That’s more than the combined alumni of MIT and Harvard over their combined 500-year history,” Agarwal observed with a mixture of pride, enthusiasm and amazement. What really pleases him, though, is something else.
Rolling his chair across the office, Agarwal waves me over to his monitor and shows me the virtual laboratories edX has been developing for its courses. We start with his own course on Circuits and Electronics (6.002x in the edX course catalog).
“Many MOOCs are just about analyzing problems,” he said. “We give you a blank sheet of paper and say, ‘Go build, design, create, construct something.’ ” With drag-and-drop alacrity, Agarwal moves the components of a circuit into place on a piece of digital graph paper and clicks a button to test its performance. “Computers do the grading,” he said, “in real time.”
“The media focus on numbers, they focus on cost,” Agarwal sighed. “But they should focus on something else – quality. And they should focus on efficiency. What is efficiency? It’s a ratio of quality and cost.”
Agarwal knows that MOOCs have their doubters, and he believes that they can only be persuaded with proof. He cites the case of San Jose State University, which licensed his own course on circuits and ran it as an adjunct to the school’s own classroom-based instruction. The results, Agarwal claims, were impressive. “The fail rate dropped from 40 percent to 9 percent,” he told me. “That’s a quality improvement.” And the costs to San Jose State were minimal. That’s efficiency. Agarwal says San Jose will be sharing more details about their experience with edX in the near future.
With the avidity of the prototypical startup entrepreneur, Agarwal talked excitedly about the potential for MOOCs to improve pedagogy. “We have our xConsortium,” he said. “All of the schools in our consortium have access to all the data in the platform in an anonymized format. This is what I call ‘the particle accelerator of learning’ – big data in learning in real-time.” In a sense, then, edX’s quality quest, as Agarwal calls it, is seeking out the educational equivalent of the Higgs Boson, as well the other fundamental elements of learning, in order to better understand what kind of learning objects, what kind of real-time remediation, and what kind of learning materials – whether analysis or laboratory or other – produce the best results from one learning context to the next.
I ask Agarwal what distinguishes edX from its fellow MOOC platforms. “We have a fundamentally different mission,” he replied. “We’re nonprofit. We’re open source. Our technology is for everyone. And we have a commitment to campus learning.”
Earlier this month, the American Council on Education completed an evaluation of five courses on the Coursera platform, developed respectively by Duke University, the University of California at Irvine, and the University of Pennsylvania. Intriguingly, all five courses were approved for credit through the ACE credit transfer program. But just in case the future of MOOCs was beginning to make sense to you, consider this – all three of these institutions have made it clear that they, at least, will not be awarding credits for the courses, irrespective of the fact that they developed the courses themselves.
MOOCs are puzzling.
Will they last? It’s not, I suspect, a question that would bother Agarwal very much one way or the other. “For us,” he said, “it’s not about MOOCs. We are trying to reimagine our own campus. The lecture wasn’t working. Quality has been static for decades, but costs are going up. There’s a trillion dollars in student debt. We are trying reimagine campus education from the ground up – with new ways of learning that are more enriching, more engaging, more efficient, and that produce better outcomes.”
How do you like them apples?
Peter Stokes is executive director of postsecondary innovation in the College of Professional Studies at Northeastern University, and author of the Peripheral Vision column.
Submitted by Ted Fiske on February 12, 2013 - 3:00am
Gimme an M! M!
Gimme an O! O!
Gimme another O! O!
Gimme a C! C!
What have we got? MOOC!
Far above Cayuga’s waters with its waves of blue,
Stand our noble M-O-O-Cs, glorious to view.
Massive Open Online Courses, loud their praises tell.
Hail O dig’tal Alma Mater, now called e-Cornell.
On, Wisconsin! On, Wisconsin!
Fight on for our MOOCs.
They make teachers into rock stars.
Who needs Yale or Duke? (rah rah rah)
We take classes in our jammies
Any time of day.
Oh, how we love to learn
The online way.
Cheer, cheer for old Notre Dame.
And for the MOOCs that bring us our fame.
Send a volley cheer on high
’Cause our instruction comes from the sky.
Though the attachments be great or small,
Our CPUs can handle them all.
Open access sets us free
To seek out an e-degree.
Don’t need classrooms, that’s for sure.
Libraries are so passé --
Remnants of another day.
We’re creating new tradition.
Ours is wireless erudition.
We eschew all printed words.
Rest in pace Gutenberg.
Edward (Ted) Fiske, former education editor of The New York Times, is author of The Fiske Guide to Colleges. Post your ideas for other college songs for the MOOC era here as comments or e-mail them here.
A bunch of educators, several of whom I know and respect quite a bit, got together last month to write a "bill of rights" for online learners. Viewable and editable here.
They included the rights to access, privacy, openness, to create public knowledge, to "pedagogical transparency" (to understand the ways you are being taught and the value of any credentials offered), "financial transparency" (Where is my tuition money going? How will this “free course” be paid for?), to have great teachers, and to become teachers.
I can’t find myself disagreeing with anything much that they had to say, except for one screaming contradiction that brings the whole thing down.
"All too often, during such wrenching transitions, the voice of the learner gets muffled," this group wrote in their introduction.
The problem is, this group didn't include any learners. Of the 12 signatories, I count 8 Ph.D.s or Ph.D. equivalents. They didn’t reach out to any learners on public forums. They didn’t ask any learners what they wanted to put in the document. The voice of learners is absolutely silent.
Sure, we’re all lifelong and informal learners in some sense, but let’s draw a real distinction here. Let’s talk about people who don’t have a bachelor’s degree and need one or the equivalent to make a decent living and participate in society on an equal footing. I’m not asking why the group didn’t poll Udacity users in Pakistan or Colombia, or YouMedia high school students in Chicago, or middle schoolers around the globe making their way through Khan Academy math videos, and find out exactly what their concerns are and how they would prefer to have them represented in such a document. Although really, it wouldn’t have taken much time or many resources to do this kind of research. I’m asking why they wrote a “learners’ bill of rights” without including one actual learner in their little group of 12.
I’m not going to be tendentious and draw parallels with other bills of rights. I’m not going to ask about the advisability of men writing a feminist Bill of Rights on behalf of the women they care about so deeply. Or of the North writing a bill of rights for Southerners after the Civil War. Or of employers writing a bill of rights for their employees.
Suffice it to say that educators are in a historical position of no small authority over learners. And when one group of people with authority over another makes up the rights for the second group, they tend to get some things wrong.
The fact is, this isn’t a bill of rights for learners at all. It’s a set of principles to support the interests of a group of educators, who share concern for learners, blended with concern for their own group. They tip their hand in the eighth principle, “The right to have great teachers.”
“Students should expect -- indeed demand -- that the people arranging, mentoring and facilitating their learning online be financially, intellectually and pedagogically valued and supported by institutions of higher learning and by society. Teachers’ know-how and working conditions are students’ learning conditions.”
I am in favor of all who work with learners being fairly paid, and I am definitely in favor of great teachers. But I am not in favor of students being drafted onto the metaphorical or actual picket lines. Students in state four-year institutions are paying more and more of the salaries of their instructors and going into sometimes-extreme debt to do it. There’s an uncomfortable moment where the interests of the learners actually diverge from the interests of the career academics, and it should be discussed openly.
But enough. The authors intended this to be a living document, and I respect that there’s time to revise and collect comments from the hundreds of thousands of online learners out there. It’s not going to be that difficult.
When I first found out about this bill of rights, I posted it to OpenStudy, the online learning community. I got this response from an undergraduate computer science major within 45 minutes, which reads in part:
“you deserve education BASED ON WHAT YOU WANT TO DO IN LIFE..
Teach kids real world problems, and have them enjoy it…
Teachers/professors who care. In my time I have met a lot of wonderful professors, mentors, teachers, coaches, and a ton of HORRIBLE ones…
The job market sucks, and with students being taught the same thing, and not really learning what they wish it's hard to distinguish someone from the rest of the pack. If we want to succeed we need to produce students who enjoy learning, and have the tools to learn what THEY WANT TO LEARN."
Another wrote: "The rights I want in the ever-growing digital era are not anything different than what I would want outside of it. We have to expand these rights to be applicable into the digital world."
That’s a good start. Now there’s time to come up with a set of amendments -- a real learners’ bill of rights.