CliffsNotes for Credit

At the crack of dawn this past May 15, an e-mail hit my inbox from a friend who knows my abiding interest in getting undergraduate education right: “Take a look at this.”

“This” was a link to an article in University Business from the day before: “JumpCourse announces 13 [online] courses recommended for credit by ACE.” Curious about the courses approved by the American Council on Education, I took a look at the article and then the JumpCourse website.

I am a pragmatist with regard to how to make higher education in America more successful at providing students a truly 21st-century education. We should be doing what works, and to be sure, a lot of what is done in traditional on-campus undergraduate education doesn’t work. But in my view that is because way too often we are not doing what we know and what the evidence shows does work: demanding, engaging forms of pedagogy focused both on disciplinary and broader liberal learning goals. If JumpCourse is better than a large portion of standard practice, amen to that.

One of the 13 courses newly approved by ACE is Introductory Sociology -- in my field. So I decided I would take the course and share a report.

The opening paragraph of the ACE CREDIT description on the JumpCourse website says: "JumpCourse believes in greater access to higher quality education. By expanding educational opportunities, we are adapting to the changing needs of college students. We are proud to announce that the American Council on Education's College Credit Recommendation Service (ACE CREDIT®) evaluated and recommends college credit for courses developed by JumpCourse. It is our mission to help students achieve their academic goals affordably and effectively, paving a road to graduation that will allow students to begin planning for their future." (Emphasis mine. This and other examples from the course materials may be found here.)

Note the claim that JumpCourse is providing access to higher quality education, though it doesn’t say higher quality than what. And in a Q&A section on the website -- JumpCourse? How is it made? -- it says: "While we can't tell you all of our little secrets, we can say that each JumpCourse is created by instructional designers, writers, video producers, professional storytellers, subject matter experts and otherwise passionate and talented individuals who want to help expand access and affordability of college education." (Emphasis mine.)

I have a Ph.D. in sociology and taught Introductory Sociology while on the faculty of Carleton College early in my career before spending 23 years as the president of two liberal arts colleges. I have been powerfully influenced by the Association of American Colleges & Universities' now decade-long Liberal Education and America’s Promise (LEAP) initiative and serve as chair of the LEAP Presidents’ Trust. Through LEAP, AAC&U has led a national conversation about what inclusive quality in undergraduate education needs to mean in today’s world -- a conversation that has produced an emerging consensus: in addition to knowledge and competence in specific fields of learning, the education students need for the 21st century must stress higher order learning goals of inquiry and analysis, critical and creative thinking, integrative and reflective thinking, written and oral communication, quantitative literacy, information literacy, intercultural understanding, and teamwork and real-world problem solving.

And this focus on learning goals above and beyond disciplinary content must start right at the beginning of college in challenging introductory courses that serve both as entry points into the disciplines and as part of what we awkwardly call general education.

How does what I encountered in my JumpCourse fit in to all of this?

Let me begin by summarizing briefly how the Introductory Sociology course I took and the associated online format are organized. You purchase the course for $149 -- $99 if you are not going to take the proctored online final exam -- and are then given access to it. There is an opening lesson, accompanied at the end by a short quiz that teaches you enough about how the software works so that you can proceed.

Then you begin the first of, in the case of Introductory Sociology, 21 units. Each unit is divided into eight to 15 lessons. For each lesson you can watch a video and/or read a short -- almost always just a page long -- text. These texts are called “lecture notes,” but they are more like CliffsNotes or what you might find in a bad textbook. The videos are just a person speaking essentially the exact same words you would read if you chose to read the text segment (I verified this by listening to and reading a number of these) along with some catchy visuals.

This is how the website describes the way the course is organized: "Our courses teach you through professionally produced videos, lecture notes in the form of an ebook and interactive quizzes. We also supply course coaches to monitor and encourage you through the course. You get to choose how you learn best: watch, read or practice. Each course is adaptive and molds to the way you learn."

The implication is that watching, reading and practicing -- and here they mean answering practice questions -- represent the varieties of ways people learn. But aren’t engage, write, debate, analyze, critique, research, encounter, participate and other activities also ways of learning that might be best for a given student?

If you need help you can contact an “instructor.” I didn’t contact the instructor, but here is one of three e-mails I received from mine: "June 25, 2015: Hi Daniel, This is your JumpCourse coach. I wanted to take the time and congratulate you on getting started on your Sociology JumpCourse class. Keep up the work and remember if there is anything you get stuck on or need some help with, please don’t hesitate to contact me. That’s what I’m here for! If you keep working hard you will be done with the class before you know it. (Name and telephone number.)"

I assume these periodically sent notes were automated communications, though it might very well be the case that a real person would answer at the phone number provided or respond to an email query seeking help and/or support.

You can also post emails asking questions or seeking advice from other students simultaneously enrolled in the course. Here is the total of what was up on the interstudent email site the day I checked:

• Jan. 31 at 4:17 p.m.: Intro to Sociology online final exam. Has anyone taken the online final exam for intro to sociology?
• July 7 at 4:47 p.m.: No, just started studying today.

After you read the short text, you move to a practice test to assess your understanding and short-term memory. The texts very briefly introduce, define and explain terms and concepts and associate them with the sociologists and others who invented them. At the bottom there are always a few references in case you want to read more, but JumpCourse does not actually link to the additional references, so you would have to look them up somewhere else, and I doubt many JumpCourse students do (I didn’t).

The practice tests are multiple choice, fill-in the blank or matching questions. If you answer a question correctly you move to the next. If you answered at least 90 percent of the questions on the practice test correctly, you move on to the next text. If you do not answer a question correctly, later on the practice test may ask the same question again, or ask you another version of the question to see if you get it right the second time. It will also return to the question later in the unit, circling back to see if you can get it right after a bit of time separation from the topic. This is the only way in which I can imagine they mean that “each course is adaptive and molds to the way you learn.”

At the end of the unit there is a unit test, and if you get 90 percent or higher on that, JumpCourse unlocks the next unit and you can proceed.

I spent roughly an hour or a little more on each of the 21 units. Since I am a sociologist, I remain familiar with the material and could read the text and move quickly to the practice test to demonstrate comprehension, and I have good short-term memory skills. If I had listened to the videos instead of reading the text I think my time investment would have doubled, since I can read much faster than the person on the video spoke.

I believe a truly introductory student might very well struggle with the practice tests and take much longer to move forward, but I don’t think student struggling is evidence that the course is demanding in the sense in which I mean it. I got a question wrong in about half the lessons -- almost always because definitions were unclear or contradictory, or distinctions were made that didn’t seem sensible, at least to me. I think many introductory students will be tripped up by the sloppiness of many of the practice questions.

For example, Lenski’s schema for defining societies at increasing levels of economic and social complexity is presented. His classification of societies in order of increasing complexity, they report, is: hunting and gathering, pastoral, horticultural, agrarian, industrial, and postindustrial. In one place pastoral societies are said to “grow food,” while in another it is said that they have a “more steady food supply” but it is not stated that they actually grow food. In another place pastoral societies are said to be “nomadic” -- a key point -- while elsewhere they are said to “look after livestock.” The final exam, of course, expected precision when it came to those distinctions.

I tried to pretend I was an introductory student as I did this. I know that introducing students to a new field does absolutely require defining terms, giving examples to clarify meaning and explaining some of the history of the ideas in the field. You can’t get to the big stuff if students don’t know what you mean when you use terms like “social system, status, role, group, organization, etc.” But in this course, presentation of terms and concepts is all there is. In fact, I wasn’t challenged at all to think through any complex problems or to use any quantitative reasoning skills.

For example, in the section on demography a projection of what the age distribution by gender of the Chinese population will be in 2030 was presented. We could have been asked to think about it and propose some interpretations of what it meant and how it got that way. But all they asked me to do was complete some matching questions where the answer was in the text.

I loved teaching introductory sociology. Last time I taught it was spring of 1979. I never used a textbook. I wanted students to read the very best, most well-written original books and articles I could find so that they could become inspired by what they read -- excited by the insights sociology could provide regarding the big questions the field was invented to address -- not just introduced to concepts, terms and people. For example, I would ask students to read major sections of a small book -- Evolution and Culture -- by Sahlins and Service to give them a sense not just of the stages of cultural evolution but of the deep insights one can gain by thinking of social history that way.

I was the textbook, but the goal was always to help students achieve insights into big questions once they had developed enough of a sociological vocabulary. Where does inequality come from? What are the consequences for peoples’ lives of their socioeconomic and other social statuses? What are the stages of cultural evolution and the present-day consequences of the fact that just about all of the stages of cultural evolution still exist in real societies around the world? What varieties of social systems exist and how did the differences come about? How do a social system’s subsystems -- polity, economy, community, family, etc. -- affect each other? How do we know any of this?

There were learning goals beyond disciplinary content. To be sure, in my JumpCourse there were one or two sentences in each lesson’s text that addressed things like this, but the coverage was superficial.

And with regard to quantitative reasoning -- a critical 21st-century learning goal -- even in the late 1970s at Carleton there was enough of a computer system to allow me to have my introductory students actually do some quantitative analysis of real data. They did secondary analysis using data from some of the great and pioneering sociologists’ research, which one could obtain from Columbia University’s Bureau of Applied Social Research or the University of Michigan’s Inter-University Consortium of Political and Social Research (ICPSR) and then wrote it up so that they could begin to learn how at least one tribe of sociologists engaged in the search for understanding.

In my course all tests were essay exams, and there were additional writing assignments. Only perhaps once in each practice test in this JumpCourse was I even asked to fill in the blank. I was never asked to write anything -- even if only to regurgitate something.

Some readers of this may say, “Of course you could do these things at a college where maybe 12 students sit around a table,” but back then we didn’t limit the number of students who could take a class. We thought it was the student's decision to choose a small class over a larger one, so my introductory classes ranged in size from 35 to 90.

The content in the Introduction to Sociology JumpCourse, as far as it goes, may be like many of the textbooks I refused to use in my own teaching -- so I don’t think, at the most basic level, that this course constitutes any kind of content malpractice. But it sets the bar way too low. If this course is even a bit typical of the online courses now being certified for credit, we should be asking many more serious questions about the quality of these new providers’ products.

In the course I took there were no expectations of students beyond taking the very simple-minded practice and end-of-unit tests and, if you wanted credit for the course, taking the proctored final exam -- 60 multiple-choice questions to be completed in one and a half hours with a 70 percent, or 42 correct answers, required to pass (I took the final and did miss three questions).

No writing or presenting of any kind, no interaction with an instructor beyond being able to ask questions electronically, no interaction with other students taking the course, no expectation of any kind of higher order thinking, analysis, or grappling with big questions, no inspiring students to want to learn more by showing them the deep and powerful insights into our social world that sociology can provide. I can’t imagine any student being inspired by this course to want to know more about sociology, and I do not believe that any skills beyond improving short-term memory will be developed through taking it.

So what to make of this? I believe my JumpCourse failed on its own terms -- providing disciplinary content employers say they want and need and that students who can least afford higher education will be able to convert into improved life chances and success -- and it failed to even come close to addressing the aims and objectives that employers need higher education institutions to reach.

We have lots of evidence, of course, that standard practice far too often fails on these grounds as well. But standard practice fails in my view when it does not hold to what we already know works to enable students to achieve the learning they need for the 21st century, and when it fails to include serious assessment up to the task of discovering what students actually know and can do relative to the learning goals of liberal education to create a continuous quality improvement feedback loop. When standard practice is a challenging, high-student-engagement process focused both on disciplinary and higher order learning goals, it is also cost-effective because four-year graduation rates go up. Despite their low cost to consumers, disruptive innovations like JumpCourse will inspire no one to learn what they must in a timely way.

One thing almost always missing from debates about “disruptive innovations” like JumpCourse and their comparison to “traditional” higher education is agreement about what the aims and objectives of higher education should be. It is impossible to assess the relative effectiveness and efficiency of an innovation in comparison to standard practice, or whether even the best examples of standard practice cost too much, if the goals being pursued are radically different. If it is job training or minimal fluency with the terms and concepts of disciplines that America wants, then maybe innovations like JumpCourse can make it above the bar someday. But if we want graduates of our colleges to be able to think, analyze, integrate, write and communicate, JumpCourses will never achieve that.

Finally, in medicine it is malpractice to replace standard practice with experimental remedies and procedures until they have proven to be superior and have side effects that are no worse, and it is unethical to experiment on humans without their informed consent. Yet disruptive higher education innovators lobby constantly for the freedom to do both, and like some of our nation’s worst ethical lapses in human experimentation, the subjects are and will be low-income disadvantaged people misled to believe they will be receiving treatment that is superior to standard practice and will cause them no harm.

Let’s conduct the absolutely necessary continuous experimentation to find new ways to improve student learning not by freeing “providers” up to do whatever they want funded with new government subsidies, but within the same kind of framework in place to test new drugs or other medical treatments for efficacy and safety.