Boost for Udacity Project

Two months after San Jose State University announced it would put its partnership with the online course provider Udacity on "pause," university officials on Wednesday touted results from the summer cohort as “significantly better” and said the project will not be disrupted by a hiatus lasting through the fall semester.

The partnership, known as SJSU Plus, was unveiled with great fanfare in January, with many politicians -- including California Governor Jerry Brown -- saying it showed the way for providing inexpensive college education for a mass audience. The project then skidded to a halt in July as the first three for-credit courses -- college algebra, elementary statistics and entry-level math -- posted pass rates between 23.8 and 50.5 percent. The university announced it would spend the fall semester evaluating the project and its results, then reopen enrollment for the 2014 spring semester.

This summer, 1,380 students completed the courses (up from fewer than 300 this spring), and the project added introductory courses in psychology and computer programming to its course list. In four of five classes, more than two-thirds of students received a passing grade. Three courses also posted a higher pass rate than their on-campus equivalents compared to an average of the last six semesters. However, the retention rate in those courses dropped.

“To all those people who declared our experiment a failure, you have to understand how innovation works,” Udacity CEO Sebastian Thrun wrote in a blog post.add link when it goes live? -sj “Few ideas work on the first try. Iteration is key to innovation. We are seeing significant improvement in learning outcomes and student engagement. And we know from our data that there is much more to be done.”

Thrun recently hinted that the summer pilot's results would be more positive, and that Udacity was getting close to finding the "magic formula" to deliver high-quality, low-cost education.

The lone holdout among the SJSU Plus courses is entry-level math, which saw the smallest increase in students who received a passing grade, from 23.8 to 29.8 percent. That places the pass rate almost 40 percentage points below the closest SJSU Plus course, and about 15 percentage points below the pass rate of the on-campus course.

Ellen Junn, provost at San Jose State, said the discrepancy can be explained by student demographics. “If you take students who are more in that remedial category to begin with, they’re more likely to not do well,” she said. In fact, a majority of students in the spring pilot were not enrolled at San Jose State.

Lillian Taiz, president of the California Faculty Association, said the sluggish pass rate among students in the remedial math course suggests the SJSU Plus project is a poor match for students in the California State University System.

"The one thing that jumped out here was that there did seem to be a sense that this worked for certain kinds of students and not for other kinds of students," Taiz, professor of history at California State University at Los Angeles, said. "It just feels like people don’t quite get that students who are struggling are often students who end up in the CSU."

While student performance is up, the retention rate dropped from 83 percent this spring to 60 percent over the summer, which Taiz said may have inflated the pass rates, as students who would have received a poor grade in a course instead decided to drop it. In comparison, data provided by SJSU showed similar on-campus classes have retained no less than 94.3 percent of students since the 2010 spring semester.

"[O]bviously it's a lot easier to walk away from an online course than a face-to-face course," Junn wrote in an e-mail.

Udacity and San Jose State officials have criticized news media coverage of the spring pilot, which they say has been too focused on the low pass rates. They point out that most students enrolled in the spring courses were disadvantaged students, and that their results are therefore not comparable to those for students who are used to the rigors of higher education.  

“We intentionally were opening up the courses for students that were more at risk,” Junn said. “With the summer, we opened it up for more students. While there were still at-risk students involved, they were not in the majority.” Of course, when the idea was introduced, it was heralded as a way to help at-risk students -- even as many faculty members warned that those students were the least likely to benefit from online instruction.

Junn acknowledged that the summer pilot’s more diverse student population may factor into the higher pass rates. Students enrolled at San Jose State and other institutions in the California State University System now make up 11 percent of the SJSU Plus population, while 71 percent enrolled in the courses from outside California. “That’s part of the pilot -- to look at different groups of students,” she said. “What we want to do is provide more access to students of more walks of life.”

Between the spring and summer sessions, Udacity and SJSU also changed “[a] good fraction” of the content, hosted orientation session and added more tools to gauge student performance. Junn said the improvements were made with existing resources.

Even though funds are being allocated to keep students engaged with their courses, Taiz said the money would have been best spent elsewhere. "When I think back to when they launched this thing in the face of massive budget cuts to higher education... this was supposed to save money and and provide more access," she said. "It hasn’t, at least from what I’ve seen, done either. I wish we could invest more time and money in students that are in face-to-face classes."

SJSU Plus will remain on hold for the fall, despite the uptick in student performance. In a written statement, Junn described it as "taking a breather this fall to set the stage for student success in the future."

“There’s no disruption,” Junn said. “The main reason is we wanted to take time to review all the data, to debrief with the faculty, to make sure that we’re following all the procedures on campus and just taking due diligence.”