Online Quality Control

As colleges and universities across the country move to start or expand online education, professors at Oregon State University worry their university isn’t doing enough to control quality at its longstanding and fast-growing online program.

Administrators and faculty themselves do not have a firm understanding of how well online students are doing and may rely too heavily on adjuncts and graduate students to provide online instruction, according to some faculty representatives.

There are also faculty complaints that the university pulled a bait-and-switch on professors when it changed how it paid online course instructors.

While the concerns at Oregon State may be notable in part because the public university is trying to get its arms around the issues, the concerns in one place with a history of online education raise questions about how well colleges across the country are controlling quality amid a race to put courses online. Nationally, about a third of college students take at least one online class and colleges are increasingly looking to expand online ventures.

The Oregon State discussions have also revealed that the university may lack enough information to evaluate what goes on in its courses -- in person or online.

Oregon State, which is accredited by the Northwest Commission on Colleges and Universities, began its online program in the late 1990s and had an enrollment growth spurt in the past several years. About 7,000 students take classes through the program -- a fourth of all students at the university.

Roger Nielson, a geosciences professor who chairs the Faculty Senate’s online education committee, said he hears widespread concerns from faculty members who want the program to be subjected to further study.

“One of the major issues they have proposed to us this year is that they are concerned about assessment of online and face-to-face courses,” he said. “In other words, are we really delivering what we say we are?”

Unlike a new wave of online education efforts – including massive open online courses – that are aimed at reducing costs for students, Oregon State charges a distance education fee that makes online courses a few hundred dollars more expensive than on-campus courses.

Oregon State’s Extended Campus, known as Ecampus, offers 900 online courses in over 80 subjects to 6,800 students. About a third of those students are entirely online. The others are picking up a few online courses while they do much of their work at the university’s main campus in Corvallis or a branch campus 130 miles to the east.

The university has studied its own performance and found mixed results.

Students who are enrolled entirely online – about 3,200, half of whom are outside of Oregon – had success rates that were “somewhat higher” than students who were also taking on-campus classes, according to a 2011 report from a task force appointed by the provost and then-president of the Faculty Senate. ​But that finding is based on judging success by completion with a passing grade, and not necessarily an examination of the courses.

Steve Clark, a university spokesman, said some of the difference might be explained by the different demographic profiles of students who come to campus versus those who do not.

The task force urged the university to conduct a deeper analysis of student success. That could take a while.

Nielson said faculty recent looked at a group of 30 courses offered both online and in-person, and meant for 300-level non-majors, in an attempt to compare the university’s on- and off-campus courses. About half the time they couldn't do it: 20 percent of online courses did not state what the expected learning outcomes were and 30 percent of on-campus courses did not say either. Nielson said the absence of such goals represented a failure to meet university policy at a basic level.

“The data to actually do this analysis doesn’t exist,” Nielson said. “You can argue about philosophy all you want, but if you don’t actually have any numbers then it’s all smoke and mirrors.”

Nielson, a former department chairman, said faculty members are part of the problem.

“Even though there are concerns on the part of the faculty, there is resistance by some of those same people to monitor and assess their work,” he said. “And where that comes from is: O.K., we’re already doing 60 hours a week of work, you want us to do more for us to prove we’re doing our job?”

He said it could be years before the university is able to do full comparisons.

University administration officials emphasized that all courses, including online courses, had to go through a standard approval process before they are offered. Dave King, an associate provost, said all courses are reviewed at their inception by the university's curriculum council.

“This is quite an extensive review process," King, who is traveling in China, said in an e-mail statement relayed by a spokesman. "For example, our online-only Master's of Natural Resources program took three years for review and was ultimately approved by the [Oregon University System] Provost’s Council.”

King said the university also monitors student evaluations.

Clark, the university spokesman, said Oregon State recently decided to bring in Quality Matters, an organization that helps colleges across the country evaluate the design of their online courses, to certify its courses. “Nationally, I think the nation needs to do a better job of developing educational outcome criteria and learning outcomes, whether it’s online learning or on-campus courses,” Clark said.

Who Teaches?

Another concern on the campus is the use of adjuncts or graduate students to teach online courses.

This is a national concern for faculty representatives. At Oregon State, there is some evidence that online courses are more likely to be taught by adjuncts or graduate students than are on-campus courses, but that research is also preliminary.

But, at Oregon State, some faculty may also be making matters worse, a charge made by some professors themselves.

Dwaine Plaza, president of the campus’s American Association of University Professors chapter -- which is working now to unionize the campus -- said some professors are putting their names on courses that are, in fact, being run by graduate students. 

“I’ve seen faculty that are just listing their names and it’s being run by graduate students or contingent [instructors],” Plaza said. “I think it’s problematic because the big picture is that students are sometimes looking at faculty members' names and are taking classes because the faculty members are teaching that class.”

Nielson said this phenomenon was a “big concern,” although it is not rampant at the university.

“You can get away with anything for a certain period of time,” he said. “Units that are doing that are sort of getting called onto the carpet now – which I think is the biggest difference between, say, us and some other for-profit online schools. We have a better idea of what we want to do and are sort of following up from that perspective, but it doesn’t mean we’re perfect.”

The 2011 task force report specifically called for the university’s Ecampus program to grow using quality instructors.

“With regard to instructors to deliver the courses, the committee felt it was appropriate to build its capacity using tenured and tenure-track professors and instructors,” the report said.

It’s unclear to what extent that is being done, however, though that tension is by no means unique to Oregon State.

Clark said the enrollment growth at Oregon State, which enrolls 26,000 students -- include those in Ecampus -- has helped the university hire more tenure-track faculty.

“The success of Ecampus at Oregon State, as well as the general growth in enrollment, has helped contribute to the hiring of more than 180 tenure track faculty in the last two years,” he said.

Plus, Clark said, its graduate students are not necessarily bad instructors. He cited one instructor who won a teaching award at the university.

Plaza also considers the Ecampus program, which generated $25 million in tuition revenue last year, to be a cash cow for the university, and he worries about pressure to make classes larger. Plaza also worries the university is relying, or will soon rely, on online instructors who do not work on the campus or have never set foot in Oregon.

Kevin Gable, the president of the Faculty Senate and a former chairman of the chemistry department, said he would not be surprised if that happened.

“The challenge we have to deal with is trying to capture what comes with this increased enrollment while at the same time dealing with reasonable expectations for what the workload of any individual faculty is,” Gable said.

He said that instead of a worse experience, the university may well be providing a better experience for its online students than for its on-campus students – but nobody knows for sure yet.

Compensation for Courses

Plaza and others have also complained about a change the university made two years ago to the way it paid the online courses instructors.

At first, the university was paying instructors per student.

Plaza said the per-student payment model was “seductive” for faculty. He said this meant some busy adjuncts could end up making more than some tenure-track faculty.

Then the model changed to pay instructors per course. The reasons for this are somewhat unclear. One university document attributed the change to an audit that said a per-student payment method conflicted with state rules.

Plaza sees a more suspect motive. He accuses the university of luring faculty, including himself, to teach online with a lucrative payment method. Then, once the program was off the ground, he said the university turned off the spigot.

“That’s where I think they did this sort of bait and switch on it because a lot of faculty were under the understanding that when the initial model came out it would be like that forever,” he said.

Nielson said there was concern that some people were “scamming the system,” but he said the changes were akin to “killing a gnat with a sledgehammer.”

“It certainly hurt some people who were just trying to do a good job,” Nielson said.