administrators

Essay on how fixation on 'inane' student learning outcomes fails to ensure academic quality

In a recent Century Foundation essay, I raised a concern that accreditors of traditional colleges are allowing low-quality education to go unaddressed while insisting, in a misguided attempt to prove they care about learning, that colleges engage in inane counting exercises involving meaningless phantom creatures they call student learning outcomes, or SLOs.

The approach to quality assurance I recommend, instead, is to focus not on artificially created measures but on the actual outputs from students -- the papers, tests and presentations professors have deemed adequate for students to deserve a degree.

I got a lot of positive feedback on the essay, especially, as it happens, from people involved in some of the processes I was criticizing. Peter Ewell, for example, acknowledged in an email that “the linear and somewhat mindless implementation of SLOs on the part of many accreditors is not doing anybody any good.”

This story began in the 1990s, when reformers thought they could improve teaching and learning in college if they insisted that colleges declare their specific “learning goals,” with instructors defining “the knowledge, intellectual skills, competencies and attitudes that each student is expected to gain.” The reformers’ theory was that these faculty-enumerated learning objectives would serve as the hooks that would then be used by administrators to initiate reviews of actual student work, the key to improving teaching.

That was the idea. But it hasn’t worked out that way. Not even close. Here is one example of how the mindless implementation of this idea distracts rather than contributes to the goal of improved student learning. When a team from the western accreditor, the WASC Senior College and University Commission, visited San Diego State University in 2005, it raised concerns that the school had shut down its review process of college majors, which was supposed to involve outside experts and the review of student work. Now, 10 years have passed and the most recent review by WASC (the team visit is scheduled for this month) finds there are still major gaps, with “much work to be done to ensure that all programs are fully participating in the assessment process.”

What has San Diego State been doing instead of repairing its program review process? It has been writing all of its meaningless student learning outcome blurbs that accreditors implemented largely in response to the Spellings Commission in 2006. San Diego State reported its progress in that regard in a self-review it delivered to WASC last year:

Course Learning Outcomes (CLOs) are required for all syllabi; curricular maps relating Degree Learning Outcomes (DLOs) to major required courses are now a required component for Academic Program Review; programs are being actively encouraged to share their DLOs with students and align DLOs with CLOs to provide a broader programmatic context for student and to identify/facilitate course-embedded program assessment.

All this SLO-CLO-DLO gibberish and the insane curriculum map database (really crazy, take a look) is counterproductive, giving faculty members ample ammunition for dismissing the idiocy of the whole process. The insulting reduction of learning to brief blurbs, using a bizarre system of verb-choice rules, prevents rather than leads to the type of quality assurance that has student work at the center.

The benefits of, instead, starting with student work as the unit of analysis is that it respects the unlimited variety of ways that colleges, instructors and students alike, arriving with different skill levels, engage in the curriculum.

Validating colleges’ own quality-assurance systems should become the core of what accreditors do if they want to serve as a gateway to federal funds. Think of it as an outside audit of the university’s academic accounting system.

With this approach, colleges are responsible for establishing their own systems for the occasional review of their majors and courses by outside experts they identify. Accreditors, meanwhile, have the responsibility of auditing those campus review processes, to make sure that they are comprehensive and valid, involving truly independent outsiders and the examination of student work.

SLO madness has to stop. If accreditors instead focus on the traditional program-review processes, assuring that both program reviews and audits include elements of random selection, no corner of the university can presume to be immune from scrutiny.

Robert Shireman is a senior fellow at the Century Foundation and a former official at the U.S. Department of Education.

Essay on value of student learning outcomes in measuring and ensuring academic quality

Robert Shireman is right. The former official at the U.S. Department of Education correctly wrote recently that there is little evidence that using accreditation to compel institutions to publicly state their desired student learning outcomes (SLOs), coupled with the rigid and frequently ritualistic ways in which many accreditation teams now apply these requirements, has done much to improve the quality of teaching and learning in this country.

But the answer, surely, is not to abolish such statements. It is to use them as they were intended -- as a way to articulate collective faculty intent about the desired impact of curricula and instruction. For example, more than 600 colleges and universities have used the Degree Qualifications Profile (DQP). Based on my firsthand experience with dozens of them as one of the four authors of the DQP, their faculties do not find the DQP proficiency statements to be “brief blurbs” that give them “an excuse to dismiss the process,” as Shireman wrote. Instead, they are using these statements to guide a systematic review of their program offerings, to determine where additional attention is needed to make sure students are achieving the intended skills and dispositions, and to make changes that will help students do so.

As another example, the Accreditation Board for Engineering Technology (ABET) established a set of expectations for engineering programs that have guided the development of both curricula and accreditation criteria since 2000. Granted, SLOs are easier to establish and use in professional fields than they are in the liberal arts. Nevertheless, a 10-year retrospective study, published about two years ago, provided persuasive empirical evidence that engineering graduates were achieving the intended outcomes and that these outcomes have been supported and used by faculties in engineering worldwide.

Shireman also is on point about the most effective way to examine undergraduate quality: looking at actual student work. But what planet has he been living on to not recognize that this method isn’t already in widespread use? Results of multiple studies by the National Institute for Learning Outcomes Assessment (NILOA) and the Association of American Colleges and Universities (AAC&U) indicate that this is how most institutions look at academic quality -- far exceeding the numbers that use standardized tests, surveys or other methods. Indeed, faculty by and large already agree that the best way to judge the quality of student work is to use a common scoring guide or rubric to determine how well students have attained the intended proficiency. Essential to this task is to set forth unambiguous learning outcomes statements. There is simply no other way to do it.

As an example of the efficacy of starting with actual student work, 69 institutions in nine states last year looked at written communications, quantitative fluency and critical thinking based on almost 9,000 pieces of student work scored by faculty using AAC&U’s VALUE rubrics. This was done as part of an ongoing project called the Multi-State Collaborative (MSC) undertaken by AAC&U and the State Higher Education Executive Officers (SHEEO). The project is scaling up this year to 12 states and more than 100 institutions. It’s a good example of how careful multi-institutional efforts to assess learning using student work as evidence can pay considerable dividends. And this is just one of hundreds of individual campus efforts that use student work as the basis for determining academic quality, as documented by NILOA.

One place where the SLO movement did go off the rails, though, was allowing SLOs to be so closely identified with assessment. When the assessment bandwagon really caught on with accreditors in the mid-1990s, it required institutions and programs to establish SLOs solely for the purpose of constructing assessments. These statements otherwise weren’t connected to anything. So it was no wonder that they were ignored by faculty who saw no link with their everyday tasks in the classroom. The hundreds of DQP projects catalogued by NILOA are quite different in this respect, because all of them are rooted closely in curriculum or course design, implementing new approaches to teaching or creating settings for developing particular proficiencies entirely outside the classroom. This is why real faculty members in actual institutions remain excited about them.

At the same time, accreditors can vastly improve how they communicate and work with institutions about SLOs and assessment processes. To begin with, it would help a lot if they adopted more common language. As it stands, they use different terms to refer to the same things and tend to resist reference to external frameworks like the DQP or AAC&U’s Essential Learning Outcomes. As Shireman maintains, and as I have argued for decades, they also could focus their efforts much more deliberately on auditing actual teaching and learning processes -- a common practice in the quality assurance approaches of other nations. Indeed, starting with examples of what is considered acceptable-quality student work can lead directly to an audit approach.

Most important, accreditors need to carefully monitor what they say to institutions about these matters and the consistency with which visiting teams “walk the talk” about the centrality of teaching and learning. Based on volunteer labor and seriously undercapitalized, U.S. accreditation faces real challenges in this arena. The result is that institutions hear different things from different people and constantly try to second-guess “what the accreditors really want.” This compliance mentality is extremely counterproductive and accreditors themselves are only partially responsible for it. Instead, as my NILOA colleagues and I argue in our recent book, Using Evidence of Student Learning to Improve Higher Education, faculty members and institutional leaders need to engage in assessment primarily for purposes of improving their own teaching and learning practices. If they get that right, success with actors like regional accreditors will automatically follow.

So let’s take a step back and ponder whether we can realistically improve the quality of student learning without first clearly articulating what students should know and be able to do as result of their postsecondary experience. Such learning outcomes statements are essential to evaluating student attainment and are equally important in aligning curricula and pedagogy.

Can we do better about how we talk about and use SLOs? Absolutely. But abandoning them would be a serious mistake.

Peter Ewell is president of the National Center for Higher Education Management Systems (NCHEMS), a research and development center.

Editorial Tags: 

NLRB Rejects Tufts Medical Faculty Union Bid

A regional office of the National Labor Relations Board dismissed on Tuesday a petition from a group of tenure-line basic science faculty members at the Tufts University School of Medicine to hold a union election. The decision was based largely on a set of tests to assess faculty members’ managerial status established by a major 2014 NLRB decision concerning an adjunct faculty union bid at Pacific Lutheran University. The NLRB regional office said the members of Tufts’ proposed unit were in fact managers under those guidelines and therefore ineligible to form a union. Additionally, science faculty members with labs and direct reports are supervisors, according to the decision.

Siobhan Gallagher, a Tufts spokesperson, said in a statement that the university is pleased the NLRB office “recognizes the significant authority that our faculty members have in critical areas of the school’s management. We look forward to continuing dialogue and collaboration with our faculty.”

Jason Stephany, a spokesman for Service Employees International Union, with which the proposed unit is affiliated, said the NLRB decision "validates faculty concerns over the definition of tenure at the School of Medicine. … Tufts faculty disagree with several key points that form the basis of the regional director's overall ruling, and we will review our options for a potential appeal in the coming days."

Ad keywords: 

Why universities hire two deans to lead their law schools

Smart Title: 

Over the last few years, universities started appointing two deans to lead their law schools temporarily. Now, a few are going that route with permanent appointments.

Survey: 90% of Students Feel Unready to Repay Loans

A survey of roughly 90,000 students, most of whom attend four-year colleges, found that 90 percent of respondents feel they do not have all the information necessary to pay back their student loans. EverFi, an education technology company, conducted the survey, which was funded by Higher One, a financial company focused on higher education. It is the fourth installment of the annual survey. This year's version found a continuing decrease in students' planning for responsible financial behaviors.Line graph showing downward-trending changes in students' attitudes about various financial behaviors, such as following a budget or paying bills on time, from 2012 to 2015.

An Apology as Duke Sit-In Continues

Duke students camp out in tents during a sit-in of the university's main administration building.As of Monday evening, nine students continued to occupy the main administration building at Duke University, protesting what they see as poor treatment by the university of its employees. One of the grievances concerns an accident involving Tallman Trask, Duke’s executive vice president, who a parking attendant has charged hit her with his car and used a racial slur before a 2014 football game. Trask has apologized for hitting the parking attendant, Shelvia Underwood, which he says was accident, but denied using a racial slur.

On Monday, he issued a public apology to Underwood. "I want to say a word to the Duke community about my interaction with Shelvia Underwood in August 2014, which has been a subject of much recent discussion," said Trask's statement. "While the details of what happened are a matter of disagreement and subject of civil litigation, I recognize that my conduct fell short of the civility and respectful conduct each member of this community owes to every other. I express my apology to Ms. Underwood and to this community and recommit myself to ensuring that these values are upheld for all."

The students occupying the building did not leave as a result of the apology, and others are camping outside the building to express support (above right).

Duke announced Monday that it would not continue negotiations with the students as long as they remain in the administration building, the Allen Building, which has been closed since the students occupied it. "Closing the Allen Building while these negotiations go on has presented a significant disruption to students, faculty, staff and visitors, and cannot continue indefinitely. As a result, the university will only continue negotiations after the nine students voluntarily leave the Allen Building," said a statement from Duke.

Ad keywords: 

Dozens of Division III colleges violated financial aid rules in past decade

Smart Title: 

Though it's a violation of the division's "bedrock principle," the number of Division III colleges that consider athletic ability when making financial aid decisions is increasing.

Harvard, Princeton Release Statements on Endowments

Harvard and Princeton Universities have released their responses to questions from members of Congress about the way they use their endowments. The questions come amid heightened scrutiny of the wealthiest universities. Both the Harvard and Princeton letters to Congress stress common themes, including the way their endowments are not general funds but collections of endowments donated for different purposes, and that the endowments directly support undergraduate student aid among many other purposes. A cover letter on Harvard's response, from President Drew Faust, said that her university's endowment should be viewed as 13,000 separate funds. Princeton's letter indicated that its endowment is made up of 4,300 separate accounts.

Harvard's endowment (at more than $36 billion) is the largest in the nation, and Princeton's (at nearly $23 billion) is the fourth, according to the latest data from the National Association of College and University Business Officers and Commonfund.

Villanova Professor Arrested on Child Porn Charges

A professor of history at Villanova University was charged with 415 child pornography-related counts, after the campus tipped off police to his computer activity, according to CBS News. Christopher Haas, 60, reportedly was observed watching child pornography on his computer in a campus hall by Villanova security personnel. He’s been barred from campus and is in jail on $50,000 bail. Haas was the subject of a separate federal child porn investigation in 2012 and allegedly had some 400 such images on his computer. Scott Godshall, Haas’s lawyer, said the federal case was closed without charges. A Villanova spokesperson said the university was unaware of the 2012 investigation. (Note: This post has been updated from a previous version to reflect that Villanova did not know about the older case against Haas.)

Ad keywords: 

Students Occupy Duke Administration Building

Nine students on Friday occupied the portion of an administration building that houses the offices of the president and other senior administrators. The students (and supporters who have gathered outside the building) have demanded the dismissals of several Duke administrators, including Tallman Trask, Duke’s executive vice president, who a parking attendant has charged hit her with his car and used a racial slur before a 2014 football game. Trask has apologized for hitting the parking attendant, which he says was accident, but denied using a racial slur. Duke has said that the incident was investigated and that it can't comment on details of some of the accusations because the attendant is suing the university.

Many of the other demands relate to Duke's treatment of its employees, and a university statement said, "Duke University has well-established internal and legal processes for addressing concerns of any employee, regardless of their position. These are spelled out in the university’s human resources policies, and are covered by state and federal law."

Students who organized the protest have received wide attention on social media under the hashtag #DismantleDukePlantation.

On Sunday, Duke announced that the administration building would be closed today while negotiations with the students continue. In addition, the university announced that it will not punish the students. "In order to facilitate productive dialogue and move towards a peaceful resolution, the nine students will not be subject to student conduct sanctions and legal penalties for their actions," said a statement from the university.

Ad keywords: 

Pages

Subscribe to RSS - administrators
Back to Top