Search Views


Browse Archives

Views

Talking the Talk, Then Walking the Walk

September 25, 2009

Share This Story

FREE Daily News Alerts

Advertisement

Without intending it, I offended my friends by speaking a foreign language.

When I left a research center for the humanities and started work in a philanthropic foundation over five years ago, I wanted to know if a foundation could make a difference to the extent and depth of student learning in the liberal arts. To answer that question, I had to learn as much as I could about how students learn and how we know about their learning. Before long, I was studying reports such as the one produced by the Association of American Colleges and Universities’ Liberal Education and America’s Promise initiative (LEAP) that argued that liberal education ought to be understood not as exposing students to certain fields of knowledge, but as helping them to develop long-lasting cognitive and personal capacities. When I started using that phrase, I was on a slippery slope.

The next thing I knew, I was asking whether colleges and universities were translating that understanding of liberal education into clear learning outcomes. The phrase did not come tripping off the tongue, but the question was such an important one that I went right ahead and asked whether their practices were truly and effectively aligned with these outcomes. Were scaffoldings in place to help students move from one cognitive level to a higher one?

Despite its efforts to strengthen teaching, almost no one at the humanities center had spoken this lingo -- or asked such questions. When I started to do so, I found myself making the strange hiss sounds of “assessment,” a sound so savagely obnoxious that my friends began to hint that I was opening the gates to the barbarians.

I tried to conciliate them by substituting the term “evidence” for “assessment,” but they were too smart for that. And when I found I needed to investigate the various instruments that had been developed to help measure student learning, it was clear to many friends that I had gone over to the dark side. Terms such as NSSE, CLA, HERI, and CIRP were shibboleths that marked me as one of them.

It did no good to explain these were just convenient acronyms for titles in plain English. The titles themselves gave the show away: the National Survey of Student Engagement, for example, was clearly code for an alien view of education. The surveys were quantitative, a classicist friend noted with horror, warning me that “You can’t measure the human soul with numbers.”

Even worse, when I learned that the NSSE surveys had produced an empirical base for identifying a few high-impact practices, ones that demonstrably improved student engagement, learning, retention and graduation rates, the terms were so off-putting that in some quarters the ideas behind it could, as they say, gain no traction.

One friend -- who has somehow remained so despite my wayward behavior -- told me I needed to find some way to “translate” phrases such as high-impact practices into language more acceptable in the more ethereal reaches of the academy.

But I had done enough translating in my days as a classicist; now I was more interested in changing practice, and that, I realized, meant changing discourse. My theoretically minded friends had taught me one thing, after all. Discourse shapes practice.

Or, freely translated, “You have to talk the talk before you can walk the walk."

So I went on to other ophidian sounds, asking how higher education could successfully make systemic and systematic changes. Teagle Foundation grants for this purpose were going well, but the sibilants still sounded pernicious in many ears. Nor did it help to “translate” systematic into the phrase continuous quality improvement. That had few sibilants, but an unmistakable whiff about it of a Toyota factory or some other banausic enterprise.

The new mode of speech had a disconcerting inflection as well as an annoying vocabulary. For example, the stress in the “teaching and learning” moved disconcertingly from the first syllable of the dactyl, “teach’ing and … ” to the penult in the spondee, “learn’ing.” That reflects the emphasis in the new discourse on student learning. It expects students to take responsibility for their education rather than leaving the burden on “great teachers” and “good pedagogy.” Goodbye, Mr Chips. Hello daily development of cumulative cognitive and personal capacities.

Although it continues to give offense, the new discourse has in the last year or two passed a tipping point. It has now become the dominant mode of arguing, thinking and doing something about higher education.

There are two reasons, I believe, for this. First the accrediting organizations now insist on clear learning goals and rigorous assessment of progress toward them. And they are “drilling down” to the department and even course level to see what is being achieved.

More important, however, is a second reason: Faculty members who approach teaching in this way report that it is energizing, empowering, refreshing. It’s a welcome change from endless debates about the literary canon, or the curriculum. They say the terminology is no more opaque than the vocabulary of the economists, or the language we philologists use in establishing the stemmatics of ancient texts, or the useful technical terminologies developed in reader-response theory, deconstruction, and subaltern studies.

Every craft has its discourse, and every discourse shapes practice. It’s the results that count. It’s worth learning some new vocabulary when new friends whose speech I have come to understand are saying that they like having students who are more intensely engaged in learning, and taking greater responsibility for their education. They even talk about greater “satisfaction.”

How’s that for a change in discourse?

Robert Connor is president of the Teagle Foundation.

See all postings »
Advertisement
Advertisement

Matching Jobs

Comments on Talking the Talk, Then Walking the Walk

  • Assessment
  • Posted by shirley browning , Professor of Economics at UNC Asheville on September 25, 2009 at 7:15am EDT
  • Robert Conner reminds me of my experience in 1984 after having been an ACE Fellow (another way to alienate colleagues by studying or showing interest in administration!). During the fellowship I was introduced to assessment of student learning. For some reason it "made sense" so one of my responsibilities in 1984 - 85 was to try and get each academic department to write student learning objectives for their program as well as develop strategies to measure how well their students had achieved those objectives. Actually most departments and programs "gave it a try" - on paper; then filed the efforts in the traditional academic dust bin with not so kind mutterings.

    A few year's later while working in academic affairs I was asked to join a team of administrators to revisit this project in preparation for a reaccreditation effort, something else I had some knowledge of and growing experience as a reaccreditation team member visiting various institutions. Assessment was beginning to be one of the more dreaded words one could utter; but my inclination not to be sufficiently sensitive to peer or colleague distress caused me to forge ahead.

    Many months later all academic units had in place a system where by they were articulating and measuring student learning outcomes. On paper that is. But when we turned to our administrative colleagues the resistance there made faculty resistance appear inconsequential! Reaccreditation came and went, we passed; and again a decade later a very similar experience Now we face the task of preparation yet again for a reaccreditation visit in a few years and guess what: our challenge is to demonstrate institutional effectiveness through a program of ongoing assessment of objectives, in academic units now called Student Learning Outcomes or SLO's tied to mission statements, etc. Academic departments do appear to be somewhat on board at this point, even if in a primitive grumbling manner, 25 years after the notion was initially introduced. The rest of the institution, not so much but various offices appear to be a little more serious.

    A former Chancellor reminded me several years ago that in academe "change comes at glacial speed". Maybe the academy is beginning to "get it" in positive and productive ways. As I near retirement and follow the trials and tribulations facing higher education I can not help but wonder how much of our current stresses, excluding some - not all - recession based issues, have been self inflicted. Having friends among one's colleagues can be elusive, but also heart warming. Trying to foster change frequently compromises those relationships. To my younger colleagues where ever they may reside I wish you good luck, but unlike myself don't expect a warm reception based on your new found insights, understandings, and confidence that you are making a positive contribution by asking your colleagues to change their academic world view at more than glacial speed. Some will, most will be resistant and find you threatening and irritating. Be patient, kind, somewhat forgiving and wish only the best for your institution and colleagues.

    Shirley Browning

  • assessment as faculty development
  • Posted by Jeff Abernathy , Vice President and Dean at Augustana College on September 25, 2009 at 7:30am EDT
  • What a difference the Teagle Foundation under Bob Connor's leadership has made. Dr. Connor has done more for the assessment movement in higher education than the Spellings Commission ever could have. We at Augustana have worked with a consortium of colleges funded by Teagle for the past four years. Together these schools have found that Connor's second conclusive point--that faculty find this approach energizing and refreshing--is all the more the case when we bring faculty together from multiple campuses to consider the value added by our our efforts to educate students in broad areas such as writing, critical thinking and civic engagement. For our six liberal arts colleges, cooperation in measuring student learning has yielded results in faculty development well beyond what we expected as we began the project.

    Congratulations to Dr. Connor for developing a vision of an academy that places student learning first, which sounds like the original academy to me. Leave it to a classicist.

  • The scope of measurability is limited
  • Posted by grammaticus ordinarius on September 25, 2009 at 10:45am EDT
  • And yet it remains true that "what is essential is invisible to the eyes."

    There might be some immediately measurable outcomes that we could expect from a reading of the Odyssey, like the ability to identify the characters and tell their stories. But remember that there are higher non-measurable educational goals that lie far beyond the realm of immediately evident results. If you stick only to the latter, you may end up killing the humanities.

  • jargon creep
  • Posted by Lamont Lindstrom , Anthropology at University of Tulsa on September 25, 2009 at 12:15pm EDT
  • Whereas "data-based management" and other assessment ritual was pushed by conservative think-tanks and lobby groups starting in the late 1970s (we see the remarkable success of these efforts in shaping federal support for public schools (No Child Left Behind) and a lot of churning in universities thanks to the capture of the accreditation agencies), much of the jargon of such magical practice derives from military language, filtered through business yak. Check out the origins of "mission statement" in your friendly OED--nowadays, one may not posit those "learning objectives" without a governing mission statement. ("Vision statements" may have softer origins.) Even our syllabi have to have governing course mission statements...and don't forget to add one to your CV too! But all jargons, even one as new and esotheric as this, bloom but then decay....

  • On Killing the Humanities
  • Posted by Hugh Manatee on September 25, 2009 at 12:15pm EDT
  • Good riddance. On the other hand, we might be worse off without them. It's hard to say.

  • well maybe
  • Posted by Chris on September 25, 2009 at 1:00pm EDT
  • I think it makes sense to look at measurable outcomes, but it is also important to realize the limitations of this approach (at both college and pre-college level). It can easily become a tail wagging the dog situation. Some important goals are just inherently difficult to measure with any precision, and what often happens is that what is easily measured becomes the organizational goal (goal displacement).

  • Kill the humanities?
  • Posted by Larry Shillock , Associate Professor of English at Wilson College on September 25, 2009 at 1:00pm EDT
  • Hugh,

    It isn't hard to say that killing the humanities is a bad idea. (And yes, I recognize that your ambivalence may be ironic.) The kind of teaching and learning that I do--including teaching The Odyssey--prepared me for the tragedy-in-the-making that gave us the Great Recession. It was obvious to me, a student of Homer and Voltaire and Flaubert, that the U. S. economic system was on the verge of a critical setback. I didn't need to be trained in economics to identify the fallacies in the field of economics. I didn't need to take out a mortgage that I couldn't afford to know that home values go up and down. I recognized the logical flaw in the statement that "times are different now and history doesn't apply" to how we invest, regulate Wall Street, and so on. Understanding "tragedy" is something we learn in the humanities and thus learn to avoid--provided, of course, that we take such learning to heart.

  • non-measurable?
  • Posted by Ken , Asst Prof at Furman University on September 25, 2009 at 3:30pm EDT
  • I disagree that some forms of student learning are non-measurable. I think this assumption is based on the perception that all assessment must be postivistic and quantitative. There are other forms of assessment that do not involve standardized tests.

    If you can imagine what you want your students to gain from your course, you can devise a way to determine whether or not they are achieving those goals. It may be hard to do, but it is not necessarily impossible.

    I think a lot of the resistance to assessment is due to a general unawareness (and devaluation) of non-quantitative social science methodology.

  • Beyond belief!!!
  • Posted by grammaticus ordinarius on September 25, 2009 at 4:15pm EDT
  • To Ken:

    You don't think some forms of student learning are non-measurable?!!

    Okay, one highly touted educational value is "greater openness to other cultures." How do you determinate precisely how culturally open the student is going into and then out of your course? How do you prove beyond all reasonable doubt that the change was due to your course and not to the tv programs, movies, dorm discussions, general maturation processes, etc., that the student experienced while he/she was in the class? Remember we want precision, quantities, measurable units here. What units of measurement are you going to use, Ken? Would you accept the students' own self-observations about what they learned? (This does happen.) How unscientific!

    Of course, we hope educators just don't want "openness" because then the student's acceptance of culture x's violent cannibalistic rituals or culture y's passionate genocidal hatreds might count as an educational good. So then how do we quantify exactly what we want in the way of openness? There have to be some other factors involved.

  • Assessment and the promotion of deep learning
  • Posted by John C. Bean , Professor, English at Seattle University on September 25, 2009 at 5:30pm EDT
  • I appreciate Dr. Connor’s hopeful tenacity as he works to change the national discourse on student learning. I also second Jeff Abernathy’s point that assessment--done in a faculty-owned, student-centered spirit--can be a powerful way to stimulate faculty discussion about teaching and learning. No doubt to many faculty—perhaps most-- assessment is both maddening and deadening: It attempts to quantify what is ineffable and transcendent; it takes up enormous faculty time that should be spent on teaching or research; and it encourages departmental cynicism in the production of low-meaning numbers aimed at producing bureaucratic reports. The only way to defend assessment is to reframe it in terms of its ability to enhance “deep learning.”
    On my campus (Seattle University) I would place faculty supporters of assessment initially into two groups depending on their inclination toward empiricism. One group of faculty, versed in statistical analysis, is inclined toward pre- and post-tests, control group/treatment group experiments, or other evidence-based means to measure student gains. In contrast, a second group of faculty (I am in this group), tends toward a simpler, less time-consuming, and less potentially positivist approach. This group is more comfortable with qualitative judgments and appreciates Barbara Walvoord’s assertion (made in a workshop at Seattle University several years ago) that the foundational assessment act is a faculty member’s grading of student work. This group approaches assessment by relying on an instructor’s grading of an assignment already embedded in a course and on his or her analysis of patterns of strengths and weaknesses based on specified criteria. Ensuing department discussion of weaknesses in student performance leads to backward design of the curriculum to improve instructional methods, assignments, or goals of certain courses.
    This approach has a remarkable ability to create faculty community around whatever learning outcomes faculty members value (whether improved capstone papers from seniors or intellectual and experiential engagement with social justice). In Dr. Connor’s terms, “Faculty members who approach teaching in this way report that it is energizing, empowering, refreshing.” Seattle University has been aided in our approach through a substantial grant from the Teagle Foundation (in a consortium with Gonzaga University) for a writing-in-the-majors program and an improved Core curriculum, so I have particular appreciation for the remarkable work of this foundation achieved through Dr. Connor’s leadership.

    .

  • On killing the humanities
  • Posted by CC Prof on September 25, 2009 at 5:30pm EDT
  • I teach philosophy, and I never worry about all of this nonsense about how the humanities are going to die out or how maybe they should be killed. Basically, the enduring questions about the meaning of life, etc. that the humanities address are not going away, and we will always have students interested in those questions. This interest will wax and wane, but it will always be there. Currently, it appears to be waxing. I teach at a community college, and I have plenty of students in my philosophy courses as do the other philosophy, religion, English, history, and humanities professors.

    As for assessment in these areas, I seriously doubt if anyone is going to discover anytime soon a simple, objective way to measure whether a student really gets the Odyssey, or Hamlet, or Plato's Republic. I assess my students by making them talk to me and by making them write. This is not much different than what Socrates, Plato, Aristotle, and many others have done before me.

    Of course, there have been efforts to kill the humanities before. The Athenians did kill Socrates. But that did not stop the sort of rational inquiry into the meaning of life that he engaged in. I, for one, refuse to be intimidated by the new crowd of critics of the humanities, especially those who call for the elimination of the humanistic disciplines from the academy rather than merely pointing out real problems with approaches, theories, etc. within the humanities.

  • Yes, non-measurable.
  • Posted by Eveningsun on September 25, 2009 at 5:30pm EDT
  • -- Non-measurable within a useful time frame (when the goal is "Lifelong learning"--life is much longer than assessment fads).

    -- Non-measurable with the resources provided to develop the assessment (when a bunch of humanists with no relevant training are told to develop, on their own, in their spare time, an assessment to measure attainment of goals like "Lifelong learning," "Openness to other cultures," "Critical thinking," "Appreciation of literature," etc.)

    -- Non-measurable when the goals to be assessed cannot be agreed upon. Disagreement over the goals of (say) literary study are deep and wide and protected by academic freedom. No single set of measures can adequately cover the most important things going on in different classrooms. You can select a single set of measures anyway, and pretend to yourself (and more importantly your administrative overlords) that what you're measuring is what the instructors are trying to teach, but the results aren't necessarily going to tell you anything useful.

    -- Non-measurable when the assessment regime changes so frequently (often because of a "culture of continuous improvement") that longitudinal comparison is never possible.

  • Ha ha!
  • Posted by cacambo on September 26, 2009 at 5:15am EDT
  • I never cease to get a big belly laugh when the High Humanists claim that our job is to spread the magic fairy dust--the effects of which can never be quantified. Yet I'm willing to bet that most of the posters above who argue for the ineffability of the humanities tally up a bunch of (numerical) points and the end of the semester and assign a grade. (And this is done without the slightest concern for or knowledge of measurement science.) Honestly, I don't have a dog in the assessment fight, but I've been in the academy long enough to have witnessed in many different contexts an astonishing lack of curiosity about what and how students learn.

  • The aesthetics of not knowing
  • Posted by Gary Phillips , Dean of the College and Professor of Religion at Wabash College on September 26, 2009 at 2:15pm EDT
  • Bob Connor's persistent effort to leverage improvement in student learning is a challenge to a double-sided aesthetics of student learning and of faculty teaching that many academicians hold dear, namely that truth is beauty, and since truth lies in the eye of the beholder there is nothing more to be said or can be known. Such an aethetics internalizes public matters, whether on the learning or teaching side. It inhibits the practical self-critique expected of all teachers about their own teaching practices and diverts attention from wanting to know whether teaching and learning is in any sense a public and institutional matter.

    As I see it, the issues are connected deeply--or should be--to matters of meaning-in-life, a concern that Bob and his colleagues at Teagle have strongly insisted must not be lost from liberal arts teaching and learning. As a faculty member whose field is religion, a declared preserve for such meaning-in-life questions, clarification of my ability to know whether my teaching has improved student capacity to think critically and live humanely is a meaningful question to ask. Or translated: is their any evidence that my teaching of Gilgamesh or Genesis has succeeded, and if so how or if not why, in deepening student comprehension of these ancient texts and their import for contemporary human self-understanding? Unless I am prepared to interrogate teaching and learning practices and make the conversation a public one -- no easy task I confess -- we hamper student ability to fully ask and interrogate meaning-in-life questions.

    Thanks, Bob, for your persistence.

  • Quantifiability
  • Posted by CC Prof on September 26, 2009 at 2:15pm EDT
  • Dear Cacambo,

    You seem to be assuming that if something cannot be quantified, then it cannot be judged.
    That assumption is certainly false. Frequently, cardinal rankings are possible where ordinal ones are not. A good judge of paintings will be able to judge some paintings as far superior to others without using any numbers.

    As for the use of numbers by humanists while grading, I would imagine, as is true in my case, that that is done largely for appearances and expediency. I would prefer to merely take some notes on student papers, rank the students in my courses, and assign grades based upon that. Actually, it would be preferable to keep portfolios of all student work for a course, and then assign grades after a thorough examination of the portfolios, including, perhaps, an oral examination of the student. I would like to do that, but with well over 100 students each semester that would be difficult.

    But if the science of measurement has figured out how to mechanically measure the level of understanding that a student has for a difficult text, a complicated argument, a nuanced position, a painting genre, a beautiful poem, a complex historical event, etc., then by all means let us in on the secret.

  • Who is responsible?
  • Posted by Skeptic on September 27, 2009 at 12:00am EDT
  • Professors are responsible for assessing individual students. Students are responsible for their own learning. The whole assessment fad turns responsibililty on its head, as if professors have control over student learning. Do you expect that people in the business community consider employees to lack responsibility to carry out their job requirements? Then why would universities not place responsibility for learning on students?

  • Difficulties in measuring
  • Posted by Assistant Phil. Professor on September 27, 2009 at 2:45pm EDT
  • Here is some information about outcomes assessment and the difficulties of oversimplified measures in philosophy courses. This publication is an official statement of the American Philosophical Association. It is long but useful.

    http://www.apaonline.org/documents/governance/APA_Outcomes_2008.pdf

  • Actually, Cacambo...
  • Posted by Eveningsun on September 28, 2009 at 1:15pm EDT
  • ...we "High Humanists" think all the time about the problems involved in grading. We give grades anyway, for the simple reason that we're required to. We also know perfectly well that we lack "knowledge of measurement science." That's one reason why we're so skeptical when administrators 1.) pretend to be taking assessment seriously but 2.) refuse to fund it and instead ask us humanists to construct the assessments.

  • What are you paid to do?
  • Posted by Bob Duniway , AVP for Planning at Seattle University on September 28, 2009 at 6:15pm EDT
  • It apparently doesn't occur to many faculty that they are being economically compensated for a combination of teaching and scholarly activity. We could get into a long conversation about how we measure the contribution of scholarship, but is there any other way to evaluate the contribution of teaching other than student learning and development? Yet many faculty have never been formally taught skills in effective teaching, and similarly many faculty have maintained that whether or not students learn is not their responsibility but rather the responsibility of the students.
    Imagine if health care providers, cognizant that they don't entirely control the health outcomes of their patients, asserted that they weren't in any way responsible for these outcomes and that the burden rested entirely with the patients. Hospitals face outcomes assessment problems where the easily measured (death rates over time intervals) is a distortion of the most desired outcomes (quality of life) or of higher order outcomes not automatically produced by the market (ethical provision of care in ways that achieve the best collective results). That doesn't mean doctors or hospital administrators can long get away with rejecting the entire enterprise of outcomes assessment by pointing to its abuses and inefficiencies. But of course they lack tenure.

  • what about tests?
  • Posted by grammaticus ordinarius on September 28, 2009 at 9:15pm EDT
  • Bob asks "is there any other way to evaluate the contribution of teaching other than student learning and development?" What about the old-fashioned quizzes, tests, and exams? Such assessment tools go back a long way. They combine with the ever-present, live assessment abilities of the teacher, which parallel those of connoisseurs, i.e., people who have developed an expertise and can make such judgments. You trust them or you don't. You don't make the connoisseur use some kind of bureaucratically-engineered tool whereby there is an objectification that can then be itself assessed by some kind of third party. (Who will assess the assessors?) "Outcomes assessment" procedures seem to want to go beyond the traditional, time-honored approach to add yet another level. Read Michael Polanyi: not everything can be explicitly articulated. There is always a tacit dimension. Hence the practices of apprenticeship.

  • qualitative data vs. quantitative data
  • Posted by Ken , sociology at furman university on September 29, 2009 at 10:00am EDT
  • grammaticus,

    I agree with your claim "openness" cannot be easily quantified. That is why qualitative data would be more valid and reliable. It would not be easy, for all the reasons you outlined, but it would not be impossible. If you want to know how it could be done, talk with a qualitative social scientist.

  • Bob Duniway
  • Posted by DFS on September 29, 2009 at 1:45pm EDT
  • Standards are standards. Technology should only act toward improving students demonstrating standards.

    That's all that matters. Any thought about 'teaching instruction' is only spin designed towards the lowering of the standards.

    That's what's been happening over these years, now.

  • Reading 'riting 'rithmetic and Reasoning
  • Posted by Craig , Director of Data Services - University Division at Indiana University on November 13, 2009 at 5:00pm EST
  • When I arrived at college as a freshman my ability to reason was limited by my mental maturation and lack of exposure to reasoning with ideas and not just facts as a high school student. Through exposure to classes requiring reasoning with ideas and maturation of my brain, the interaction developed my reasoning abilities. I received the right kind of stimulation at the right time. It was not one class, but a maturation process that occured over four years. As a math major, it was probably the non-math courses that developed my reasoning abilities the most (writing about historical concepts, political science, education theory, and others). As a senior I was required to prove my understanding of the required math courses (comprehensive exam) and do an oral presentation on a topic of the Math departments chosing. I think it would have also been a good assessment of my 4 years of education if I had also been required to research and write a paper on a non-mathematical topic.

    I think Bloom's taxonomy is still a good general measure of the mental demands of a college course. Not all students will be able to function at the highest levels after a set period of time, but the courses should continue to challenge them to have "continual improvement." Not all students will get A's.