The Tricky Relationship Between Assessment and Learning

Sound clip: You’re listening to a Times Higher Education podcast.

Miranda Prynne: Hi there, welcome to this episode of the Campus podcast, in which we are putting assessment under the examiner’s lens and asking what purpose does it serve? Is it a help or a hindrance to learning? We are assessing assessment, if you like. And by the end of this episode, you might even be able to offer assessment a final grade. Or you may decide that written feedback would be more helpful. I’m Miranda Prynne, Campus editor, your host for this week. And to help me get to grips with this topic, I am speaking to three US academic experts in assessment, teaching and learning, who have all devoted many hours to researching and writing about how the three fit together in the context of higher education. As you’ll hear shortly, assessment is not without its fair share of controversy, particularly around grading, with a growing body of evidence suggesting constantly rating or measuring students’ progress may in fact undermine their learning. But in higher education, as in life, nothing is black and white. And these conversations provide insight on how assessment and grades can be reframed as feedback in the service of learning, and why assessment done right is a valuable tool. So, joining me today is author, educator and anthropologist Susan Blum, who is a professor at the University of Notre Dame. She has written a number of books, including a trilogy critiquing the way university teaching is delivered, with the latest, Schoolishness, coming out last year. Catherine Wehlburg is president of Athens State University and president of the Association for the Assessment of Learning in Higher Education. Catherine is known for her research on assessment of student learning outcomes and is currently editor in chief for the New Directions for Teaching and Learning series. And finally, Josh Eyler is director of the Center for Excellence in Teaching and Learning and clinical assistant professor of teacher education at the University of Mississippi. He has also published a number of highly regarded books on the science of learning and most recently on grades. Each of these educators offer different perspectives on the role assessment should play in higher ed. We’ll start with Susan who provides a fantastic summary of the problems associated with traditional university teaching methods and how lecturers and tutors can harness students’ natural desire to learn and of course where assessment fits into all this. Now a quick disclaimer. I’m afraid we had a number of Wi-Fi issues during this recording. Susan was very forbearing, but it did mean there was some chopping and changing during our call, which might be evident, but happily at no point did it render us incomprehensible.

Miranda Prynne: Hi Susan, it’s lovely to talk to you.

Susan D. Blum: Lovely to talk to you too, Miranda.

Miranda Prynne: Now you have written a trilogy of books critiquing the way our schools and universities teach, with the most recent coming out last year under the title Schoolishness: Alienated Education and the Quest for Authentic, Joyful Learning. Can you tell me what you mean by schoolishness and why this phrase felt appropriate?

Susan D. Blum: What I meant by the phrase schoolishness was the things that we do in school that are only for the purpose of school, not things that actually occur outside school. So, things like multiple choice tests, or rooms where only one person is talking all the time and everybody else is trying to please them, or certain kinds of writing assignments that only really exist in the academy, or genres of materials like textbooks, which also only occur in school. And once I began to notice, then I began to see the contribution of the schoolish dimensions of university life on the fact that so much of what some students were experiencing was alienation and also suffering, literally anxiety, depression, boredom, all kinds of things. And it doesn’t have to be that way because we can still help students learn well without incorporating all of these schoolish dimensions.

Miranda Prynne: So, it sounds like what you’re describing there really is what we would now perceive as a traditional approach to university teaching, which lacks that authenticity of what the real world is actually like.

Susan D. Blum: Yes, I mean, one of the things that has become really clear to me since I began researching the nature of education 20 years ago now is that outside school, human beings are incredibly avid and successful learners. It’s part of our nature as the kinds of animals we are without a lot of instincts, but we build on the kind of accumulated knowledge of society. And we do that because it matters, because we want to, because we need to. And what happens in school is that students become kind of passive so that they are waiting to be told what they’re supposed to do. And one of the things that really has troubled me is that they often do it only for the grade or the credential or the mark, and then they forget it. And students tell me this over and over and over.

Miranda Prynne: What you describe is very much reflected here, too. That means-to-an-end approach to learning to get a grade or whatever a very short-term goal is, rather than that broader idea of becoming an educated human being. You mentioned that this system is not only ineffective, but can be alienating and do active harm. Can you explain that a bit further?

Susan D. Blum: Well, a lot of research has shown there’s a lot of mental illness, mental suffering that comes because of the kinds of structures of higher education. So levels of depression, anxiety, self-harm, even suicidality are often attributed to the kinds of alienated goals that happen in the case of the academy. In the U.S., the National Institutes of Health have declared high-achieving schools or attending high-achieving schools as a risk factor for mental illness and suffering. Students who attend such schools are at much greater risk of having adverse outcomes. At the same time, of course, they are also having benefits, social benefits of increased social capital and cultural capital. So it’s a kind of damned if you do, damned if you don’t situation because we know that attending a highly selective institution does offer more life choices, but at the same time, it imposes all of these other risks. So that is something that I think we really need to take very seriously.

Miranda Prynne: That’s a very sobering finding. There’s a strange irony to the fact that these are places that people desperately want to get to and they aspire to get to. I’m very keen to come on to talk more about where grades and the whole assessment system sit within all this. But first, I did want to stick with this wider philosophy that you are proposing because you talk about education needing to return to learning in the wild and you’ve alluded to the fact that humans are natural learners. Again, I’d love you to point to how it actually could be implemented, what a teacher in a university, how they might take this and apply it.

Susan D. Blum: Yes, well, I think if you begin from the premise that we don’t have to force people to learn, but that our role is more facilitating their learning or helping to provide conditions in which learning can occur. And if we are seen as partners in the learning of our students or our learners rather than the sole authority, then it changes everything. So many people have gone to active learning, project-based learning, inquiry-based learning, question-based learning, problem-based learning. I had done some ethnographic field work in a summer internship, which I talk about in a chapter in the book, where basically all of the right dimensions come together. So, students are motivated because they care, they have responsibility to the community and to others, they’re fascinated. It doesn’t mean everything is easy, it means that it’s meaningful. And when it’s meaningful, people are willing to do hard work. And there are no ways to cut corners because the actual outcome is authentic rather than the kind of artificial schoolish metrics that we’re so used to. And the questions are motivated by genuine concerns. This can be done in classrooms when students choose problems, projects that they care about. And then the more technical learning, the facts, the skills can be learned along the way in the service of something more compelling. And I saw this happen in the summer internship. So the students were from all different backgrounds, but in order to solve these often environmental problems or engineering problems, they had to learn programs or they had to learn about policies or something, which in a traditional classroom would have been taught first with some kind of vague allusion to application. But here they began with the application and then they learned the tools that they needed in order to solve this problem. And they learned them well. The teachers, there were mentors, not actual teachers. The mentors were available to help them develop these skills, and the students were really motivated. So, to the extent that we can include any of that in our classes or modules, the more students can really care about authentic questions, authentic problems, and authentic outcomes, the more likely it is that their learning will be genuine and lasting and meaningful and enjoyable. It also helps develop independence of mind as students have to generate their own queries, and it also helps them develop confidence in themselves as learners. And these are all things that we want in the people who leave our institution. We want them to go and know how to ask questions, know how to solve problems. And the more we answer everything before they even ask, the less likely it is that that will happen. So, in my own courses, for instance, I often have students generating questions, which can then be answered by things that some people know. Sometimes we don’t know the answer to. That’s actually what leads to the generation of new knowledge. But there are a lot of things that every faculty member can do where you really have students pique their curiosity before you answer them and tell them how it works.

Miranda Prynne: That makes perfect sense. Giving people the reasons to learn. And I think the key word that you said was meaningful, giving the learning a meaning provides that intrinsic motivation. Everything you’ve described there sounds perfectly possible to do within universities. You’ve talked about student choice, allowing students to direct their learning, to apply the learning and situating it in the real world, and generally enabling them to focus on the things they care about. Do you see more of it happening?

Susan D. Blum: There are several different trends in higher education. Some of them are a kind of increase in depersonalization, an increase in mechanics. And with huge scale, it’s going to be harder to engage meaningfully. Some people are looking at AI or generative AI as a solution to personalizing learning, which may or may not be possible. But I think at the same time, there is so much research on active learning and flipped classrooms and engagement and genuine relationships in the classroom and differentiation. And there is so much known about really how people learn. And many, many people are using these sorts of well-documented techniques in their classrooms.

Miranda Prynne: Yes, you mentioned AI there, and it’s almost impossible to talk about higher education now without that coming up. I just wanted to hark back to your book from 2009, My Word!, because that looked at issues around plagiarism in US universities. There was a kind of growing voice of concern that more and more cheating was taking place. Now, since then, digital tools and technology, the advance of that has only accelerated. So, how much do you think has really changed in terms of the core issues that you raised in that book?

Susan D. Blum: Yes, I think that book really kind of holds up. I remember my argument quite well. And there were three dimensions of plagiarism that I talk about in the book. One is the fact that students often don’t really know how to cite properly. And so that’s a matter of education. And that’s always an issue no matter what we’re doing. The second issue is the idea of honoring someone as an author. And that is a moral question which is not really relevant in the context of AI because there is no author. I mean, it is a question of intellectual property, whose intellectual property is being mined, but that’s not usually what people are talking about. So one of the things that most people really had not spent a lot of time thinking about is what in Europe they call the moral rights of an author to be known. So plagiarism is the use of someone else’s words or ideas without attribution. So we’re allowed to use them, we just are supposed to attribute them because we see the value in someone’s original product and we want to give them credit. But the third dimension of plagiarism is where plagiarism and cheating are seen as ways of accomplishing a goal by unauthorized, unallowed methods. So it’s not that you forget to put the quotation marks around a quotation because you’re lazy. It’s that you’re trying to get away with something. And that has been a dimension of education for a very long time. And there are a lot of different methods. So the latest method is using AI to write a paper, to answer questions. And what happens with that is that the focus is on producing a product that will satisfy an assignment without actually doing the learning. And if what you care about is the learning, then this is cheating yourself. I saw a quote, possibly was John Warner. He has a brand new book, which I highly recommend for everybody. I got it in the mail just yesterday. It’s called More Than Words. And the subtitle is How to Think About Writing in the Age of AI. And several people have talked about the fact that if you are not engaging with your writing, you’re not thinking. Writing is not just a matter of producing a product that satisfies requirements, but writing is a way of grappling with what you’re learning and coming out different at the other end. So, if we’re outsourcing that, then the learning is not happening. But to the extent that the student goal is the credential, then having AI write their paper is an effective and very efficient way of making that happen.

Miranda Prynne: Taking in everything we’ve discussed and the need that you point to to inject joy back into learning, where does assessment fit in all this?

Susan D. Blum: Well, assessment is usually looming in a kind of dreadful way over all of our students. And they often see and they tell me that they see the point of education is getting good grades. And if that is the point of education, then any method will work. And cheating will work, learning how to butter up a faculty member will work, extra credit will work, even if the extra credit has nothing to do with learning the material. So, to the extent that assessment is the core goal of education, we have everything backward, I think, and it is a formula for not just joylessness but alienation. So, if we can flip it and put assessment in the service of learning, then there is a role for assessment. If you are learning to play an instrument, you may get feedback. You should get feedback all the time. You can get feedback by yourself. You can say, Wow, I didn’t do that fast enough, and then you can practice and practice and do it faster. That’s a kind of assessment. It doesn’t have to be a number or a letter that’s final. If you believe that our mission, as I believe, is to help every student learn well, then our job is to facilitate that through multiple forms of feedback. Institutions have also charged us with evaluating for the purpose of comparison and ranking and sorting. And that is a very different task than helping our learners learn. And so, those things have become conflated so that it looks as if the entire purpose is to arrive at this final summative assessment. But assessment, in and of itself could be in the service of learning. So sometimes we distinguish learning for assessment, which is how students often think about it, and assessment for learning, which is how I personally think about it. So, grades, you know, many of us have really questioned the centrality of grades and have proposed alternative models, some of which are standards-based grading, contract grading, labor-based grading, specifications grading, portfolio grading, collaborative grading. There are a whole bunch of different alternative systems. Sometimes these are unified under the umbrella of ungrading and sometimes under the umbrella of alternative assessment. But many, many people are questioning now the centrality of conventional grading, especially because, at least in the US system where there are many, many examinations, the intermediate examinations are often averaged together so that students are penalized for beginning without the knowledge that they get by the end.

Miranda Prynne: You’ve mentioned ungrading, as you say, this is definitely an area of growing interest among many in academia. However, even if you don’t particularly want to offer a grade, most institutions would say you have to. So you’ve mentioned some of those alternatives to what we would consider traditional grading. How might some of those work in the current system in which most of your colleagues would be working?

Susan D. Blum: Yes, so different people do it differently, but it may be that students complete work and it’s just marked complete or incomplete, and each small assignment may not be given a grade in and of itself, but there may be feedback. Because we know, research has shown for many decades that if we provide a grade and comments, most of our students will disregard the comments and focus only on the grade. And a grade of B plus isn’t very informative. So it’s much better to give comments and have the students really look at the comments, perhaps not too many comments. I had a student back in the day when I was a strict grader who came to see me who said, why did I get a C plus on this paper? And I said, what did you think of my comments? And she said, I never read the comments. So that taught me something. I also talked to a colleague several years ago who told me that for each student paper, she spent an hour talking about every single thing that the student needed to improve on. And this faculty member was so frustrated because the students kept making the same mistakes. But of course, if they’re not reading the comments, if they’re not engaging with them because they really only care about the grade, then it makes sense. So it seems to me that what we need is opportunities for feedback leading to revision, either on that assignment or on an adjacent assignment, so they can make use of that feedback.

Miranda Prynne: So what you’re proposing is that we lose the number or the letter, whatever it is that is used in the metric, and focus on providing a verbal descriptive feedback system.

Susan D. Blum: Yes, because any metric that is simple, whether it’s a letter or a number, can’t be very informative. It can only average many, many, many factors. So we can have a really creative product that maybe has some grammatical infelicities, and so it can’t be perfect, but you want to celebrate the creativity and you want to make suggestions about how the grammatical flaws can be fixed, and that can be conveyed verbally much more effectively than just by giving some sort of metric.

Miranda Prynne: I guess the challenge that ultimately all educators will come up against is the demand by institutions that there is a final grade at the end of any given program.

Susan D. Blum: To be precise, though, there are institutions, smaller ones usually, that don’t use grades even at the end. So, what they have is a portfolio, they have narratives, they have portfolios of the work, which can then demonstrate what the students are able to do. It’s much more useful for either employers or subsequent levels of education, but it takes more effort. It takes effort on the part of the student, the institution sending the student forth. So, of course, metrics are much simpler to use, but they actually, if you’ve ever talked to employers, they are often frustrated at the kind of information they get from even high-achieving students who have high grades, but maybe aren’t able to function independently or something. But you don’t necessarily get that from the grade.

Miranda Prynne: Yes, I mean, a grade is an incredibly blunt instrument. That’s interesting that some places are doing this because it shows it is possible with a slight change of focus and process. Now, I just wanted to finish by asking you to offer some practical tips to lecturers or tutors who may be struggling to balance the demands of their educational system against their student needs. Can you offer some quickfire tips on how they can just carve a path to better learning?

Susan D. Blum: There are no magic tricks, but I think if we honor our students and give them respect as learners and people who can actually be in charge of their own learning, and rather than being sort of in opposition to them, then I think a lot of other things fall into place. So, instead of trying to ensure that they’re not tricking us, help make everything we’re doing inherently meaningful and worthwhile in and of itself.

Miranda Prynne: Yeah, it’s empowering the students.

Susan D. Blum: Well, they’re the learners. We can’t learn for them. Only the learner can learn. And so it’s essential to keep that in mind.

Miranda Prynne: Absolutely. Look, Susan, it’s been a delight talking to you. And thank you so much for your time.

Susan D. Blum: Thank you so much, Miranda, for having me on your show. I look forward to talking to you again someday.

Miranda Prynne: So it’s pretty clear that Susan feels that the current dominant models of assessment and the associated grading systems work against learning. But there’s a huge amount of great advice in that interview for anyone teaching on how to shift their approach in order to pique students’ curiosity and give their learning a greater meaning. Now our next guest takes perhaps less dim view of assessment and grades as you’ll hear. While Catherine Wehlburg acknowledges the problem of students focusing on grades over feedback, she also does see assessment as an incredibly useful vehicle for informing and improving teaching. So let’s hear more from Catherine.

Miranda Prynne: Hi Catherine, great to have you with us on the Campus podcast today.

Catherine Wehlburg: Well thank you. I am looking forward to our conversation.

Miranda Prynne: Great. Well you have a huge amount of expertise across all of teaching and learning but a specific interest in assessment. You are the president of the Association for the Assessment of Learning in Higher Education, and it’s an area where I know you’ve researched, you’ve published. So, I wanted to start by asking why, over the course of your career in higher education, you’ve become so interested in assessment specifically.

Catherine Wehlburg: Well, so my background is in educational psychology. I got my PhD in educational psychology from the University of Florida back decades ago now, which is scary to say, but true. And one of the things that I talked about quite a bit in my graduate studies and beyond was how we measure learning and how people learn best and how we can use data on what students are learning to improve our teaching. And I spent a long time as the director of a teaching and learning center at Texas Christian University. And one of the things that I found there was that when we talked about teaching, we talked about how to teach better, but we rarely talked about what students are actually learning. And so we had kind of almost divorced the idea of measuring student learning and looking at how we make decisions about good teaching. And I’ve always seen them as kind of two sides of the coin. We can certainly talk about teaching, but teaching without talking about learning is, you know, rings a bit hollow. And so looking at assessment and how we measure learning and how we use that to improve has always been something that just seems to make a whole lot of sense to me. And it’s certainly something that in higher education we can do a better job.

Miranda Prynne: You are interested in assessment from the point of view of what it can tell you about teaching. But it does have to serve a number of purposes. So how do you balance that need for assessment to inform the teaching against the other functions it has to serve, like assessing the students’ learning and the progress that they’re making, and of course, universities demonstrating their performance to accreditors, for instance?

Catherine Wehlburg: Right. And so it gets all complicated and we tend to sometimes lump these things together as though they are identical and they aren’t. So I like to think of it as levels of assessment. And there have been several people who have written about levels of assessment, but one of the first levels is assessing the student, whether that involves feedback directly to students, grading in their courses. That type of assessing learning is something that in higher education we do on a daily basis. Now, can we do it better? Can we do it in ways that make more sense? Can we do it in ways that are more formative? And the answer is usually yes. Another level is at the course level and the program level, where we’re looking across student assignments, sometimes across assignments from different courses, where we can really look and see: are the students learning what we want them to learn across that, where we’re actually giving feedback to the program faculty. And to me, that’s the core of what I consider to be assessment. And, you know, maybe that’s capital A assess, that we are providing information that will inform decision making about curriculum changes, types of teaching, ways of teaching, approaches to all of that. And so there’s that. And then a third level is really thinking about the assessment that we do and providing data and information to our public stakeholders through the accreditation process or through information that’s required in the U.S. It’s at the state and federal level. And so that is more compliance. And we can use the same data points at all levels. It’s just using them a little bit differently. And there are different purposes for that. And one of the things that I try to do is make a distinction between assessment at the program level, assessment of learning and of improvement, and accreditation. And with accreditation, it’s really more about compliance in many instances, which is unfortunate because it means we’re trying to show that we are doing something well, even though we really should be using those data to make good decisions and to improve. And if we’re doing everything well, then there’s no need to improve. And I think that one of the problems that we’re seeing in the world of assessment and assessment of learning is that it so often gets focused on that third level, where what we’re trying to do is prove that we are compliant, prove that we are good enough. So that means that we’re really missing an opportunity to take the work that we’re doing to look at student learning and use that as an improvement rather than as a compliance piece.

Miranda Prynne: Yes, I can see there’s a conflict between the desire to demonstrate that you’re doing everything as you need to be and the need to be really transparent and honest and actually look for where there are some weaknesses in order to keep improving. And I can see that’s a difficult conflict to bridge. What trends or evolutions in ways of assessing have you noticed in the last few years? Have there been any notable changes of direction?

Catherine Wehlburg: One of the trends that I’m seeing that can be very helpful is some of the uses of technology where we can use digital portfolios. And so, data are able to be rolled up from a course level to a program level and beyond in some very specific ways. And so, for example, if a faculty member is creating an exam, they can actually link exam questions to program-level outcomes so that the student is still getting feedback on what they submit, but individual items can actually provide course-level or program-level data across students.

Miranda Prynne: Yes, and presumably, it’s affecting the way you assess as well, the method of assessment. So, you’ve mentioned digital portfolios. This brings me on nicely to authentic assessment, which is a term bandied around a lot, but it seems to have quite a broad definition. I wondered if you could talk me through how you define authentic assessment and why it’s important.

Catherine Wehlburg: Sure, and I think that authentic assessment, and you’re right, it’s used in a lot of different ways by a lot of different people. But the broad way that I typically define it is that it is some type of a demonstration of an application of knowledge or an application of a skill to something that would be more of a kind of a real world task. And that is different from answering questions on a multiple choice test or even an essay exam in some instances. So when I think about authentic assessment, what I mean is what is the student learning how to do and can they do it? And so if we want students to be able to write, an authentic way to assess that is to look at their writing. If we want them to be able to weld something, can they go and weld a particular item as part of their project? So it is not just being able to talk about something, it’s being able to show it and do it, whatever that it happens to be.

Miranda Prynne: Which makes sense given that the students that are being assessed generally are taking these courses and undertaking the learning in order to go out into the world and use these skills. Do you feel it’s a more equitable way to assess?

Catherine Wehlburg: It can be if it’s done well, but for me, it’s less of a question about equity and more of a question about what are we trying to measure. So for example, in the U.S., students who are going to be teachers at our K-12 schools, they have to go through multiple levels to show that they have the competencies that that are needed, which is, which, which is great. Some of them are tests where they have to sit down and actually answer correct questions, and that is not really an authentic assessment, but we do want our teachers to have a base knowledge in whatever field they’re going to to do that in. They also have to demonstrate in the classroom that they are able to present a lesson plan. They have to write a lesson plan and present that lesson plan and then they are observed on that. And that is certainly an authentic assessment. But I’ve seen and I think we all have seen instances where the rubric that was used to rate that demonstration may have been questionable or it was too short or it was something happened that day and that particular teacher wasn’t at his or her best. So is it equitable? I think is a question that we have to look at on a on a case-by-case basis because I think authentic assessment can be very, very useful and it’s something that we need to have, but it is also more difficult to do and it’s more difficult to scale. If I’m teaching a very large class, doing an authentic assessment is going to be much more complicated than if I’m teaching a very small class.

Miranda Prynne: You mentioned rubrics. It strikes me that if it’s not a written test or a multiple choice or a more traditional form of assessment, there is potentially more room for the grading to be subjective. What is the answer to that?

Catherine Wehlburg: Well, and yes. And anytime there’s always going to be that subjectiveness that’s added into something. And as you get into more of the authentic assessment measures and with different types of rubrics, it’s always going to have that level of subjectivity. And so I think that what we really have to do is keep looking at the validity of the way we’re measuring that learning and make sure that we’re looking at something that has a valid theoretical framework behind it. And whenever possible, have multiple measures. If we can have 10 smaller quizzes, that’s probably going to be a better measure, depending on the questions, obviously, than one single final exam. And anytime we’re measuring learning, there’s always going to be some level of error in that, where a student didn’t understand the assignment or something happened. And the more we can look at and recognize that there is error in there, there’s no perfect way to measure learning, then we’re going to be able to be a little bit more holistic as we look across measures of learning. So, I think part of what I’m really getting at here is that when we get into really high-stakes assessment, that brings with it more difficulties than if we’re looking at several different ways to triangulate around a particular identified learning goal.

Miranda Prynne: Yeah, so higher stakes brings higher risks because so much is riding on this one approach. Whereas, if you’re offering a number of different ways that students can demonstrate their knowledge, that gives you a much more useful overall picture.

Catherine Wehlburg: Yeah, I think so. And it means that we have to, as educators, we need to look at things across a course or across a body of knowledge rather than just look at a single assignment.

Miranda Prynne: Now, I’ve seen research cited which suggests that grading can have quite a negative impact on learning. And I wonder what your experience of this was during your extensive higher education and teaching career.

Catherine Wehlburg: You know, I think that anytime you’re giving feedback, that feedback can be taken in multiple ways. And some of that depends on the shape and the type of feedback. But when a learner does something and they get feedback, if they feel that it is constructive and and they feel that it is appropriate and valid, they will take that very differently than feedback that is is given and the the learner feels that it was unfair or that it was too harsh or that it was too easy. And so I think that as we look at that grading piece and how we provide feedback to students, the shape of that feedback, the timing of the feedback all play a role in how the learner is going to use that. And when we give feedback and we give grades, it is to provide information to the student. And if a student does something and I say, oh, you got an A and they think, well, I really didn’t work very hard on that, they are probably not going to take that feedback in serious ways. And the opposite is also true if they feel it was harsh or unfair. In a similar way, if the feedback doesn’t come for a very, very long time, there isn’t that option for using that feedback to really enhance learning, which is the point of feedback to students. There’s a wonderful study that was done years and years ago in psychology where individuals had to draw a picture, but they were not able to see what they were drawing. They were blocked, so they couldn’t see the results of that. And their drawing never got better. But when they were able to do the drawing and then see what they drew, their drawings got better. And so feedback is such an important piece of it, but that feedback needs to be relatively immediate and it needs to be very focused on what the student was able to do. And if the assignment is the kind where, okay, this is it, you turn it in, you’re done, often students don’t even read feedback. So on a final papers, faculty and educators will spend a lot of time writing comments. But if it’s a final paper, some of the research shows us that students aren’t actually reading that feedback because they’re seeing it as summative and not formative.

Miranda Prynne: Is there perhaps a job then for educators to reframe how grades are perceived? Because in the US, in the UK, similarly, we are graded from a very early age, and these grades then do define the opportunities that are afforded and which institution, for instance, we might be able to study at, which job we might later be able to apply for. However, as you say, it’s also incredibly important that people are able to see how they’re progressing and learn from it. Therefore, is there a job in reframing the purpose of grades to be as an assist to the students rather than as a measure of them?

Catherine Wehlburg: Oh, I think you, yes, you’ve opened up a huge question here, which I think is really important because, you know, when we look at report cards or grade cards that students get from early ages, as you say, and we see that getting an A grade or a top grade is better than not getting a top grade. And so we end up using the grade kind of as a definition of how well somebody, how good somebody is and how much they have learned, when in reality, grades can sometimes mask individual skills that need to be highlighted. So for example, if a student turns a paper in late and the instructor takes off points for it, the student may get a lower grade, but it may have been an incredibly good paper. And so taking that kind of scenario, the grade is not really representative of the learning that the student is demonstrating. And so I think we need to be really careful as we look at grading as a way of giving feedback to students to make sure that we’re giving the feedback we intend. But I have found that in many instances, there are other elements imposed on that, you know, whether that’s a timeframe, you didn’t use the right cover page. Grades are not always a measure of learning. They can be, but they are not always. And so that’s one of the reasons why I think rubrics can be very helpful, because yes, you can have a score from the rubric, but if the rubric has multiple parts to it, a student can go back and look and see, okay, I didn’t do my APA style right, but my concepts were really good and my writing style was really good. And so that can help them to focus in on what feedback they are supposed to be getting from this particular grade.

Miranda Prynne: So grades are useful as long as they are delivered alongside constructive feedback and as part of it.

Catherine Wehlburg: Right. And I see in the US, I work a lot with accreditation and I see sometimes faculty or programs will say, well, we want all the students to get at least a B in this course. And my pushback on that is, you know, what does a B really mean? Does that mean that they have learned the material? So for example, I taught for a long time intro to psychology. And at the end of the semester, I’d give, you know, however many A’s, B’s, C’s, D’s, F’s. And if two years later, somebody came back and wanted to know about a student who had gotten a B in the course, well, how much psychology do they know? I would have to go back and look at the grade book and see, well, on the second exam, they did really well, or on the third exam, they did really badly, or, you know, whatever. And then I’d have to go in and look at that exam to see what knowledge was that. So the grade itself doesn’t really tell us what was learned.

Miranda Prynne: And I suppose because assessments, by the nature of grades, often do define level of student success, they are quite charged from the student’s perspective. But assessment seems to just be a very charged subject. It’s one that people get really passionate about and you see really, you know, robust discussions about how it should be done, how shouldn’t it be done? Do you think that’s fundamentally why? Because there’s so much riding on these?

Catherine Wehlburg: You know, and I think that is part of it. I also think that when we talk about assessment, it is sometimes seen as an evaluation of the faculty or of the teaching, not of the student. And I think that has been part of the reason why it has not always been embraced by faculty who may use it very, very well when they’re giving feedback to students. But when a faculty member has to turn around and say, okay, well, across the board, my students didn’t really learn this concept, that then is sometimes seen as an evaluation of the teaching. And that can cause distrust in the system. You know, it’s one thing for me as a faculty member to give feedback to my students, but I don’t necessarily want that feedback to be turned around onto me. So I think that’s one of the issues with assessment being used at the program level and for compliance is that we really start thinking about: are we doing as good of a job as we could be doing? And the answer is usually no, we’re not. We can always do better. But that is a hard conversation to have.

Miranda Prynne: Yeah, that comes back to the conflict mentioned earlier, or the innate tension between open and honest assessment and wanting to learn from it, but also wanting to exhibit a positive and successful performance. What innovations in assessment have you introduced to Athens State?

Catherine Wehlburg: So, one of the things that I think about when we look at assessment in higher education, you know, when I look at myself as a faculty member, I want to be able to be seen as somebody who is competent and that my teaching is good. And that if a student isn’t learning, how much of that is my fault and how much of that is their fault? And so, there’s lots of things and emotions that go into when we look at this. And so, one of the things that I have tried to do, not just at Athens State, but at other places, is to talk about assessment as a way of looking for areas that are not meeting expectations. And that that’s a good thing. If we can find areas where we’re not doing as well as we could, that gives us a focus. That gives us a place where if we invest resources and time, we’re going to be able to make improvements. Rather than saying, oh, everything is great. Everything is fine. I meet all the levels of compliance. And so go away and leave me alone. Instead, being able to say, let’s find something where we’re not meeting expectations. And that means sometimes that we have to set the assessment bar a little bit higher. You know, what is our aspirational goal for student learning? And no, not every student is going to meet that. But if we find an area that students are not learning as well as we want them to, then that gives us something to focus on. Do we need to change the way we’re teaching? Do we need to add something? Do we need to change the order of the curriculum or add a new section? So it’s the idea that compliance is not the point, even though many accreditors will say that you need to be in compliance with standards. And yes, I think we do need to be in compliance with standards, but there’s a bigger issue, and that is focused on improvement. So one of the things that I’ve tried to do is to look at assessment of student learning as not a way to show that we’re good enough, but as a way to find where we are not good enough and that we celebrate finding that so that we can improve on it. And, you know, in these days in higher education where everyone has limited resources, if we can focus those resources on an area of improvement and make that improvement, then we are doing a much better job for our students and for our community and really in meeting our mission as an institution.

Miranda Prynne: So again, it comes back to this reframing of using assessment as a tool for improvement and informed decisions. Have you had any challenges bringing your faculty along on this journey?

Catherine Wehlburg: Yes. And because, you know, we all want to look good. No one, there’s almost no one who wants to say, oh, I’m not doing this as well as I should be. And so it is challenging to say, yes, you know, in what areas do you still need to do work? It’s a lot nicer to say, nothing. I’m perfect. I’m doing everything right. And so it can be very intimidating to look at our data on learning in our program and say, okay, well, this really hasn’t been as good as we want it. So one of the things I’ve tried to do is see where we can tie this to budget. For example, in the traditional way that we often at the university level do budgets, if there’s a program that is failing, we tend to take money away from it. And I think that is the opposite of what we want to do. What we want to do is say, okay, if you can show me that you’re not doing as well as you want to in this particular area, and you have a plan for what you need to do. And if we had a different lab or a different equipment or something additional, we could really make an impact here. If I can use that as a way to budget resources so that we get improvement, then we’ve made a difference. That is a great use of resources.

Miranda Prynne: So it’s providing incentive to take this much more practical approach, looking at ways to improve.

Catherine Wehlburg: Exactly.

Miranda Prynne: It’s a fantastic way of flipping it because it’s counterintuitive that you would remove support for something that needs more attention. That’s a very practical thing that could really change the way assessment is perceived.

Catherine Wehlburg: It does help to kind of reinforce that looking for improvement rather than looking for compliance.

Miranda Prynne: Exactly. I can see that sits at the core of it. Catherine, it’s been great talking to you. I wish we had longer, but thank you so much for joining us and sharing your thoughts on all things assessment.

Miranda Prynne: It’s interesting to hear from Catherine, who as president of an institution has to consider important administrative issues around compliance, but also needs to look for ways to inform institutional strategy, to direct budget and resources. So while she is a realist and clearly aware of the failings of certain types of assessment and grading, she can also see a lot of useful and practical applications for measuring the effectiveness of teaching and thus learning. Now, our final guest, Josh Eyler, could be labelled as a poster boy for ungrading, as his latest book really delves into what he terms the harms caused by grades. But really, Josh’s mission is to improve learning outcomes for all students. And with many years of teaching experience within the US system, he is well aware of the practical realities of this job. So, he offers stark insights, but also a lot of practical tips on what can be done differently.

Miranda Prynne: Hi, Josh. Great to talk to you today. How are you doing?

Josh Eyler: Good, I’m Miranda. Thanks for the invitation.

Miranda Prynne: Well, it’s lovely to have you. Now, we are discussing assessment with a focus on grades because you had a book out last year in 2024 called Failing Our Future: How Grades Harm Students and What Can We Do About It. So, can you summarize for us how do grades cause harm? Perhaps you could talk us through the key elements of this.

Josh Eyler: Certainly. I think the key point right off the bat is that grades cause a number of different kinds of harms. And so, depending on which category we’re talking about, that’s going to shape the nature of the conversation and also the potential solution. So, one bucket of those harms has to do with the effect of grades on learning. Grades stand as obstacles to intrinsic motivation and deep sustained learning because grades, as extrinsic motivators, largely are useful in getting people to comply with things they would rather not do. So, yes, grades can get students in seats. It’ll get them to turn things in on time. It will even get them to speak in class sometimes. But just because they are in that environment, in no way can grades guarantee that a student learns. That’s driven by interest, curiosity, intrinsic motivations. But there are a couple of other ways that they affect learning as well. One is that grades push back on the natural or they stand in opposition to our natural process for learning something. The way human beings learn anything is to try something out, make a mistake, get feedback from someone who knows more about that area, and then try again. Grades arrest that process at the trying, so that we are never able to really take a kind of intellectual risk, or we are dissuaded from taking intellectual risks because the penalty, the potential penalty of grades, is too high. So that’s problematic for the kinds of learning that we’re interested in. And then just students’ general approach to new topics and new assignments, new ideas. Grades create a very utilitarian philosophy for the classroom or approach to the classroom where students are not driven by trying to pursue questions that interest them, but instead they are just playing the game of school. They’re trying to get the grades that will get them to the next step, whatever that next step is. But that’s just the academic harms. We also have research to show that grades are contributing to the mental health crisis among teens and young adults, largely due to the academic stress that is tied to grades. And that stress has been noted as being a contributing factor to especially depression and anxiety among those populations of students. Grades also exacerbate inequalities in educational systems, that students who have fewer opportunities in their K-12 schooling, that opportunity gap, we call that, is reinforced and reflected by the grades they often get in the first and second year of college. So that’s a lot of different harms. I think when you add it all up into an equation, it doesn’t speak highly of grades. In fact, it points us toward making efforts to reform the ways that we grade in higher education.

Miranda Prynne: So, really, the three key pools is the harm to learning or the academic process, harms mental health, and reinforcing inequalities. Everything you’ve described there, for instance, taking the harm in really working against the learning process as something where one has to trial, to test things, to make errors and mistakes, to learn from them. I think anyone who has been through the school system, university system in this country, in the US, and in many others, that will really resonate. Given all of that, why on earth in the US, the UK, and indeed many other countries, education systems, why are they so addicted to grades?

Josh Eyler: That’s a fantastic question. I think there are a couple of answers to that. One is grades are easy methods of communication. That’s why they were developed in the first place. Or that’s why the systems that we have now were adopted in the first place, that they were easy methods of communicating across institutions and to different constituents. Not because A, B, C, D, and F were somehow the best reflectors of student learning, but because it was assumed that everyone would understand what A, B, C, D, and F meant. So they were easy. That’s one. Another one is the tendency toward inertia in institutions of higher education, that grades are seen as so much a part of the foundation of what happens in higher education that for institutions that have a hard time changing anything at all, it is rarely taken up as something that even needs to be considered to be changed. So inertia really works against us. And the other reason that I would mention here is that there is a mythology about grades that has been crafted over time, that they are somehow scientific, objective measurements of learning, and therefore necessary in higher education to be able to certify and signal the progress of students, both within the institution and for things that students will apply to after higher education. The problem with that is we have research now to show that, and honestly, this is intuitive as well. Grades have never been objective and they have never been scientific indicators of progress. Grades are at best subjective indicators of a student’s progress on an individual instructor’s learning goals. They are not universal certifications of knowledge. They are not consistent across sections within departments. They are simply markers of progress in an individual course. And that is all they are. And that is all they should be taken for. But that mythology about grades has been longstanding because it feeds into our need for objectivity and the ability to say or to convey to others that the work that is happening in higher education has this stamp of approval and that stamp is grades.

Miranda Prynne: I’m glad you’ve pointed to the mythology because the PR line that some might turn to when it comes to grades is saying, Well, yes, they put pressure on students, and yes, of course, they are quite a blunt instrument for presenting the kind of wide, rich breadth of student achievement. However, they are an opportunity for students to demonstrate their achievement. Do you come up against that argument much?

Josh Eyler: Oh, all the time.

Miranda Prynne: And what’s your response to that?

Josh Eyler: One response is that pressure and stress need not be necessary components of learning or of education. They just happen to be because of grades. That, yes, we want students to be challenged. We want them to feel that to kind of wrestle with difficult issues and topics and work, but it’s grades that really ramps up that pressure and stress. And it’s not a necessary component of learning. The other thing, though, when people bring this line of argument into the conversation that they are necessary because they are markers of progress, what I say to that is that if you look at syllabi, you look at the work happening in classrooms across higher education, my syllabus for a course and the goals that I set and the assessments that I design and the assignments and activities I give are different from my colleagues who is teaching the same exact course in the classroom next to me. And if that is true, which we all, I think, can agree that it is in almost all cases, except for those rare courses that have standard syllabi and things like that, then a grade that I give cannot be seen as being the same marker as the grade that my colleagues give. And until we really reckon with this truth, I think we’re going to keep going around in circles about this. But the simple reality is that grades are not those advancements of progress that higher education has told us they are.

Miranda Prynne: Let’s step back because this is something you’ve been thinking about and writing about for some time. What first led you down this road to assess grades themselves and by that same token, assess assessment and how it’s done?

Josh Eyler: My origin story for that is pretty complex, actually. Part of it is my own teaching, being dissatisfied with the types of grades or the process of grading for my own students. And so, you know, after grading hundreds of essays, I got to a point where, you know, wrestling with, is this an A minus or a B plus, was just not certainly not fulfilling for me and not conveying, not supporting the learning for my students the way I wanted it to. Because what I found was the feedback that I gave on those papers was often in defense of the grade that I had given. That if I gave it a B plus, part of my feedback was defending as if I were in court why I had given that B plus. And that just wasn’t why I became an educator. I wanted to be more of a coach for student learning. So that drove me to the community within higher education and even K-12 in the US as well that was experimenting with alternative models for grading. So that was part of it. Another part of it was becoming a parent and seeing my daughter talking incessantly about grades, even from a very young age, five or six, and thinking something is not right here. And it starts so early and kind of connecting those two things together. And then the third piece of this was that community of reform in higher education that I was seeing so much, especially over the last 10 years, about the problems with grades, about people trying out these different models. And that was really inspiring. And I thought, this is work I want to be a part of. So it’s really those three things together that pushed me in this direction.

Miranda Prynne: It does feel like there is maybe the beginnings of an upswell against this. The growth mindset becoming a much greater focus and this idea of embracing failure as a process of learning. But that doesn’t answer the issue of the fact that throughout a child and then teenager and young adults’ life, they are constantly subject to grades. So what can be done on a practical level to address this? Because there is a growing awareness that it’s a problem, undoubtedly. There’s a lot of research now to point to all the harms that you listed. So can you point to any examples of where you or other educators have successfully found alternative routes that work within the realities of the system in which we’re functioning?

Josh Eyler: Absolutely. And without diving too deeply into the weeds, if you look at the history of the last 100 years or so of education, you find several periods of grading reform, some of which were more successful than others. But right now, I’d say from 2014 onward, we are in perhaps the most fruitful period of grading reform that at least American educational history has ever seen. The reason for that, technology and the ability to communicate in ways that were not possible during other moments of reform, like the 1960s, for example. So, technology, social media, the ability to communicate, where educators who are trying experimental approaches were able to find each other and build on each other and build resources and try new things out. So, yes, a lot of good things are happening, largely at the moment in individual classrooms. So, people trying out things like standards-based grading, contract grading, portfolio grading, ungrading, all of these things are being tried in classrooms. There are a few institutions who are responding, I think, to the growing research and really reflecting on grades. For example, you know, I think one point of reform that I think is really important to mention is that during COVID, when institutions closed down and went to remote learning, many, many institutions in the United States changed their grading model overnight. Some went to pass-fail, some went to a system where students could cover their grade if they got a low grade in a course. So it is possible. Institutions know that they can do it, but again, that inertia and tradition often prevents it from happening. But that said, UC San Diego, University of California, San Diego, they started a new writing program that was all collaboratively graded. And so no grades throughout the semester, just feedback. And then students propose their final grade, and that is discussed with the faculty member. The whole writing program for first-year students is designed with that model. There are other institutions that are changing the range of grades that are possible to give students. So there was a university in Oregon that dropped D minus and F from the possible grades to be given as an acknowledgement of some of these sorts of things that we’ve been talking about, the role of grades in stress and mental health. So, there are things happening institutionally that we can point to, but I see the biggest groundswell and it is growing and growing. There is an online conference that has been happening the last five years, for example, called the Grading Conference. Last year, a thousand people attended that conference. And so, this is a movement that is growing more and more every year.

Miranda Prynne: Yes, technology in all its forms, but most recently with the proliferation of generative AI, is really forcing a complete rethink of assessment across the board. You touched on a few of the alternative modes of grading there. For those who are maybe less engaged with this topic, can you take us through how some of them work?

Josh Eyler: I’m writing a new book now with a colleague that is designed to give people overviews of all of these different models. So, I’m happy to do that. Standards-based grading, I have found is most popular with STEM faculty. So, with standards-based grading, the faculty member sets out a number of content and skill-related standards for the course. The student’s grade is calculated based on how many of the standards they have met by the end of the term, rather than what do they get on individual assignments. So, let’s say you’re teaching introduction to biology. You, as a faculty member, set, I’m just going to pick a number, 20 content and skill related standards. The student’s grade is based on how many of those they meet.

Miranda Prynne: What would a standard look like in introduction to biology?

Josh Eyler: Well, one standard might be understanding the chemical process of photosynthesis. That might be a content-based standard. A student could demonstrate mastery of that standard through exam questions, through homework, lots of different ways to meet it, but they check that off once they have met it. A skill-based standard might have something to do with their work in a lab, but then their grade is determined by how many of those standards they have met. So an A might be meeting 18 or 19 of those standards, a B, 16 or 17, something like that. Contract grading is a little bit different. It was developed in the 70s. The basic model of a contract grade is that there are a menu of assignments and activities for a course that have been carefully and intentionally designed by the instructor so that if a student completes them at a satisfactory level, they can be said to have met some of the learning goals for the course. And the grade is determined by how many of the menu items they complete at a satisfactory level. It’s called contract grading because in the original model, the students would say, I’m going to contract for an A. So I’m going to do this many of the on the menu of assignments. And they signed their name as if they were signing a contract. So I’m contracting for an A. I’m signing that. I’m committing to it. And then if they hit it, they fulfilled their contract. If they didn’t, they were taking the next rung down to whatever was necessary for a B. So that’s contract grading. I’d say maybe the more familiar model, portfolio grading, very common in the arts and humanities, though I’ve seen it in science courses as well, portfolio grading. The idea here being that students are given lots of opportunities to revise assignments and in some cases, exams over the course of the semester based on feedback. And then the end of the semester, they put their best work into a portfolio. They usually write some kind of reflection at the beginning. And the grade for the course is largely the grade given to that portfolio. So that’s more a little bit more traditional, but the idea that students are given so much time during the term to really refine their ideas before they get that grade, I think really connects to what is important and significant about all of these alternative grading models: that they honor and acknowledge that learning is a deeply complex process that unfolds for different students at different rates. And they give multiple ways for students to demonstrate that learning.

Miranda Prynne: Yes, what strikes me and what you’ve described is that it moves the focus towards the process rather than one output or outcome. But ultimately, they do all result in some sort of grade or metric at the end.

Josh Eyler: Yes, because most of our institutions require us to submit grades. So, yes, we have to give a grade, but what we want to focus on during the term is the learning that happens along the way rather than that final output. So, we’re trying to reorient students’ approach to an understanding of what a grade is meant to reflect.

Miranda Prynne: Of course. It’s very hard to talk about US higher education at the moment while separating it from politics with so much going on. So, I wondered how you felt the current sway of US politics might affect assessment reform and this mission to rethink grading and what it stands for.

Josh Eyler: Yeah, you know, I think it’s still early days, and so it’s hard to know for certain what might happen. But I will say that one thing that you notice in the political rhetoric that is coming out right now is an emphasis by conservatives on the perceived meritocracy. And grades are a part of that mythology that they have constructed. In other words, the meritocracy that everyone, given the chance, can succeed. We know socio-politically that structurally and the inequities baked into those structures, it can be much harder for some students to succeed than others. And so grades are a part of that narrative of a meritocracy that if you just work hard enough, you can get the A’s that will get you whatever you need. So in that sense, I think grading reform does have some degree of scrutiny on it. And I know that because there are major conservative publications that review every single book about grades very negatively, my own included. And they point to this notion of getting rid of grades as being some sort of utopia that will damage students in the long run. But another thing that I’ll say is that we have several institutions in America that don’t and have never given grades. Evergreen State University in Washington is one that I talk about in my book, but there’s one in Florida, a small school called New College of Florida. Their board was overtaken by the Republican governor of the state, and they have instituted many, many changes to that college. But one thing that has not yet changed, although many, many things have, is the grade-less policy. So they’ve dismantled quite a lot of that college, but the fact that it doesn’t give grades has remained. And so that to me suggests that grading reform is not at the top of the list, that it is flying, at least right now, a little bit under the radar.

Miranda Prynne: And I imagine that you’re probably hoping it stays that way. I mean, at the risk of getting slightly philosophical, is there not a deeper question about just the nature of capitalism embedding a culture of competition and, I don’t know, scarcity throughout society? And higher education has almost been colluded into that as a tool to underpin this competition because it’s such a deeply embedded, accepted norm. Do you think there’ll always be a bit of a challenge there in winning over the masses on this issue?

Josh Eyler: It’s an important question to ask because what we want to think about is how do we grow these efforts and how do we make them sustainable? And the fact that higher education exists in the capitalist society and often recapitulates and reinforces those capitalistic structures. And as you say, competition is baked into that system and that philosophy. And so how do we advance grading reform if there’s such an emphasis on competition? Will we ever be able to solve that issue? I don’t think so. But I think that what we can do is stress the positive benefits of changing the way that we grade for developing student well-being more holistically, more holistic learning for students and students who come out of higher education as being driven by ideas and by learning for its own sake, more so than what do I get out of this.

Miranda Prynne: Ultimately, it points to the idea that if you can demonstrate students coming out of this system have the skills and the learning required, regardless of grades, then you’ve won that battle in many ways.

Josh Eyler: Yes, and we do know from college career centers that employers are relying much less on GPAs now or caring even much less about GPAs than they are about things like communication skills, critical thinking, because I think there has been a shift amongst many employers to say we can teach people some of the on-the-job things, but if they’re not sound thinkers and good communicators, that’s much harder to be able to teach. So I think that’s important. The last thing I’ll say is that some of these alternative grading models provide students with more information about what they know and can do than just a traditional grading system. So getting a 92 on a biology test doesn’t tell you very much, but being able to say, I understand the chemical process of photosynthesis and here’s what it is, that’s different, right? And that does give the possibility for students graduating from these kinds of classrooms with much more thorough information about what they have learned.

Miranda Prynne: Yeah, absolutely greater self-awareness. Thanks so much for talking to me today, Josh. It’s been fascinating.

Miranda Prynne: Josh makes a very strong case there for rethinking traditional grading systems. But importantly, he offers numerous ways to navigate this grade-hungry system to shift the focus towards learning as an ongoing process. So while all three of our guests have slightly different focuses, there are some universally accepted facts that emerged from these conversations. Notably the fact many students will disregard feedback if it is delivered alongside a final grade and that any assessment should be designed, delivered, marked in the service of learning to be used as a tool to inform and improve.

Miranda Prynne: If you’d like more advice and insight on assessment, we have just pulled together a spotlight guide looking at authentic assessment specifically. But we also have many, many other collections and individual resources on assessment design, feedback methods, ungrading. You will find all of that at www.timeshighereducation.com/campus.

Miranda Prynne: If you’ve enjoyed the podcast, please like and subscribe and I’ll be back with you in a couple of weeks. Thank you. Bye-bye.

Sound clip: You’re listening to a Times Higher Education podcast.