Search News


Browse Archives

News

How to Tell Whether Writing Instruction Works

December 28, 2007

Share This Story

FREE Daily News Alerts

Advertisement

Complaints about students' poor writing skills have prompted many colleges to create new programs or adopt curricular changes, but do these efforts work?

As writing program directors gathered Thursday at the annual meeting of the Modern Language Association, many voiced confidence that their efforts are making a difference. But at one of the kickoff sessions for the meeting, many of these officials said they worried that views of their success were based more on hunches and intuition than solid evidence. That may be changing, however, as composition scholars described a range of projects designed to test the effectiveness of their efforts. Some said they see a shift in composition away from theory and toward more practical research on student learning and instructor strategies.

"For writing centers and programs, the dearth of empirical research is dangerous," said Linda S. Bergmann, director of the Writing Lab at Purdue University. Too much of what writing instructors believe is based on "lore," she said. At a time of political demands for assessment and commercial companies promising quick results if they take over tutoring services, writing instructors need evidence of what works, she said.

The research projects described at the meeting, in Chicago, are generally small scale, involving one or two campuses each. But those conducting them -- and audience members -- said there was a need for more such studies, and for efforts to enlarge and replicate some of those being conducted.

The research efforts included the following topics:

  • How writing tutors and students set up relationships and agendas. Laurel Reinking of Purdue has been studying the conversations (through transcripts) between tutors and those they help in the writing center. Issues related to how students express their needs and accept (or reject) advice are crucial to the success of these tutoring programs, Reinking said. Her hope is to identify ways that tutors can get the information they need about students' needs without just giving in to what students say they want. "The bottom line is: We need to know what makes the agenda-setting part of this relationship work," she said.
  • How peer advising on writing changes student learning. Dara Rossman Regaignon, director of writing at Pomona College, is testing the impact of "writing fellows," two students who are assigned to a course and who review student writing assignments and suggest revisions. Pomona is testing the impact of this approach by conducting surveys of students and professors in similar classes with and without the fellows, and by having outside experts examine portfolios of student writing in classes with and without writing fellows. The early results are encouraging, and suggest that the gut feeling of many that writing fellows help is something that can be backed up, Regaignon said.
  • How teaching assistants teach writing. E. Shelley Reid, director of composition at George Mason University, is exploring which skills teaching assistants are confident of, and which they aren't, after they start teaching. Reid is also doing surveys to see how much TA's use the pedagogy they are given in orientation programs prior to teaching. Among early findings: First-year male TA's have difficulty balancing time demands with responding to student writing. Second and third-year female TA's are more likely to worry about pressure to give students higher grades than they might think are deserved.

Chris Anson, director of the Campus Writing and Speaking Program at North Carolina State University, said that there were many reasons to support such research projects. Politically, he said, writing programs need to be able to defend their programs. But educationally, he said the reality is that research could find flaws in current practice. "We need to be ready to abandon cherished practices if they don’t work," he said.

The projects discussed suggest "a reinvigoration of our research agenda," Anson said, and that could ultimately get to what really matters, he said: Finding out "what really works and what doesn't work."

See all postings »
Advertisement
Advertisement

Matching Jobs

Comments on How to Tell Whether Writing Instruction Works

  • Care with discussing research
  • Posted by Shelley Reid on December 28, 2007 at 10:00am EST
  • One of the points of doing more formal research is to be able to be more specific and accurate about what we claim -- and what we do not claim.

    So it's important to have the reporting on that research be as clear as possible. As I noted at the beginning of my talk, I don't have answers yet to my questions, only a small pool of initial data and a lot of additional questions. So, for instance, I can't and didn't make the claim that

    "First-year male TA’s have difficulty balancing time demands...."

    Instead, what I can and did say is that in one cohort of first year TAs at my university, the male TAs who participated in the first year of my study reported being strongly concerned with how they would balance time demands of responding to student writing when they started teaching for the first time the following year.

    Similarly, I can and did say that second and third year TAs from my school who participated in the first year of my study were more likely than first-year TAs to report higher concerns about making assignment requirements clear to students and responding to pressures to raise grades -- and that within that small sample, women were more likely than men to note those concerns.

    Are these local or personal anomalies? are they more consistent trends? It's tempting to make guesses based on our own experiences or anecdotal evidence, but the fact is that I don't know for sure. That's why my study is designed to run for at least three years at my university, and why I'm expanding to survey TAs at different university sites as well.

    This kind of research into how people learn to do complex tasks like write and teach writing, and more specifically into what kinds of methodologies are most effective at helping the most people learn these tasks better, is necessarily slow; the claims from such research, especially early ones, are necessarily partial; and the need for replicating the research and then aggregating the data is high.

    --Shelley Reid

  • better grad education needed
  • Posted by cga on December 28, 2007 at 12:05pm EST
  • For rhetcomp research to really be invigorated, our grad programs need to be teaching empirical research methods. Many grad students seem adept at discounting empirical research--not many understand even such basics as validity/reliability. Also, our journals need to do a better job of soliciting and accepting such research.

  • Posted by JBM on December 28, 2007 at 12:45pm EST
  • The obvious way to determine whether writing instruction works is to read the writing that it produces. Kids either can or cannot write a correct sentence; they either can or cannot organize ideas in a coherent way.

    It is all too easy to determine whether instruction is effective, or whether it sacrifices essential skills in favor of meaningless theory and research.

  • Status of Research on Writing Instruction
  • Posted by Anne Beaufort , coordinator, writing-across-the-curricululm at University of Washington Tacoma on December 28, 2007 at 12:45pm EST
  • There have been many strides forward in researching the effectiveness of writing instruction in the last 20 years. Evidence:

    1) more Ph.D. programs in composition and rhetoric are emphasizing empirical research methods--so we have a growing number of qualified researchers in this field

    2)a small conference for writing researchers held 2 years ago at U of California Santa Barbara School of Education has grown to a significant international gathering of top researchers in the field this coming February at the same institution

    3) two edited collections of research on writing (Smagorinsky, Teachers College Press) and Bazerman, Lawrence Erlbaum) have published significant research in the field over the past 20 years

    4) writing programs are doing more and more assessment of instructional effectiveness.

    The problems, though, are not with assessing and researching writing, although those challenges are significant (how does one effectively get inside a writer's cognitive processes?)

    The problems are systemic--underprepared high school and college level writing teachers, poor wages (and therefore, a weak labor pool), economies of scale that make it difficult for institutions to improve salaries, working conditions, and assessment funding for writing instruction, and a textbook industry driven by maintaining the status quo in writing textbooks rather than break the mold in light of new research findings.

    Anyone who can stem the flow of these mega-forces is Herculean.

  • Posted by Beth Hewett on December 28, 2007 at 1:20pm EST
  • I certainly agree that empirical research is needed in rhet/comp studies -- particularly those studies that attempt to determine whether and how students benefit from particular kinds of conferences, commentary, or other writing assistance.

    For example, in my own empirical study of online asynchronous and synchronous conferences between online instructors (in this case, Smarthinking e-structors)and first-year English students, I found that the students in the study did, indeed, make content (global) and sentence-based (local) changes to their writing in response to the conferences. Among other findings, they made their own decisions about what they would change, to include enlarging upon a suggested revision and/or choosing against such revision. In other words, students used the help, but remained autonomous. Interestingly, the research, which partially replicated and extended Anne Gere's research of 1987, revealed a number of areas where the instructor commentary needed adjustment for the entirely textual nature online conferences. There were very specific kinds of comments (particularly "suggestions") that students seemed to find confusing and difficult to decipher into revision changes; I address those professional development needs in a book manuscript under review. Regarding publication, I have been fortunate to find interested publishers of the general research in Computers and Composition (23.1, 2006) and Readerly/Writerly Texts (Vol 11&12, No 1&2, 2004-2005). Other research that has led to publication includes that conducted by Christa Ehmann and myself in Preparing Educators for Online Writing Instruction: Principles and Processes (NCTE, 2004).

    That said, I wonder if one thing that steers some scholars away from empirical research is the false impression that it reveals "truths" or certainties. Most people who conduct such research realize that there are relatively few "certain facts" that they can generalize to a broader population. But, given a contexualized understanding of the study, some sound suggestions usually can be made -- and these are more valuable in rethinking our practices than anecdotal statements and lore.

    Another thing that makes studies like the one Shelley Reid describes more challenging to do is that such work: (1) is very time consuming, (2) usually would benefit from more than one committed researcher, and (3) often is costly in materials other than person-hours. In other words, such research would benefit from being institutionally supported. An empirical study of almost any size is a big job and it takes a concerted effort to accomplish both the study and its analysis and ultimate reporting. So, while I agree with cga that grad studies need to support learning such research, so, too, do the departments and institutions in which the research would be conducted. It sounds like Shelley has that support.

    This is a great topic to lead into a new year. Glad that Scott Jaschik has opened it up for us.

    Beth Hewett

  • Which competencies?
  • Posted by David PhD on December 28, 2007 at 2:05pm EST
  • I'll be very interested to know which competencies Writing Programs (WPs) decide are the best ones to indicate whether a student has learned adequately or not. It is not as simple as the ability to create a grammatical sentence, nor even the ability simply to create a logical paragraph. That's high school competency. Rather, will WPs test, for example, the ability to shift perspective or tone in order to craft content for a different audience? How about the ability to qualify a claim? These things seem to me very important, but by no means do I see any easy kind of test that will be able to show such competencies. Nevertheless, plow on, Shelley Reid and others, for I believe that if WPs can define and set a standard for outcomes measurements, they might just lead the humanities in meeting the inevitable tide of "accountability."

  • Posted by JBM on December 28, 2007 at 2:55pm EST
  • "It is not as simple as the ability to create a grammatical sentence, nor even the ability simply to create a logical paragraph. That’s high school competency."

    Yes, of course it is. It is also competency that college students regularly lack.

  • Posted by Charles Bazerman , Professor at University of California Santa Barbara on December 28, 2007 at 2:55pm EST
  • The welcome presence of writing research at the MLA convention is part of a larger upturn of writing research in this country and others. This February over five hundred of the leading researchers of writing from all continents will be sharing their methods and findings at a conference at the University of California Santa Barbara. (Google "Writing Research Across Borders")

    In a communication and information age, the ability to write is even more essential than previously for participating in professions, businesses, governance, and citizenship. Writing instruction should be one of the highest priorities at all levels of schooling. Methodically collecting and analyzing data about how people write and learn to write, the motivations people have for writing, how the resources of writing can serve their needs, and how instruction can support the development of writers can help us accomplish our educational goals responsibly.

    The complexity of writing and the way it reaches into the core of our social, psychological, and creative beings suggests our research be multidisciplinary and sophisticated as well as comprehensive and careful. The best antidote to reductionism is more knowledge.

  • A comment from the trenches
  • Posted by Denis on December 28, 2007 at 4:20pm EST
  • I've been a community college comp teacher for more than 30 years.

    Here's why students are poor writers: They don't read. Reading--not formal classroom instruction--is the way most of us learned spelling, punctuation, and the grammar of written English. Improving student writing is as simple as turning students into people who read for pleasure, not because they have to. Of course, that's the trick. If I knew how to turn students into readers, I would have retired rich and famous years ago.

    Researchers can research all they want and innovators can innovate until they're blue in the face, but teaching academic English to students who don't--or won't--read is like teaching music to the tone deaf or painting to the color blind.

    And, by the way, the complaint that kids today can't write is nothing new. Pick a time in American history, and we teachers were saying the same thing.

  • Posted by Peder Halverson , Instructor in Liberal Arts at College of the Marshall Islands on January 2, 2008 at 7:25pm EST
  • Reading is the key. Reading takes time and is a habit of good writers. Programs which stress reading well written material from the early grades produce quality thinking and writing. There are no short cuts.

    Having said that we must do what we can with what we have. Kudos to research. More kudos to those parents (the most important link in the chain) and teachers who foster the love of reading and the music of our language.

  • Writing if Crucial
  • Posted by Milt Eisenhardt at College Money on January 3, 2008 at 11:05am EST
  • I am not an educator nor part of any interest group other than a business man who assists high school students in choosing a college. I am excited for those of you who are venturing forth on all fronts of the "writing is critical" issue. My daughter teaches special education in the poorest district in our state. Her goal is to get third graders to write a sentence. How basic yet so crucial for their future. Please don't let your efforts go unheard. Evangelize government, business, and your college presidents to treat this as seriously as medical and scientific research for if we can't communicate effectively, we can't move forward effectively. Thank you all.

  • Good news about writing research
  • Posted by Sherry Swain , Senior Research Associate at National Writing Project on January 3, 2008 at 1:45pm EST
  • Applause for the research that’s reported here and for the cautions to our field! Over the past several years, the National Writing Project has sponsored a number of research studies at universities across the country on the teaching of writing in grades 3 – 12. The results, all positive, show that the type of instruction offered by teachers makes a difference in the writing growth of students. What’s more, the research measures qualities of writing deemed important and relevant by leaders in the field of writing and assessment—as we put together a team of such people from universities and schools to develop and refine a system for determining “what’s good” in student writing. Perhaps the most important contribution of our multiple studies, called Local Site Research Initiative studies, is that we found it possible to conduct rigorous quasi-experimental research that does not damage writing instruction, yet examines what those of us in the field consider important to us. The LSRI studies have shown that good research is possible---IF we take on the challenge to do it ourselves rather than wait for outsiders to “do it to us!” Here’s a link to summaries of LSRI studies: http://www.nwp.org/cs/public/print/programs/lsri

  • Reharding research in writing
  • Posted by Leif Fearn , Professor at San Diego State University on January 10, 2008 at 5:30pm EST
  • The need is enormous. What is available is thin. I refer only to efficacy research, and my focus is K-12. The Smagorinsky research review, and the MacArthur, et al review are both terrific collections, but neither offers the critical point: "If you do these kinds of things with these kinds of students over this period of time, you increase the probability of getting these minds of writers. We have such efficacy evidence in reading, mathematics, piano, middle linebacking, sculpturing, and a host of other teaching/learning endeavors. The need in writing is desperate. We are mired in classroom management plans, in off-the-shelf diminutive lessons, in love affairs with this or that activity. We need evidence on what to teach, to whom, how, and for how long. We also need evidence relative to what to assess, how, and to what end. A bare start on the former appears in Action in Teacher Education (Summer 2007) and Journal of Basic Writing (Spring 2007). A response to the assessment question is out for review. We need as many advanced graduate students in writing doing basic research as there were in reading during the last third of the twentieth century. And they need to approach their research dispassionately.