On Twitter (I will not call it X), Kim Mitchell goes by the handle @academicswrite, and recently had a tweet in which she described an experiment with using ChatGPT in her writing course go viral. I asked her to write it up into a blog post because I thought others might benefit from her experience. What stands out the most to me is that Dr. Mitchell’s approach does not require deep knowledge or background of AI or ChatGPT. Instead, she works from her first principles as an instructor and models a collaborative and open approach with students. I’m pleased to be able to share it here. —John

A ChatGPT Teaching Experiment

By Kim M. Mitchell RN PhD

I first heard about ChatGPT when most people heard about it, in the fall of 2022. There were guaranteed to be two camps of thought about student’s use of the platform: those who condemned it as cheating and something to be banned, and those, like me, who preferred to keep an open mind. When I was assigned to teach a summer session course in the nursing program I am affiliated with and was planning to assign an academic writing assignment, I knew ChatGPT could not be ignored. I am forever a risk taker when it comes to new ideas. I made the decision to put ChatGPT on the table for the term and told students they were allowed, yes ALLOWED, to use it. The catch—they were going to be required to tell me about it by writing a reflection they would submit when they handed in their paper.

When I planned this experiment, I knew little about ChatGPT. I had never used it myself or even played with it. I knew there had been some controversies in my department about student’s using it to write discussion posts on the learning management system. Instructors had tested it by feeding it their discussion post prompts and found themselves appalled at the excellent renditions of their assignments it was spitting back at them. Some students were even taking the time to cite course readings into what ChatGPT wrote. Instructors found they were not able to detect who used it and who didn’t.

Mindset and Attitude

Critical to the success of my experiment were two key aspects of mindset and attitude. First, I was brave enough to use it because I was convinced—or just arrogant enough to believe—that there was no way an AI bot could write an appropriate rendition of the assignment I had created. The assignment was created for a course exploring evidence-informed practice and nursing research. The students were being asked to write a qualitative synthesis paper. They were assigned to find five qualitative research studies and thematically analyze those studies and report on two to three similarities among the findings. The paper had five sections: a background to their topic, their personal inspiration for their topic choice, a description of their methods of identifying their study themes, and a section describing potential nursing practice implications from their findings. The referencing and citation requirements are very specific, and each section has to be cohesive with the others. If you are interested in learning more about the paper and its rigorous scaffolding processes you can read about it here (https://academicswrite.ca/2018/11/25/student-peer-review-process-heres-my-version/)

Second, I knew that after telling the students it was allowed, I could not hedge on this decision. There were to be no ifs, buts, or exceptions. I wasn’t going to teach them how to use ChatGPT—I couldn’t have anyway, I didn’t know how—but they had to trust that I meant it when I said it was allowed. Building a classroom culture of trust was essential to the success of my experiment. I needed them to feel free to tell me the truth in those required reflections. If they for one second doubted that I trusted them or felt I would punish them for certain kinds of ChatGPT use, the whole experiment would have backfired.

How Students Used ChatGPT

Approximately 52 percent of the class reported using ChatGPT with varying levels of success. Here is what I learned about how they used the platform to help with their paper.

• Preparation activities: ChatGPT helped them organize their thoughts, provided them with definitions, helped them decide on a topic to write about, and it was used as a search engine (this was not successful as it often provided made up references). It also helped them get ideas for how their topic was connected to nursing or for their nursing implications.

• Writing the paper: ChatGPT was not very effective at writing the paper, but it was helpful in the introduction if it was provided with a specific bullet point list of items to include. It was also useful for writing conclusions. It could not help them thematically analyze their five chosen qualitative studies. One student found a clever way to have it write practice implications based on their chosen themes but given all this information needed to be cited, I imagine they still had to go back and insert citations in places and continue to edit the text.
• Sentence level corrections: ChatGPT seemed most successful at the sentence level with helping with rephrasing, simplifying sentences, grammar, paraphrasing, finding synonyms and writing transition sentences.
• Revision activities: Many of the sentence level activities were also revision activities. Some of them asked ChatGPT for global revisions, some of them used it to summarize their paper to help them understand if they had been clear and their intended message was getting across. Chat GPT was a good proofreader. Others used it to give better titles and heading labels. Someone used it to shorten their paper but found it just summarized three paragraphs into one.
• Overall ChatGPT was more unhelpful than helpful: Students also reported on ways that ChatGPT was not helpful. If I had a crystal ball, I would predict that many students, now that they’ve tried it, wouldn’t use it again. They found they still had to edit what ChatGPT did as it often it would not give the meaning they were hoping for. Some felt that using it seemed like cheating themselves. Others were simply not successful with it at all and found it vague, nonspecific, and repetitive. A couple of students pointed out that they felt ChatGPT stole their unique writing voices from their writing. Some felt it took too much time to do small things that you could do on your own. Overall, they were still doing a lot of their own writing and thinking as writing assignments are intended to do.

Nearly all the students used ChatGPT in ways that I felt were of legitimate help for improving the quality for their writing in particular with revisions and rewording. Given these findings, this is an experiment I would try again in future classes. I now have the ability to be more specific with how ChatGPT could be useful for them with their assignment and my main task in future course offerings would be to watch how ChatGPT grows and adapts to my writing assignment, because that’s what AI is designed to do.

Dr. Kim M. Mitchell is an Assistant Professor in the College of Nursing, University of Manitoba, Winnipeg, Canada with a research program in nursing student literacy and retention.