So What Did You Learn in London?

With assessment and accountability at the center of policy discussions in Washington and elsewhere, international educators emphasized an increased need for research on measurable study abroad outcomes and what particular program characteristics cause student learning gains at several sessions during this week’s annual NAFSA: Association of International Educators conference in Minneapolis.

“It is no longer a fringe activity,” with more than 200,000 American college students going abroad each year and new federal funding initiatives for international study, Richard C. Sutton, senior advisor for academic affairs and director of international programs for the University System of Georgia Board of Regents said Wednesday afternoon. “But that money will not be free. It will come at the price of accountability and assessment measures."

In a session on “Changes That Occur Abroad,” Sutton highlighted Georgia’s systemwide research of study abroad outcomes, the Georgia Learning Outcomes of Students Studying Abroad Research Initiative (or GLOSSARI). The ambitious six-phase, six-year-old project covers a lot of ground, including:

• Comparing learning outcomes of study abroad participants with those of their peers who stay stateside.
• Tracking learning outcomes of study abroad participants by administering pre- and post-tests.
• Comparing the experiences and learning of students taking a particular course abroad versus those taking that same course at home.
• Performing a statistical analysis on graduation and persistence rates relative to study abroad participation.
• Identifying and conducting case studies on study abroad programs that produce strong results in student learning.

Some of the research depends on a battery of skills tests measuring intercultural learning outcomes -- among them a functional knowledge of cultural practices, increased knowledge of geography and knowledge of global interdependence.

Among some of the potentially surprising results of the Georgia study, at least within the study abroad world where longer is typically perceived as better: Students who studied abroad in short-term programs of eight weeks or less had higher four-year graduation rates, regardless of SAT score, than did students who studied abroad for longer periods of time.

Sutton suggested the possible explanation that shorter-term, mainly summer programs might not interrupt other academic plans, whereas spending a semester or more abroad could get in the way of degree completion. (Over all, however, the analysis does suggest that study abroad participants have improved academic performance and much higher persistence/graduation rates than their peers upon return. In a test sample of three Georgia universities with the lowest graduation rates, researchers found that 92 percent of those who studied abroad in 2000-1 were still enrolled or had graduated in spring 2003, compared to 22 percent of those who didn't go abroad.)

Following up on Sutton's presentation, Michael (Mick) Vande Berg, vice president for academic affairs at the Council on International Educational Exchange, made a case for programs that emphasize mentoring while sharing data from a multi-year study of more than 1,300 students from Georgetown and Rice Universities, Dickinson College, and the University of Minnesota. Students who reported receiving more mentoring on site also showed higher gains on the Intercultural Development Inventory (IDI), Vande Berg said -- perhaps a particularly important finding when it comes to reaching male students.

Not only did male students abroad record just half the language gains as females abroad, where female students gained in inter-cultural sensitivity as measured by the IDI, Vande Berg said, but males actually went backwards from their pre- to post-tests. Their language gains were just barely above those of the control group that didn’t study abroad, while their intercultural sensitivity scores were actually lower than those of students on their home campuses.

“The research simply doesn’t point to having students participate in any old study abroad program and coming back with positive results,” Vande Berg said -- again emphasizing the importance of intervention and mentoring for all students (though perhaps more critically for males).

The importance of ensuring a particular program's components are effective in enhancing student learning was also a topic during a session on “Qualitative Research” Thursday morning, where Lilli Engle, co-founder and director of the American University Center of Provence, described her research on gains associated with adjusting the components of her own program. “The fun part is realizing that the laws of cause and effect do work,” Engle said.

For instance, she described observing that students who traveled on group flights tended to stay in a pack with other program participants. After rearranging the program design so that students traveled individually and were met at the airport by their host families, she found that they were more able and willing to experience the culture on their own -- one of her goals for them.

“We can never in this day and age rest on our laurels, because things always shift and change,” Engle said. “Research is just a way of keeping abreast of that.”