Preparing for 'Era of Data Ubiquity'

Imagine if a college, using learning analytics, has determined that students of a specific ethnic background who live in a handful of zip codes and score a certain way on standardized tests are highly likely to earn a low grade in an important course -- potentially jeopardizing their chances of graduating on time. Should the college actively prevent those students from enrolling in the course?

That is an example of the type of dilemma researchers from more than a dozen colleges and universities debated earlier this month as they made progress toward developing a set of shared standards for ethical use of student data, including how the data should be used to improve higher education.

Asilomar II: Student Data and Records in the Digital Era was a sequel -- both in name and scope -- to a similar event held in 2014. Attendees, most of them administrators and faculty members, met that summer to discuss what researchers should do with data created by learners engaging with their universities through free initiatives such as massive open online courses.

While the first convention was a response to what the organizers called “MOOC mania,” the 2016 reconvening was a recognition that, today, copious amounts of data are gathered about students regardless of whether they study online, in person or in a blended setting. Beyond ensuring that they collect the data, colleges now have to worry about whether or not the information is being put to good use, and also if the traditional college transcript is the best way to represent it.

But many colleges are in their present state unprepared to use the data to improve teaching and learning, attendees and organizers said. Enabling them to do so will take the simultaneous development of new data systems, organizational models and ethical guidelines, they said.

“Fine-grained data on student activity, assessments and accomplishments, recorded online, offer tantalizing potential for dashboards, living transcripts and real-time feedback to teachers and administrators,” Isaac Chuang, a professor of electrical engineering and physics at the Massachusetts Institute of Technology who attended the assembly, said in an email. “As higher education is thus driven to evolve into a new kind of learning organization, what are the right principles and policies for safeguarding student (and teacher) data, while allowing the potential to be realized?”

Conversations at the 2014 event revolved around creating and agreeing to principles where there previously were none. This year, participants shifted their focus to enrolled students, whose data are already protected by federal law, institutional review board policies and other regulations. To tackle what organizers called a “more difficult conversation about responsible use of student data in an era of data ubiquity,” the attendee list this year expanded to include officials of higher education associations and ed-tech companies.

A handful of colleges, including the University of California, Los Angeles, and the Open University in the United Kingdom, have already issued their own policies on ethical data use. But Elana Zeide, a privacy scholar at New York University's Information Law Institute, said the diversity of higher education makes it difficult to create a one-size-fits-all solution. “As a result, the primary responsibility for student privacy and ethical information practices will fall on institutions, not lawmakers,” she said in an email.

Unlike in 2014, this year’s meeting did not produce a brief report with a handful of recommendations for the ethical use of data. The organizers, representing Stanford University and the research and consulting firm Ithaka S+R, plan to publish a set of white papers in August, but spoke with Inside Higher Ed about some of the major themes that came out of the event.

Much of the work took place in three subgroups, which discussed how data collected about students should be used in research about learning, applied to improve instruction and represented to reflect student accomplishments.

“Wrestling with those ethical issues and having a framework to work within is something that unites institutions across the sector,” said Martin Kurzweil, director of Ithaka S+R’s educational transformation program. “There is a value to having a sectorwide conversation and coming up with a set of standards.”

Essentially, said Mitchell L. Stevens, associate professor of education at Stanford, the groups debated “What kind of science do we want to do, by what discipline do we want to apply that science and how do we make sense of official representation of student interaction?”

The groups made progress on articulating a handful of ideas that participants said should guide the responsible use of student data.

The first, Stevens said, can be described as “shared understanding” -- the idea that students, colleges, vendors and others all contribute to the creation of data, and that ownership does not rest solely with one party. “It’s important that all those parties that contribute to the product mutually understand what purposes data are and are not being put to,” he said.

The second is a commitment to transparency. When data are used to evaluate students, the systems should be easy to understand and offer ways for students to appeal their grade, as the “clarity of evaluation is the hallmark of humane learning,” Stevens said.

Third is the idea of “informed improvement” -- in other words, that colleges have a responsibility to use the data they collect to improve student learning and support. One attendee, Stevens said, described the idea as a “standard of care,” only for higher education.

Finally, the groups endorsed a vision of “open futures.” Stevens described it as “the idea that education should create opportunity rather than prevent it.” In the case of a system that uses analytics to predict student success, for example, that system should not be used to prevent students from pursuing their dreams -- to “foreclose the opportunity that students can be different people than they were yesterday,” he said.

Additionally, attendees returned to the idea of “continuous consideration” -- a commitment to keeping an open conversation about privacy rather than attempting to settle the issue during a two-day meeting. That idea was among the principles endorsed during the 2014 convention.

The group tasked with discussing representation of student data spent part of its time focusing specifically on the college transcript, which Stevens said is “not unnecessary but not a representation of student learning” once colleges begin to make use of student data streams.

Instead of tearing up the transcript and brainstorming concepts for a brand-new way of tracking student learning, the group leaned toward using the transcript as a foundation and building new features on top of it, Kurzweil said. Many of the participants are already doing so on their campuses, such as the University of Maryland University College, which this fall plans to pilot a digital extended transcript, as well as other colleges that have experimented with electronic-portfolio platforms, including Arizona State University and institutions in the California State University System.

“Many in higher education are interested in how new forms of online learning could let us reconfigure education in new ways, such as developing competency-based degrees, combining credits from different institutions -- even those beyond the university -- and credentialing work and life experiences,” Justin Reich, executive director of the Teaching Systems Lab at MIT, said in an email. “One of the places where new ideas bottleneck is at the humble, venerable, nearly invisible student record. Rethinking the student record is a way of rethinking other parts of the system of higher education.”