Before the Fact
ANAHEIM, Calif. -- Student data analytics are not a miracle cure -- nor are the models easy to set up -- but some institutions represented at the annual Educause conference said they are already seeing the benefits of identifying at-risk students before they drop or fail a course, as opposed to after the fact.
Speakers during back-to-back sessions here Thursday gave participants an introduction to building their own analytics models, then showed how the University of Kentucky is using student data to improve retention. Common to both sessions was a charge that institutions can do more with data they already collect than they do today.
“Just knowing that students dropped out is not enough,” said Rajeev Bukralia, CIO and associate provost for information services at the University of Wisconsin at Green Bay. “Once you know what happened and how it happened, then we want to know what could happen in the future.”
Most universities are using analytics on a descriptive level, Bukralia said, where they analyze how students have performed in the past and make changes that will benefit future students. Introducing Kentucky’s approach to analytics in a second session, James David Hardison, an industry principal at the enterprise software developer SAP, said “What if we changed the tense of the question? What if we made the question ‘How are they doing?’ ”
Vince Kellen, Kentucky’s senior vice provost for academic planning, analytics and technologies, later followed up with a sobering reality: “Everything I’m going to show you here, you’re not going to be able to do,” he said. “Your organization won’t be ready for that.... I don’t think the institutions as they’re constructed today can take advantage of analytics. Based on how we’re organized today, something has to change.”
Change may need to come in the form of education. When Bukralia in the earlier session asked how many audience members had heard of prescriptive analytics, two or three hands reluctantly went up in a crowd of about 40 people. While his session was marketed as a roadmap to building university-specific models, Bukralia listed some of the many factors that could derail such an effort, including bad data, disagreements between administrators and IT staff, lack of expertise or funding, and poorly defined goals.
Most importantly, he said, “A model that you are developing with your data will be unique. I am skeptical to those solutions that are based on one-size-fits-all.”
Kentucky’s push to improve student retention with analytics began in March 2012. The university merged its institutional research and business intelligence teams, hired three data scientists and moved to SAP’s High Performance Analytic Appliance platform. The team (whose size, 15, drew awed reactions from the audience) rebuilt the institution’s mobile app (originally created by Blackboard), which relaunched this semester. The app still serves as a digital gateway to the university, but now does so while collecting tiny bits of information about students.
Every time students open the app to check their course schedule or the date for the next Wildcats game, they may be faced with a quick question: Have you bought all your textbooks already? Do you own a tablet? On a scale from one to five, how stressed are you? In five weeks, students have answered those and other questions more than 40,000 times. Students can ignore the mini-survey, but response rates range between 50 and 80 percent, Kellen said.
Kentucky also pulls engagement data from the learning management system Blackboard Learn and tracks when students attend campus events through the incentive system TallyCats. The result is the “K score,” which lets students know how much they interact with the university. For example, should a student stop uploading assignments to Blackboard Learn, the mobile app could send an alert that could only be cleared once the student meets with the professor or an adviser -- and completes the work.
Kentucky’s freshman to sophomore retention is up to about 81.5 percent, posting an increase of about 1.3 percentage points, Kellen said. He was hesitant to attribute the increase solely to the university’s investment in analytics, however.
Kentucky’s next project in the realm of analytics is to offer more real-time feedback, Kellen said. Through a combination of online discussions on Blackboard Learn and data from Wikipedia, he said, students could soon receive personalized article suggestions based on what they are discussing with their classmates. He showed a mockup of video capture software that would automatically extract metadata from lectures. Analytics could identify high-performing students who could get paid to serve as on-demand tutors.
Kellen even suggested the mobile app could go one step further and automatically set alarms to remind students to get out of bed and go to class. “Why not?” he said. “We know their class time. Don’t laugh!”