• Just Visiting

    John Warner is the author of Why They Can't Write: Killing the Five-Paragraph Essay and Other Necessities and The Writer's Practice: Building Confidence in Your Nonfiction Writing.


Data and Judgment; Hurricanes and Student Retention

Balancing data and judgment when individuals aren't averages.

September 12, 2018

I am trying to work on a blog post I’ve been noodling on for days, but instead, I am watching this GIF over and over.







I’m watching the GIF because I’m trying to decide if I think we should stay and hunker down for Hurricane Florence, or if we should decamp from Charleston to Columbia.

Of course I’m not only watching the GIF. I’m checking my Hurricane Tracker app, refreshing Twitter to check out what various weather-related accounts have to say, periodically watching The Weather Channel, and once or twice, stepping outside to look at the sky, as though this might tell me something.

I’m guessing here, but I imagine that there is nothing subject to as much data gathering and analysis as a hurricane which threatens land, and yet even with all that data, and all that computer power crunching away, no one can tell me if me, my wife, and our two dogs need to amscray before the storm makes landfall sometime Friday.

South Carolina Governor Henry McMaster has ordered an evacuation of much of the South Carolina coast, including Charleston County, but when I went to the Publix to stock up on supplies in case we do stay, I saw a lot of people who looked like they were planning on hunkering down, rather than heeding the “mandatory” evacuation.

Judging from the nearly empty frozen pizza case, lots of people think they’ll still have power as well.

I suppose I can consider that additional data as we make our decision. What does it mean to go if most people have chosen to stay?

I’m thinking about how I’m processing and using this data in the context of a recent Inside Higher Ed article on the “mixed bag” of early-alert systems designed to identify struggling students so institutions can intervene.

Lycoming College in Pennsylvania with its 1260 students, has a system where “worried faculty members simply talk to their students or pick up the phone to call the dean and share their concerns.”

Others, such as Georgia State University (over 50,000 students), have had “remarkable success with heavily data-driven student success systems.” But when Lycoming experimented with a more formal, data-driven system, they found it inferior. The biggest issue was instructors relying on pre-determined checkpoints in the semester before intervening, rather than using their judgment to step in or speak up when they felt necessary.

The time spent on pleasing the data collection regime was also a problem, as was the feeling that monitoring students in such a way felt out of sync with an institution of higher education.

Other schools of varying sizes have tried different systems, some being preferred by some faculty and loathed by others. There doesn’t appear to be a one-size-fits-all solution, and each institution faces its own challenges.

Even when the systems do work well, they are not panaceas for issues like student retention. Ana Borray, director of professional learning at Educause remarks that “technology can only do so much.”

Even as I work on this post, the data available to evaluate whether or not we should evacuate has shifted. A run of the 12Z Euro model shows the storm hitting the brakes just off the coast of the North Carolina/South Carolina border and then looping south, potentially getting as far as Florida, strafing the entire coast in between with hurricane force winds.

I do not possess the knowledge and experience to know what that model means by itself, other than seeing that particular illustration made my heart beat a little faster. The expert meteorologists say that we shouldn’t put too much stock in one model run that’s trying to predict events 72 hours or more away.

If you watch six straight hours of The Weather Channel, you realize that professional meteorologists know a ridiculous amount of stuff about how hurricanes work, and yet even with all that knowledge, how hurricanes behave remains not entirely predictable.

The data informs the expert meteorologists' judgment and their judgments inform my judgment. At least that’s the process I’m trying to stick to. 

But it’s hard. There’s an argument that as a layperson, I shouldn’t even be exposed to this unaggregated data because of its potential to cause unnecessary worry. The local weather people are now tweeting all in a row about how to trust the forecasts, not an individual model? 

But what if they’re wrong? They’re human, flawed, prone to bias. The model is the model, data-driven, AI-informed!

Ultimately, the forecast does not make the choice for the individual, and individual circumstances often limit the ability to even make a choice. The hourly employees of the Charleston food and beverage industry are virtually forced to stay, rather than evacuate. 

My wife and I, on the other hand, can easily afford a hotel out of town. Her employer is closing, and my work travels wherever I am. We have the privilege of having resources which creates slack, which allows us to make this choice.

These early warning systems and guided pathways are designed around averages and probabilities, as they must be, but in the end, what happens always comes down to the student, or at least should come down to the student. We should also make sure all students have sufficient slack to change their minds and make the choices most consistent with their goals.

We should give students as much support in pursuing their higher education goals as possible, but I hope we don’t get to a place where the algorithm gets to make the choices for them, or when institutions trust the algorithms over professional judgment.

We have until Wednesday afternoon to make a decision on whether or not to evacuate.

Until then, I’m sure I’ll flip back and forth a dozen times.

Good luck to all those in the storm’s path.


Be the first to know.
Get our free daily newsletter.


Back to Top