You have /5 articles left.
Sign up for a free account or log in.
The Delta Cost Project, created in 2007, is a database meant to make higher education finance easier to understand. Since its inception, the project has helped clarify how colleges spend their money.
But because of how the data are grouped, a new paper suggests that there's a flaw in the Delta Cost Project's database.
Delta gets its data from the federal Integrated Postsecondary Education Data System, known as IPEDS. All colleges that participate in federal student financial aid programs are required to fill out IPEDS surveys, which are conducted by the Education Department's National Center for Education Statistics. Delta's work is meant to make that data usable and accessible to administrators and policy makers.
The problem, according to the paper, has to do with something called parent-child reporting: sometimes one institution will report certain data for multiple branch campuses -- or even for an entire university system.
When this happens, some numbers may look like they represent one institution or campus, when in reality they represent three or five or eight.
And when one university is reporting numbers from multiple institutions, things get complicated.
Take the University of Texas. According to Delta’s database, the University of Texas at Austin contains data that represent multiple institutions in the 14-campus University of Texas System. In 2009, according to Delta’s database, the University of Texas at Austin enrolled 164,000 students and spend $2.5 billion on instruction.
Those numbers actually represent multiple campuses in the university system. But when the researchers created an alternative database, which didn't collapse data from state systems, they saw different results: the University of Texas at Austin enrolled 47,000 students and spent $573 million on instruction that year.
And what if a researcher wanted to merge Delta’s data with other datasets? When looking at the University of Texas at Austin, Delta’s data would cover eight different institutions. But data from the National Science Foundation, for instance, might cover only the University of Texas at Austin.
The flaw in Delta's approach mostly affects public institutions. Ozan Jaquette, assistant professor of educational policy studies and practice at the University of Arizona’s College of Education and one of the paper’s authors, said it’s unwise to use Delta’s data to analyze publics -- or to compare publics to privates.
“The problem affects a minority of institutions, but it makes certain types of analyses problematic,” Jaquette said. “I wouldn't want to use the Delta Cost Project to analyze public institutions at all.”
Even though the problem affects a relatively small number of institutions, the higher education world relies on Delta’s data. Since the database was created, it has been used in countless analyses and research papers. Policy makers and researchers have been able to use and study data that would have previously been prohibitively difficult to understand. In 2012, The New York Times called Jane Wellman, the Delta Cost Project’s founder, “an oracle of the college-cost conundrum.”
“I would be nervous about making statements based on analyses from the Delta Cost Project data,” Jaquette said. “I wouldn't know, is this a real finding, or is this due to spurious problems in the data?”
Delta’s leaders think that’s rather harsh. Steven Hurlburt, director of the Delta Cost Project, pointed to a section of Jaquette’s paper comparing expenditures among three different databases. Compared to the other databases, Delta’s numbers were different -- but they weren’t that different.
“The differences in dollar amounts are insignificant,” Hurlburt said. “I don’t think the authors support those broad, sweeping conclusions that they make.”
And without grouping certain institutions, Hurlburt said, researchers will run into other problems. Say there’s one “parent” institution and three “child” institutions. When you measure per-student spending, you need to group the institutions in order to get an accurate ratio. If you don’t, you’ll be counting spending from all four institutions, but you’ll only be counting students at the “parent” institution.
And in that case, Hurlburt said, “you're going to make it look like they’re spending a whole lot more money than they actually are.”
In his paper, Jaquette also suggested that researchers create their own datasets. Delta’s database has a narrow focus, he argued, which makes it difficult to measure a wide range of issues.
“The Delta Cost Project was founded and funded based on this mainstream policy consensus about what are the important problems in higher education,” Jaquette said. “That narrow focus led to a narrow data set, which led to a narrow analysis being done.”
Wellman, the project’s founder, said that it would be a lot of work for researchers to create their own databases, and many would shy away from that solution. But there are other ways to improve the data, she said, noting the TCS Online system, another Delta product, which doesn’t use as much grouped data. Fewer colleges group their data today than in the past, and TCS Online only goes back to 2007, so it's able to display ungrouped data for any institutions that have been reporting separately since then.
She hopes that the paper will spur the National Center for Education Statistics to put resources toward updating the database and improving the TCS system.
“I completely agree that the data can be made better,” she said in an email. “I don’t agree that the solution is to stop doing the analyses because of it.”
