When Public Data Isn’t Enough: Lessons From Aligned Impact Muscatine County
How local data reveals what public data obscures
By Hephzibah Okorie, FCP Program Officer
Much of the social sector’s ability to understand how communities are faring is dependent on access to federal public data.
Surveys such as the ACS offer vital population data on social, economic and demographic outcomes and characteristics of American communities, down to the county level. Resources such as the Consumer Price Index help us understand price changes across consumer categories that impact the average cost of living.
Access to these resources allows us to answer the big questions: How are children faring? Can people afford to live here? What are their health outcomes?
However, anyone who has relied on our public data infrastructures is at once familiar with both the depth of information available and the limitations of the system.
In this resource, we reflect on some of the challenges that arise with sole reliance on public data infrastructures, and look to Aligned Impact Muscatine County, an FCP grantee who is tackling these problems with localized data sources. We outline five gaps in public data and show how AIM is closing them:
The Gaps In Public Data Infrastructures
This section outline five challenges that arise from sole reliance on public data infrastructures. These challenges include gaps within the infrastructures themselves as well as gaps in the depth of information they provide:
Federal and regional data sets are often the most available and easily accessible data sources. However, they typically only offer granularity up to the census tract or block group level. This means that key differences within smaller units such as neighborhoods are often minimized within averages. For instance, in small rural communities such as St. George’s, ME (contained within one single census tract), a prime destination vacation location, the massive wealth disparities between year round residents and seasonal residents or second home owners are obfuscated within a single census average. The resulting census data reflects a much different, often rosier reality from the locally validated narrative.
Public data often lacks the depth of disaggregation needed to understand what is happening in diverse communities. Often, these datasets are not required to demographically disaggregate data or they simply do not reveal counts for certain racial categories that do not meet certain population thresholds (to protect identities). Once again, disparities across groups are hidden within the average and discovered too late for early interventions to be impactful.
Public data is often one to three years behind. While longitudinal data and historical narratives are incredibly valuable, communities need the stories of right now so that their strategies can reflect current needs and attend to present realities.
Public data can tell the story of what happened but it cannot always tell you why. If communities cannot accurately diagnose the structural influences of their conditions, then solutions and strategies will lack a strong foundation or an effective theory of change.
The metrics communities choose to track are often shaped by what is readily available in public data sets. . These measures—third-grade reading, graduation rates, housing cost burden, employment rate—while relevant, don’t always present the full breadth of metrics needed to help us understand if our communities are thriving. This begs the question: What do we miss when we rely solely on readily available public data?
Despite these challenges, community organizations are leveraging their local partnerships to tap into existing data sources, de-siloing and incorporating them to form a comprehensive, more timely picture of how their communities are faring. One such example is FCP grantee, Aligned Impact of Muscatine County.
Located in Muscatine, Iowa, AIM is a community‑wide cradle-to-career initiative of United Way Muscatine that utilizes the collective impact framework to address complex challenges facing children, families, and the workforce in Muscatine County. By convening schools, families and local employers in pursuit of a set of common goals, AIM is able to use data to drive decision making that reduces achievement gaps and improves outcomes for children and families. The results so far are telling.
Since 2020, despite declining national student literacy rates, Muscatine County has seen steady increases in 3rd grade reading scores, up by 16 percentage points(pp) since 2021. They have also seen precipitous improvements in 8th grade math scores, up by 15pp since 2021 and closing the 14pp gap between Muscatine students and Iowa state averages. Their high school graduation rate has risen to meet state levels and disparities across demographic groups have significantly reduced in the same timeline.
Much of this is possible because AIM has, since its inception in 2016:
- Invested heavily in strengthening local trust and buy in.
- Developed shared data systems that offered a detailed view of students’ in and out of school experiences and outcomes. AIM has structured data sharing agreements with all three school districts, twelve organizations offering 18 programs and fifteen local employers.
- Leveraged that increased visibility to structure collaboration and collective accountability. AIM has developed working groups around each of the metrics, holding community data release events where movement in metrics is assessed and strategies are devised to address key issues.
How AIM's Local Data Infrastructure Fills The Gaps
AIM’s community relationships allows integration between public and local private data sources enabling them to close the gaps outlined in the challenges above
AIM has a population made up primarily of white and hispanic families, a diversity that public data doesn’t adequately attend to, hampering their insight into differences between ethnic and socioeconomic groups. But these differences matter and are often the key to understanding overall average declines or disparate progress. For instance, this year, Kindergarten FAST scores showed a 2% overall increase. Wonderful news, until AIM disaggregated the data and saw a 14% decline for Hispanic students. Without access to local data, this decline would have remained obscured within the average and Hispanic kindergartners would have gone without the interventions needed to support them.
On average, most states take over five months after student testing to release test scores. By that time, budgets are set, learning plans are established and most students are already back in classrooms that will not account for the challenges they are leaving the previous year with. In other data sets, public data is even farther behind, sometimes as much as 1-3 years. AIM’s data sharing agreements means that they can return critical data back to the community at a much faster rate. Long before the state reports on Kindergarten to 6th grade literacy proficiency , AIM is able to see those scores, identify progress or locate declines and work with community partners to embed or sustain strategies within the coming school year that address those challenges. Even better, they are able to see mid-stream disparities crop up which allows for interventions within the school year. With national data that shows that most kids who fall behind in 3rd grade reading never catch up, timely data and thoughtful interventions can make all the difference.
The cradle to career framework posits that by supporting children through every stage of life, we increase the possibility of economic mobility in adulthood. However, data gaps can constrain many communities from determining whether these efforts truly result in the desired outcome. One of those gaps exists in tracking post-secondary outcomes for students who do not attend two or four year college. When students do not appear on the National Student Clearinghouse, it becomes harder to track them. For Muscatine, that amounted to 160 out of 350 students, 45% of the class of 2024. Our current grant to AIM is supporting them in filling this gap. By tapping into the community, AIM is developing relationships with local employers, vocational training academies and other non-traditional pathways to meticulously account for all their students. The ability to identify all student pathways provides many opportunities. Not only can they better assess the impact of their cradle-career strategies, they can also identify ROI for employers who fund or participate in student vocational programs and critically, expand the web of community partners who are invested in the wellbeing of Muscatine youth.
While public data can show us that indicators have changed, it typically can’t offer any detail on the WHY. It can’t tell us which programs are working better than others or what combination of interventions is more likely to move the needle on an outcome. However, Muscatine’s access to student level local data offers them critical insight into the strategies that work. They do this by comparing outcomes between students who have received certain interventions and their peers who did not. For instance, by mapping public literacy data to programmatic service data, they were recently able to determine that students who received Reading Mentor services saw a 31% improvement in literacy in one school year compared to a 7% improvement in students who did not receive similar services. Such insight allows them to scale and increase access to programs with proven success in improving outcomes while also supporting their program partners in understanding their own data and making better decisions.
The challenges with public data systems aren’t minor inconveniences; they are real challenges with actual power to curtail opportunity for people and communities around the country. Addressing those challenges on the federal and state level is critical. However, as demonstrated by the work happening in Muscatine, access to and integration of local data presents a transformative opportunity.
By de-siloing and harnessing the data that is already being collected by our institutions, we return the people’s data back to them in ways that are timely, accessible and actionable, and strengthen local agency to improve the outcomes that matter.
