The Power of a More Complete Picture

By AJ Gutierrez, Alaina Boyle Ph.D., and Elizabeth Sykes

For more than a decade, Equal Opportunity Schools has been asking a deceptively simple question: Who are schools missing? Schools have more information about students than ever before. We know their grades, test scores, attendance, course history, and much more. All of that matters. But it rarely tells the whole story. A student may have an aspiration no one has asked about. They may be deeply interested in a subject but have never imagined themselves in an advanced class. A teacher, counselor, coach, or another trusted adult may see something in that student that is not obvious from a transcript.

For years, EOS has been exploring what happens when schools bring those different pieces of information together. What if we combined traditional academic data with what students tell us about their interests, aspirations, experiences, and barriers, then put that information in the hands of adults who can do something with it? Our research partnership with Mathematica has given us an opportunity to test that idea more rigorously.

The research journey

The work with Mathematica unfolded in two stages, with the second study building directly on what we learned from the first. Together, they allowed us to move from a question about who schools identify to a harder question about whether identifying and engaging students differently actually changes what happens to them.

The first study focused on identification. EOS does not rely on prior academic performance alone when helping schools determine which students may be ready for advanced coursework. Our approach combines traditional academic information with student survey data about strengths, assets, aspirations, prior courses taken, and recommendations from educators. Mathematica compared that approach with AP Potential, which uses PSAT and SAT performance to identify students who may be prepared for specific AP courses. Across nearly every demographic group and school context Mathematica examined, EOS identified more than twice as many of the students our model was designed to reach. The study also found important differences between the students identified through the two approaches. Students identified through AP Potential were more likely to enroll in AP, and among those who enrolled, they were more likely to pass their AP courses and exams and tended to earn higher average exam scores. EOS, meanwhile, was identifying a broader group of students who might otherwise have been overlooked. 

That first study gave us an encouraging answer to one question. When schools broaden the information they use to understand student potential, they see more students. But it also raised the more consequential question. Does seeing more students actually lead to more opportunity?

That became the focus of the second Mathematica study. Researchers examined EOS partnerships spanning the 2015 through 2023 school years and connected EOS information with national data from the Civil Rights Data Collection, College Board, and National Student Clearinghouse. Depending on the outcome being studied, the analyses included hundreds of EOS partner schools and thousands of comparison schools. Mathematica used a rigorous quasi-experimental approach called difference in differences. In simple terms, researchers examined how outcomes changed in EOS schools after partnership began and compared those changes with what happened in other schools over the same period. This was not a randomized controlled trial, so we should be thoughtful about what the findings can and cannot tell us. Mathematica also tested whether important assumptions behind the analysis held and avoided drawing strong conclusions when they did not. 

That is exactly what we wanted from this work. Research should not exist simply to confirm what we already believe. It should help us understand where there are strong signals of impact, where the evidence is less certain, and what we need to learn next.

The clearest signal was access

Mathematica found that partnering with EOS increased AP enrollment by 3.7 percentage points overall and 4.3 percentage points for students from underrepresented racial and ethnic groups. At the average EOS partner school in the analysis, that translated to approximately 52 additional students enrolling in AP courses, including 34 additional students from underrepresented racial and ethnic groups.

I think the story becomes even more compelling when we put the two studies together. The first found that when schools look beyond traditional measures of academic performance, they identify substantially more students who may be ready for an opportunity. The second found that when schools partner with EOS to act on that information, more students actually enroll.

Mathematica did not isolate which individual component of EOS produced that result, so we should not claim that student surveys, educator recommendations, or any single part of the model caused the increase. But taken together, the studies provide encouraging evidence for a broader idea that has increasingly shaped our work. Combining traditional data, student insight, and the knowledge and actions of adults can help schools make different decisions about students.

That matters because data by itself does not change a student’s trajectory. Someone has to act on it. An adult has to recognize the student, understand what matters to them, have the right conversation, and help connect them to an opportunity.

Making better use of what schools already have

This may be the part of the research that I find most compelling. We are not talking about redesigning schools from the ground up. AP courses already exist. Dual enrollment programs already exist. Career pathways already exist. Tutoring, mentoring, extracurricular activities, and many other opportunities already exist. Schools and districts have already made enormous investments in creating these programs. They have also made an even larger investment in people. Teachers, counselors, school leaders, coaches, advisors, and other staff members represent the infrastructure of public education.

The opportunity may be to help schools use those investments more effectively. Imagine a counselor knowing not only that a student has the academic preparation for dual enrollment, but that the student has expressed an interest in health care and is worried about the cost of college. Imagine knowing that a student beginning to disengage from school has identified a particular teacher or coach as someone they trust. Then imagine giving that adult useful information and thoughtful guidance about how to engage the student.

That is a different vision of educational data. The goal is not another dashboard that asks educators to spend more time analyzing information. The goal is to help an adult understand what the information means, why it matters for this particular student, and what they might do next. It is not about replacing educator judgment. It is about making that judgment better informed.

Access is only the beginning

The Mathematica findings also make clear that connecting students to an opportunity is not the end of the work. Across all schools, Mathematica did not find a significant increase in AP exam participation. But a different pattern emerged in schools serving larger concentrations of the students EOS has historically focused on. In schools where a majority of students were from underrepresented racial and ethnic groups, AP exam participation increased by 2.6 percentage points overall, 3.2 points for students from underrepresented racial and ethnic groups, and 4.1 points for students from low income backgrounds. Similar patterns appeared in schools serving primarily low income students. 

We do not yet know why the model appears to operate differently in these environments. Mathematica points to several possibilities, including stronger implementation, greater educator buy-in, and differences in the environments students experience once they enter rigorous coursework. That uncertainty is useful because it tells us where to look next.

The performance findings were similarly instructive. Bringing more students into AP courses did not significantly change average AP exam scores overall or for students from underrepresented racial and ethnic groups. There was an important exception. Average AP exam scores for students from low-income backgrounds declined by about one tenth of a point on the five point AP scale. The study could examine average scores, but not whether the overall number or share of students passing AP exams changed. 

We should not explain that finding away. Instead, it points directly toward another important question. Once we help a student reach an opportunity, what information, support, and guidance do they need to succeed? The study also did not find evidence that EOS increased postsecondary enrollment. The primary analysis for that outcome did not meet an important statistical assumption, and Mathematica’s subsequent sensitivity analysis also found no statistically significant effect. Those findings help us understand both the promise and the limits of our historical model. 

Where the research takes us next

We began this research journey by asking whether schools were overlooking students for advanced coursework. Increasingly, we are asking a broader question. How can we equip adults with the right information and guidance to strategically engage students and connect them to the opportunities and supports that matter most?

That opportunity could be AP courses. It could be dual enrollment or a career pathway. It could be tutoring. It could be reconnecting a student who is becoming chronically absent. The application changes, but the underlying challenge is remarkably similar. We need to see the student more fully. We need to understand what our traditional data tells us and what students are telling us about themselves. We need to understand the relationships around that student. Then we need to put useful information and guidance in the hands of an adult who can act on it.

This is particularly important because schools already devote such a large share of their resources to people. If educators are one of the biggest investments we make in public education, there is enormous potential in helping them make more informed and strategic decisions with the students they already serve. That does not require rebuilding the entire education system. It asks a different question about how we make better use of what is already there.

The Mathematica studies do not give us every answer. They were not designed to. What they give us are strong signals that schools can connect more students to opportunity when they broaden how they understand potential. They show us that access does not automatically translate into success for every student. And they give us much better questions about how information, relationships, and guidance can help educators determine what to do next.

Perhaps most importantly, this research suggests that improving outcomes for students does not always require inventing something entirely new. Sometimes the opportunity is to get much better at using the people, programs, information, and opportunities we have already invested in.

Here is the full research summary from Mathematica: