In 2019, the education department in Brazil, handed economists 178 public schools and let them randomize which ones got an AI essay-feedback tool called Letrus. The state bought Letrus for every public high school senior only after the result came back.
On September 14, the Gates Foundation pledged about $400 million for AI in schools, ahead of any comparable result for the tools its own annual report holds up as proof.
Brazil had the order right. Gates has it backwards.
Bill Gates makes the case for speed in the 2026 Goalkeepers Report. Many AI tools are “still in pilot mode,” he writes, and the distance from pilot to scale “could be much shorter than it was for earlier technologies.” Within three years, he says, the most promising tools will be broadly deployed. Gates Foundation CEO Mark Suzman told GeekWire that the education work is “still on a relatively small scale.”
We’ve covered the education beat for two decades on ICTworks. I have watched the distance from pilot to scale get shortened by decree. It never got shorter. It got skipped.
Sign Up Now for more AI in education insights
Letrus Evidence is a Working Paper
I praised Letrus in May 2025 and held it up as the model in January 2026. Both posts leaned on J-PAL’s summaries. The primary source is the Ferman, Lima and Riva working paper. I found no peer-reviewed version.
It is still the strongest study of AI in classrooms I know of from a middle-income country. The authors filed a full pre-analysis plan with the American Economic Association before the results. The trial covered about 19,000 students . The state delivered 8,000 notebooks to all 178 schools, control schools included, so the experiment tested software and held hardware constant.
Over the 2019 school year, both versions of Letrus raised scores on the full ENEM essay by 0.094 standard deviations . The version with paid human graders added on top did no better than AI alone. The authors benchmark 0.094 against a median of about 0.05 standard deviations for interventions aimed at high school seniors.
By that yardstick, Letrus did well.
The Best Trial Still Missed the Weakest Students
The paper reports two limits that J-PAL’s summaries leave out.
First, Espírito Santo picked the schools using a 2017 survey of how ready each school was to adopt online technology. In the sample, 93% of schools had broadband against 80% statewide. Only one school was rural, where rural schools make up about 7% of the state’s schools.
The second limit matters more. The authors find what they call suggestive evidence that the gains came from students in the top three quartiles of baseline language scores, with no gain in the bottom quartile. They conclude the tools could not help very low-achievers.
That is what good evidence looks like. It tells a buyer who the tool fails before the buyer pays for the failure. Espírito Santo bought Letrus with that caveat on the record.
Government Would Reject Gates’s Example
The Goalkeepers Report says grade 7 students in 21 US middle schools gained six months of learning with Kiddom Atlas. The foundation’s data sources page cites an opinion piece in The 74 by Kiddom co-founder Abbas Manjee, and Kiddom’s own press release.
Manjee describes Kiddom’s analysis of about 5,700 Bronx students. Classrooms whose teachers used the tool’s daily checks often scored 68% on those checks. Classrooms that used them rarely scored 52%. Manjee calls that five months of learning. Gates prints six. The outcome is the vendor’s own quiz, the comparison is heavy users against light users, and nothing in Manjee’s account describes random assignment.
Achievement Network CEO Michelle Odemwingie said on the launch press call that outside researchers will now study why the tool works. Michigan State’s Sarah Reckhow told the Chronicle of Philanthropy that the foundation has not named who will evaluate any of it. Gates committed $13,952,000 to Kiddom in December 2025.
Espírito Santo paid for evidence and then for software. Gates paid for the software.
Sierra Leone Trial Repeats Brazil Problem
The Goalkeepers Report’s other education example is a real trial.
Google DeepMind and Fab AI ran a pre-registered randomized study of Gemini Guided Learning with 1,763 students in 48 classrooms across 12 schools in Port Loko District. After eight weeks, treatment classrooms scored 0.258 standard deviations higher in math. Gates grant paid for part of it. EducAid kept a field monitor in every school to charge devices and fix problems.
The same pattern showed up. For each standard deviation of math skill a student brought in, the gain rose by 0.195 standard deviations. The strongest students gained the most. Two trials on two continents, with two different tools, found that AI helped the students who needed it least.
The Goalkeepers Report mentions neither finding.
Short Trials Report Big Numbers, Usually
Martín De Simone’s World Bank team reported 0.31 standard deviations after six weeks with Microsoft Copilot in nine Edo State schools. They called it nearly two years of learning. I ran that result in January 2025. DeepMind converts Sierra Leone’s eight weeks into 1.2 to 1.7 years. Letrus, measured over a full school year in 178 schools, came in at 0.094.
I cannot point to a law that says effects shrink as trials grow. What I can document is this. Every large number in the Gates report comes from 12 schools or fewer over two months or less. The one tool a government bought after a full-year trial moved scores by about a third as much.
These trials tested different subjects, grades and countries, so the comparison is rough. It is still the right way to price a plan: budget for 0.09 and treat anything larger as upside.
Technology arriving in low-income classrooms has also lowered scores at scale.
Ronak Jain and Samuel Stemper linked 3G rollout to 2.5 million test scores in 82 countries. They found declines of 0.04 to 0.08 standard deviations, concentrated outside high-income countries. I wrote that up last September. Santiago Cueto and colleagues followed 531 Peruvian schools for 10 years after One Laptop per Child and found no effect on academic performance.
Long-term Letrus Trial Died with USAID
Letrus now reaches more than one million students across Brazil, according to J-PAL.
USAID’s Development Innovation Ventures funded a randomized evaluation that would have followed 100,000 public high school students in Ceará from 2024 to 2026 and measured university enrollment and employment. J-PAL reports it was canceled when USAID closed in 2025.
Nobody yet knows whether a 0.094 gain on a practice essay turns into a university seat or a job.
That trial is the single cheapest evidence purchase available to anyone holding $400 million for AI in schools. It has a tool already in use by state governments, a research team that has run this kind of study before, and an outcome that matters more than any eight-week test score.
Restarting it would cost Gates a sliver of the pledge. It would answer the question every ministry should ask before it signs: does this help students years later, and does it help the ones at the bottom?
