ICTworks

Many scholars have identified an interesting phenomenon that manifests when introducing educational technology to populations in developing countries. The Matthew Effect, coined by Robert Merton, posits that students, who start better off, typically stay better off, and students who start worse off, often stay worse off. The Matthew Effect comes from Matthew 25:29:

“For unto every one that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath.”

As Michael Trucano aptly puts it in his blog post titled, “The Matthew Effect in Educational Technology”, the biblical verse “roughly translates [to] ‘the rich get richer”.

Not only would the Matthew Effect suggest that those with pre-existing advantages tend to benefit more, and more quickly, from the use of new technologies, but it shows that education technology needs to be re-evaluated as it raises important questions about the potential for new technologies to further increase existing education divides rather than help close them.

This effect is quantified in an article, “Educational Technology Isn’t Leveling the Playing Field” in Slate by Annie Paul. She documents work conducted by Susan Neuman, a professor at NYU and previously at the University of Michigan, and Donna Celano, an assistant professor at LaSalle University. They observed children at two identical libraries in the rich and affluent neighborhood of Chestnut Hill and impoverished neighborhood of Badlands. Their study found that:

“Granted access to technology, affluent kids and poor kids use tech differently. They select different programs and features, engage in different types of mental activity, and come away with different kinds of knowledge and experience.”.

One interesting occurrence noted was that a parental figure was present with the children at Chestnut, but not at Badlands. The two researchers believe that this might be a factor for the relative computer usage. Students at Badlands used the computer for “homework only 9 percent of the time, while 39 percent of the Chestnut Hill tweens’ information searches were homework-related.”

From a simple perspective of access to technology, access compounds the lack of hardware and software. Rural students have to spend time on other labor-intensive activities, so the amount of time that rural students have is limited.

Even if access is solved and infrastructure issues are resolved, the Slate article suggests that, generally speaking, many rural kids might use technology differently than urban kids do, for a variety of reasons.

Why does this matter?

In December of 2015, Tata Trusts announced a partnership with Khan Academy to implement educational technology in India. To what extent might the Matthew Effect be relevant for the Khan Academy with Tata Trusts, specifically in math?

I analyzed two pilot programs that introduced educational technology to Indian schools to help answer the Matthew Effect question for Khan Academy’s plan to introduce its education technology platform to India.

Case Study 1: Central Square Foundation: “Teaching with Technology”

The Central Square Foundation looked at teacher profile, school profile, and ICT usage. They gathered data from various groups with ties to schools, teachers, and online forums.

The study found that, except for computers, technology was both more available for students in urban schools, and (not surprisingly) used to a greater extent as well. What’s striking is that only 11% of rural schools had the ability to use functional computers, while 34% of urban schools had access to a working computer. 74% of rural schools had electricity problems and 61% of rural schools had hardware problems, while only 35% of urban schools had electricity or hardware problems.

The lack of working computers in rural schools suggests that some important conditions which often contribute to a Matthew Effect in Educational Technology might well be in place. Due to hardware and electrical issues, the gap between rural and urban students will only widen.

Ultimately, even if Khan Academy or even any organization were to implement a broad sweeping educational technology program, hardware and software issues would plague rural schools, thus widening the gap between rural and urban students.

Case Study 2: Abhijit Banerjee et al: “Remedying Education: Evidence From Two Randomized Experiments In India”

While the previous pilot program looked at infrastructure problems and access to technology, Abhijit Banerjee of MIT looks at student achievement with the introduction of education technology, namely specific deficiency targeting, in two randomized experiments in urban schools.

Students’ math scores in the computer program increased by .47 standard deviation. In Year 2, students in the bottom third of the class reported an increase of .425 standard deviation in math scores and students in the top third of the class reported an increase of .266 standard deviation in their math scores.

The study and results suggest that it is possible to ameliorate, or even counter-act entirely, the impact of a related Matthew Effect – provided that certain related approaches are considered and implemented to ensure that less advantaged students do not fall further behind their classmates as a result of the introduction of ICTs. In urban schools, educational technology helps student in the bottom third of their class improve their math scores more than the top third.

However, some important questions arise: For lower performing students, would more interventions such as more explicit instructions or more access help them academically? Is it that better performing students can self-study better?

All in all, the study shows that technology can be introduced in ways that benefit less well-performing students, provided that their specific learning needs are taken into consideration. Math scores significantly improved as a result of educational technology in the bottom third of the class.

What does this mean for Khan Academy?

The two pilot programs and resulting studies show that a more nuanced variation of the Matthew Effect is in play with the introduction of educational technology in India. It seems that it is possible to utilize technology to help support low-performing learners – provided that specific attention is paid to the specific learning needs and contexts of those learners.

What the Matthew Effect suggests is that, all other things being equal, simply introducing technology into a given context is more likely to benefit students who are already performing better. If, however, technology use is tailored to the needs of low performing students, the effects of a so-called Matthew Effect can be lessened – and perhaps even eliminated.

However, when it comes to comparing urban and rural schools, factors like software, hardware, and electrical problems make access to working computers a major problem for rural communities. Computer time will get significantly shortened, thus limiting the realized benefits of educational technology.  Hence, there can be a pronounced Matthew Effect between urban and rural schools.

For the Khan Academy and Tata Trusts partnership, it means that they should start by looking to improve infrastructure in rural schools or consider an offline version of Khan Academy, known as KA lite, that is not as dependent on infrastructure. However, improving infrastructure is not a silver bullet solution. There is a great deal of evidence that, even where access appears to be equivalent, usage, and the impact of this usage, are not.

Dealing with a lot of ‘non-access’ issues is the real challenge. If this were a simple matter of just making sure that all classrooms had the exact same technology set-up, and that all of the technology worked, things would be much easier.

Countering the Matthew Effect

It turns out that Khan Academy’s India strategy is taking steps to address the Matthew Effect. Their initial goal is to build an offline platform and Android application. This would increase access to rural communities but still would not solve hardware and software issues. To ease their transition into India, they are partnering with local schools and organizations to create highly specialized programs. And, to personalize the content, they are translating it to Hindi and later other languages and also adding an accent and terms familiar to Indian students.

Although these are the right steps, the current plan is not sufficient to address the entirety of the Matthew Effect. Access is not the only issue. Even if two schools in an urban and rural neighborhood have the same technology and same access to that technology, urban students would use the technology better than that of rural students. The Matthew Effect posits that those who already possess various advantages (by virtue of wealth, or class, or education background, or family situation, or geography, or ‘cognitive endowment’, or a variety of other advantages, as well as the confluence of many such factors) are more likely to benefit more from the introduction of new technologies than those students who do not enjoy similar advantages.

This is not to say that all is hopeless; rather, it suggests that deliberate steps may need to be taken to address the specific needs and learning contexts of lower performing or less advantaged students if such students are to benefit from the use of technology.

Ultimately, the Matthew Effect is an area that merits scholarly attention and should be further pursued.

By Rahul Shukla, Intern, World Bank