⇓ More from ICTworks

New Toolkit: 5 Methods to Value ICT4D Project Impact for Donors and Government

By Wayan Vota on January 8, 2021

ICT4D project impact valuations

There is a growing demand from donors, implementers, and governments to apply valuing impact approaches to ICT investments. Decision makers want to understand the social value created by digital development to inform which interventions are the most appropriate and best value for money.

Yet, ICT4D projects are known for having social, economic, and environmental impacts that are difficult to capture. Investments in digital infrastructure and innovation often have high up-front costs with largely medium-to-long-term benefits, not quick, short-term outcomes desired by policy makers.

The new DIAL Valuing ICT4D Project Impact Toolkit is a practical guide to identifying appropriate valuing impact methodologies to forecast the benefits of planned ICT investments. The toolkit can also be used to evaluate existing digital development activities in low-income countries. It presents five valuation methodologies for ICT4D projects:

  • Cost Effectiveness Analysis, Cost Benefit Analysis, and Social Return on Investment explore the relationship from investment costs through to impact.
  • Multi-Criteria Analysis explores the relationships from activities through to outcomes.
  • Econometrics explores the relationship from activities through to impact.

The methodologies considered vary in their application, but they largely explore the relationship between investment costs, related activities and outputs, and the desired change in outcome or impact. Implementing these methodologies can be complex and resource intensive depending on the scope and level of detail desired.

Cost Effectiveness Analysis

Cost Effectiveness Analysis focuses on one outcome and describes how much money was required to create the change observed in that outcome. It allows comparison between how effective different types of investment are at achieving the same outcome.

This type of comparability is useful for ICT investments, as the tool can help highlight how a digital technology is more cost effective at achieving change in a specific outcome than other types of outcomes, something that may not be apparent at face value.

Given its focus on one outcome, CEA is best used when the intervention’s aims are relatively simple. It is less effective in capturing the complexity of impact often found in ICT investments.

Cost Benefit Analysis

Cost Benefit Analysis is a widely used methodology for comparing the benefits and costs of a given project or programme to understand whether it offers good value for money.

A key characteristic of ICT investments is their scalability, where a pilot’s scope can expand quickly to impact issues not within the original scope. With this can come higher investment costs. CBA allows the capture of both the expanded impact and the associated expanded costs.

A CBA approach is applicable across the long timelines involved in ICT4D. As the scope of a technology intervention expands, incorporating new information on types of impact and costs is straightforward. The focus of CBA on multiple outcomes makes it useful to capture the complexity of impact often found in ICT investments.

However, the Cost Benefit Analysis method involves minimal stakeholder engagement, so might not capture all the complexity of intended or unintended impact from an ICT intervention.

Social Return on Investment

Social Return on Investment is an outcomes-based impact evaluation that captures the full value for money of an investment through a high level of stakeholder engagement. It is more often used for retrospective evaluations.

The comprehensive nature of an SROI approach is also well-suited to capture the non-linear aspects of digital development when projects are scaled-up. An SROI allows the capture of both the expanded impact and the concomitant expanded costs. For example, if only a few people have access to a phone there is not much impact on communication, but phones become increasingly beneficial the more people gain access to them.

The significant importance placed on stakeholder engagement makes this method particularly suited to capturing the complexity of impact often found in ICT investments. Hearing from those impacted can ensure all the intended and unintended consequences stemming from ICT intervention are accounted for.

Multi-Criteria Analysis

Multi-Criteria Analysis evaluates an intervention by establishing preferences between several possible options assessed against defined criteria. MCA is particularly suitable for structuring and providing a consistent approach when handling large amounts of complex and intangible information. It can aid decision-making by forming preferences when comparing between interventions or options within an intervention.

A key feature of Multi-Criteria Analysis is its emphasis on judgement. This involves stakeholder engagement to establish objectives, criteria, and their relative importance. Although quantitative data and methods can be included, MCA does not necessarily result in a monetary value.

MCA involves assigning impact with different levels of importance. This is useful in an ICT context, as given the wide range of impacts these interventions can have, some outcomes will be of more importance than others in terms of the intervention’s aims.

Econometrics Analysis

Econometric analysis is a statistical approach to understand whether there is any change in an outcome due to an intervention. This could be at project, programme, or national level. The researcher usually selects a headline outcome (eg increase in productivity) and uses statistical or modelling tools to assess the impact of an intervention on the selected outcome.

While econometrics only focuses on measuring one outcome, the method incorporates various variables to deal with the complexity of the factors that cause change. This is useful for evaluating ICT investments. Scalability is a significant aspect of ICT interventions.

For example, the scope of an intervention can expand to cover a wide geographical area. Econometrics techniques are well-suited to scale-up to capture the expansion of impact (if the data is available).

A focus on large datasets and less emphasis on stakeholder engagement makes econometrics methods suitable for national level interventions.

Filed Under: Management
More About: , , , , , ,

Written by
Wayan Vota co-founded ICTworks. He also co-founded Technology Salon, MERL Tech, ICTforAg, ICT4Djobs, ICT4Drinks, JadedAid, Kurante, OLPC News and a few other things. Opinions expressed here are his own and do not reflect the position of his employer, any of its entities, or any ICTWorks sponsor.
Stay Current with ICTworksGet Regular Updates via Email

Sorry, the comment form is closed at this time.