⇓ More from ICTworks

Transactional Data: How Mobile Network Operators Leverage Data for Better Service and What It Means for M&E

By Guest Writer on December 24, 2014

gsma_1

Mobile for Development Impact were delighted to support the recent M & E Tech events in Washington D.C and New York. We see our role as catalysing the creation of commercially sustainable mobile enabled products services that also have a social impact. These events were a fantastic opportunity for us to learn from a range of people working in this area, as well as to share some of our recent relevant work.

Mobile presents an unprecedented opportunity for those working in international development, in that it opens up a two way communication channel with billions of underserved people around the world. As a result a huge number of products and services have sprung up in in sectors including finance, health, education and agriculture. Crucially, when an individual conducts a transaction, for example through sending money to a friend or requesting agronomy advice, it generates a data trail that can be used to diagnose how a particular service is being used.

What constitutes a user?

Often an evaluation of a service like this will start by looking at a random sample of the user base. However, many services have a large number of registered ‘users’ that aren’t actually using the service. While registration does indicate some level of interest or interaction with a service, if we are considering how satisfied with it a person is (or ultimately what impact it is having on their lives) we may well be asking a nonsensical question. It is perfectly possible that this ‘users’ only interaction with the service was through an event they attended a couple of months ago, or a one-off response to that blast SMS marketing message that is now forgotten.

In reality, the vast majority of people will not become the ‘ideal user’ (often synonymous with ‘regular user’) of a given service. In our experience, early stage mobile products and services often have less than 20% of users interacting with the service in the way envisioned by the creators. Active user bases take a large amount of work to create and need to be cultivated through extensive testing and refinement of the product. We think that this is something that M&E approaches of mobile products and services often miss. The assumption often seems to be that the baselines and midlines are samples taken from an already active user base.

Our approach

As mentioned, one of the most exciting elements of mobile from an M&E point of view, are the vast quantities of data that are generated by users, simply through their interaction with a service. Through segmentation and analysis, we can begin to see how users are actually interacting with the service. The first stage in this process – segmentation – is illustrated in the above diagram, drawn from our recent study of a mobile agriculture service.

If we have a way of understanding which users are at which phase of this journey, we can start to ask questions that glean real insights. For example, for someone who has used the service once, ask ‘which of the following best describes why you haven’t used the service again?’. This is clearly important for the service provider, who is able to learn about why bottlenecks have emerged in the service. However, this could also be the starting point for evaluators determining the impact a particular service is having.

In the case of the actual agriculture service mentioned previously, after segmenting and analysing the user base, we found that the majority of ‘users’ had registered for the service but had never actually used the service (see below).

gsma_2

Why is this relevant to the M&E Sector?

Participants at M&E Tech were very positive about the approach we presented – in feedback validating the fact that a lack of frameworks or approaches to analysing data was the biggest constraint. However, participants raised three issues with this approach:

  1. That it is difficult for those working in the M&E field to get a hold of the kind of data that the GSMA can.
  2. Analysing transactional data can only tell us part of the story and that other approaches are necessary.
  3. Participants felt they needed further discussion to understand how what they did in M&E fit in with this kind of approach.

With regard to the first point, the GSMA clearly occupies a privileged position with regards to access to data. However, we are confident that third parties have a much better chance of working with operators if they align their requests with the operator’s interests.

Secondly, this kind of data is only a piece of the puzzle. Clearly there are important things that we can’t learn from this data, a lot of which will require us to find out through further qualitative research. However, understanding how users are actually interacting with these kinds of services should be seen as the starting point for understanding the impact a service is having. Qualitative research will only benefit from being built on the kinds of insights that emerge from the method presented above.

Finally, in the workshop we began to tease out how the M&E sector could utilise (and possibly build upon) this approach. While the workshop was a huge success, we only began to tackle this. What is clear, is that M&E professionals can deepen our understanding of the kinds of social and economic impact of mobile for development services are having.

As a result, we are keen to hear more from you to see how the work that we are undertaking in this area can complement your own. We are particularly interested to understand:

  • What existing practices in monitoring and evaluating of M4D services we could best integrate with this kind of mobile data analysis
  • Where you’d like to see GSMA (and others) develop work in this area, and who do you think we should work with

If you have a view on the above, or have any questions, then please share your ideas here.

Note: we will be putting out a report soon entitled ‘Utilizing the value of data across mobile for development’, more recently we released a set of 10 mobile for development case studies where the topic of data use is highlighted. Please let us know via the same form, if you are interesting in receiving upcoming research in this area.


About GSMA
As the representative of 800 operators in over 220 countries around the world, the GSMA has strong links with operators around the world. The Mobile for Development (M4D) Impact team exists to spark, nurture, and scale a vibrant and thriving community of innovators in the developing world mobile ecosystem that serve underserved consumer segments, driving positive socio-economic change through mobile.

Part of our work involves working closely with some of our members in order to help drive mobile services with social impact to scale. We currently working with operators in order to help them to unlock the power of this data, in order to help drive service improvements. If you would like to find out more, please get in touch.

Filed Under: Agriculture, Management
More About: , , , ,

Written by
This Guest Post is an ICTworks community knowledge-sharing effort. We actively solicit original content and search for and re-publish quality ICT-related posts we find online. Please suggest a post (even your own) to add to our collective insight.
Stay Current with ICTworksGet Regular Updates via Email

Sorry, the comment form is closed at this time.