ICTworks

After five years researching the effectiveness of non-profit organizations (NPOs) in the USA, Stanford University lecturer Kathleen Kelly Janus found that while 75% of NPOs collect data, only 6% feel they are using it effectively. (Just to be clear, these were not all tech organizations.)

She suggests the reason is because they don’t have a data culture. In other words, they need to cultivate “a deep, organization-wide comfort level with using metrics to maximize social impact.” Or, in ICT4D speak, they need to be data-driven.

Perhaps NPOs feel that if they start collecting, analysing and using big data, that need will be satisfied. But one cloud server of big data does not a data culture make. While big data can be a powerful tool for development, there are three other data types that could significantly improve the impact of any ICT4D intervention.

Thick data

Technology ethnographer, Tricia Wang, warns us about the dangers of only looking to big data for the answers, of only trusting large sets of quantitative data without a human perspective. She proposes that big data must be supplemented with “thick data,” which is qualitative data gathered by spending time with people.

Big data excels at quantifying very specific environments – like delivery logistics or genetic code – and doing so at scale. But humans are complex and so are the changing contexts in which they live (especially true for ICT4D constituents). Big data can miss the nuances of the human factor and portray an incomplete picture.

As a real-life example, in 2009 Wang joined Nokia to try to understand the mobile phone market in China. She observed, talked to, and lived amongst low-income people and quickly realised that – despite their financial constraints – they were aspiring to own a smartphone. Some of them would spend half of their monthly income to buy one.

But the sample was small, the data not big, and Nokia was not convinced. Nokia’s own big data was not telling the full story – it was missing thick data, which led to catastrophic consequences for the company.

Adjacent data

Sometimes there is value in overlaying data from other sources onto your own to provide new insights. Let’s call this “adjacent data”. Janus provides the case of Row New York, an organization that pairs rigorous athletic training with tutoring and other academic support to empower youth from under-resourced communities.

To measure success, Row started by tracking metrics like the number of participants, growth, and fitness levels. But how could they track determination or “grit” – attributes of resilient people?

They started recording both attendance and daily weather conditions to show which students were still showing up to row even when it was 4C degrees and raining. “Those indicators of grit tracked with students who were demonstrating academic and life success, proving that [Row’s] intervention was improving those students’ outcomes.”

Pinpointing adjacent data requires thinking outside of the box. Maybe reading Malcom Gladwell or Freakonomics will provide creative inspiration for finding those hidden data connectors.

Lean data

Lastly, there is a real risk in just hoovering up every possible data point in the hope that the answers to increased impact and operational efficiencies will emerge. That’s not referring only to the data security and privacy risks related to the sponge approach. Rather, that’s because it’s easy to drown in data.

Most ICT4D initiatives don’t have the tech or the people to meaningfully process the stuff. Too much data can overwhelm, not reveal insights. The challenge is gathering just enough data, just the data we need – let’s call this the “lean data”. When it comes to data, more is not better, just right is better. In fact, big data can be lean. It’s not about quantity but rather selectiveness.

Lean data is defined by the goals of the initiative and its success metrics. Measure enough to meet those needs. When I was head of mobile at Pearson South Africa’s Innovation Lab, we were developing an assessment app for high school learners called X-kit Achieve Mobile.

With the team we brainstormed the data we needed to serve our goals and those of the student and teacher users. We threw in quite a lot of extra bits based on “Hmm, that would be cool to know, let’s put it in a dashboard.”

The company was also preparing to report publicly on its educational impact, so certain data points were being collected by all digital products. Having a common data dictionary and reporting matrix is something worth considering if you’re implementing more than one product.

After building the app we only really used about 20% of all the reports and dashboards. Only as we iterated did we discover new reports that we actually needed. The fact is that data is seductive, it brings out the hoarder in all of us. We should resist and only take what we need.

So, perhaps the path to building a data culture is to always have thick data, be creative about using adjacent data, and keep all data lean.

Image: CC by janholmquist