Did you know you could set up a Google Scholar Alert for academic papers that mention your favorite topic? I have one set up for “ICT4D” just to see what research is new or interesting in my favorite profession.
I usually find gems like Malawi Television White Spaces (TVWS) Pilot Network Performance Analysis or Brokers of Public Access: Infomediaries, infomediation, and ICTs in public access venues, and occasionally, I see a title that just jumps out at me.
That’s what happened when I saw this one: Taarifa: Improving Public Service Provision in the Developing World Through a Crowd-sourced Location Based Reporting Application. I expected Mark Iliffe, Giuseppe Sollazzo, Jeremy Morley, and Robert Houghton to take me through a deep analysis of the impact ICT4D can have on government services.
Instead I got everything wrong in ICT4D academia.
Now let’s not personify this as unique to our 4 protagonists. They mean well, I’m sure, and they did write a good paper. It’s just not useful to us in any way, except as a demonstration of how not to think about an ICT4D project. They committed the 3 deadly sins of ICT4D with stunning accuracy.
- Focusing on Westerners: The paper starts with a long detail about a Random Hacks of Kindness hackathon that was the start of the Taarifa software. Nice enough, but they spent 2 whole pages on it – almost 1/3 of the total report.
- Focusing on Software Developers: They spend the next 2 pages of the report going into detail around Ushahidi developers and who did or didn’t commit code to github. Okay… interesting to a point, but do we really care about the standard deviation of commits per contributor?
- Forgetting about Community Members: Remember the title of the paper? Well exactly 1.3 pages of the report, less than 1/3 of the total, was spent talking about the actual community impact. You know, if crowd-sourced location based reporting can improve public service provision? And they didn’t even answer the question!
I am saddened that the authors seemed more interested in the complete rewrite of Taarifa, using the Django framework, than if community members could use technology to change their relationship with government.
Most egregious of all, is that the paper’s authors knew what they should’ve written about. In the “Related Work” section, they correctly identify a “definite gap in the research would be an ethnographic study of when a crisis occurs, observing and reporting the ’value chain’ and how the data is used,” yet they didn’t spend a second focusing on that – exactly what I thought the paper was about.
Ugh. Better luck next time guys. Maybe if you had a more gender-balanced team…
Wayan, this speaks to one of my pet peeves not only involving academic papers but also academic presentations at conferences. Thanks to you for bringing this up and even mentioning how the paper might’ve benefited from a gender balance. You know how to speak to my ICT4D heart!
Well there is always this academic presentation to wake you up.
I’d bet far more papers and project ignore, don’t understand and don’t take into account the engineering in ICT4D – and that sin is absolutely as bad in my opinion.
Lines of code is a metric of the work effort needed to make software, and how many developers per line of code in a type of project tells us how many human resources we need to accomplish a task. The discussion is interesting to me as it leads me to the wonder why not construct it as a plugin, minimize the development effort, and add a way in Ushahidi to hide functionality not relevant to a particular instance. A big mistake to learn from, though apparently not flagged up as such – “not invented here syndrome”.
If a project is scoped such that the scenario would not involve any need/desire/improvement to change the software or hardware, then no need to include software engineers. Given that certainty, predictability, and long practice histories are pretty unusual in ICT4D; I’d say the bigger, worse, and more prevalent sin is actually downplaying the engineering. Good engineers who understand both the tech and the user are critical to making things that work in a reasonable manner.
Mike makes good points. The title of the article is off, maybe way off, but it doesn’t follow that the whole piece is worthless. We need as much detail out there as possible about donor-funded projects involving software development.
It’s also published in a journal concerned with FOSS used for Geospatial projects, so shouldn’t be expected to be an ethnography of a detailed use case. The journal’s focus, authors’ backgrounds, and viable options for publishing may have played a larger role than negligence.
Seems like a good candidate for someone else to pick up and dive in further.
Then they should have called the paper “An Analysis of Software Development Efforts to Create Another One-Off Pilot” and I would have safely ignored it. But to use that title, I expect to learn how it improved public services (or not).
LOL I had not seen this before. That’s on par with the Charlie Brown teacher presentations!
This is not an ICT4D paper. Agree with Scott about the venue of the publication – this isn’t a fair or useful assessment.
Then they should not have given their paper such a lofty ICT4D-centric title.
To their credit, this paper is actually readable. There are so many ICT4D papers I try to start, but I can’t even get through the abstract because they are so boring and obtuse.
Let’s not go overboard in academic bashing. 🙂
Here are four venues that I highly recommend for keeping up with the academic research in ICT4D. These venues have among the most competitive acceptance thresholds for ICT4D academic research:
ITID – The Journal of Information Technology and International Development by USC Press (previously, MIT Press)
ICTD – Proceedings of the IEEE/ACM International Conference on Information & Communication Technologies and Development. This series of conferences started in 2006, run on an approx. 18-month cycle, is the primary academic research conference for ICT4D, and the next conference takes place in May 2015.
CHI – Proceedings of the ACM International Conference on Human Factors in Computing Systems. This series of annual conferences started in the early 1980s, is the largest annual venue for human-computer interaction researchers that attracts approx. 4000 attendees each year, and has a sizeable number of ICT4D papers presented every year starting 2007. Many papers in the CHI proceedings exceed the quality of a typical peer-reviewed journal article.
DEV – Proceedings of the ACM DEV symposium, which has more of an engineering focus compared to ICTD, ITID and sometimes CHI.
(Full disclosure: I am a regular reviewer for DEV, ITID and ICTD; have been a reviewer and meta-reviewer for CHI; co-founded the ICT4D special interest group at CHI; and am co-chair of the program committee for ICTD 2015)
And for those who wish I had focused on good research papers, I have and I do. I give you years of them here: http://www.ictworks.org/tag/research-report/
According to Leland Smith, this is Revi Sterling’s case study in awful ICT4D academia: Overcoming Blind Spots in Interaction Design: A Case Study in Designing for African AIDS Orphan Care Communities
As someone who does a multitude of hackathons around social good, I understand these complications (tho I don’t like to see them as complaints). See hackathonfaq.com for more on that. People tend to want their skills to be applicable to making the real world better, including those with technical skills. It would be easy to mistake this paper for that – with the focus on hackathons, on the code, and on the coding community.. except considering the venue for which the paper was published. Context is important in this, as in development and community work.
I’ll write a fuller blog entry in direct response to this – AFTER I’ve gotten some sleep (just finished 11 hours in the car ride back, have 22 hours of flights ahead of me, a gathering at which to disseminate the results of using tech to bolster existing efforts… after a week of working with Tanzanian software developers and community owned water supply organizations), but I hope you’ll check out the updates at taarifa.org.
– Willow, one of the Taarifa team (and also lending to gender balance – thanks for looking out for us!)
Wayan, I am clearly too self-interested in this and I’d like to avoid a flaming response, so I’ll just add a couple of remarks to Willow’s comments.
The one thing I’d like to mention, is that a publication needs to consider its intended audience, something your analysis doesn’t focus on. This was submitted/presented at the FOSS4G conference. This stands for “Free and Open Source Software for Geo”. Hence, the “software development” cut we had to give it. The OS Geo Journal is dedicated to proceedings from that conference.
The paper’s title could have been changed? Well, why not, you are right. I agree it’s a bit off-track compared to the contents of the paper (although, in my opinion, not in a massively relevant way). No peer-reviewer made any comment about the title, otherwise we would have considered it.
In any case, the process involved in a conference paper approval, as you might know, can result often (and did in this case) in a paper that was slightly differently scoped from the original submission.