⇓ More from ICTworks

3 Systematic Reasons Why M&E Technology Projects Fail

By Guest Writer on June 26, 2015

6936496722_7426bc233d_o-2

Having worked for ten years in tech-enabled monitoring, evaluation and learning (MEL), I think we need to address three systemic issues: complexity, interoperability and affordability. This is important because money and opportunities are being wasted through a lack of joined-up thinking. I have some ideas about what we can do, but first I’ll state the problems as I see them:

  1. Complexity – MEL technology is a complex and confusing domain
    For an NGO worker without a background in IT, deciding which tech tools to use for MEL is an intimidating challenge. How does she find out what is available? How does she evaluate the options? When is it appropriate to commission her own?
  2. Cost – tech-enabled MEL is expensive
    Licenses, consultancy, support – the costs add up. Our NGO worker finds it hard to fundraise or allocate internal resources because IT is seen as a low priority. Tech-enabled MEL may not be sustainable for her smaller, poorer NGO.
  3. Interoperability – tools don’t work together
    Our NGO worker has chosen the Acme Tool for planning and monitoring; now she discovers that a sister organisation is using a different one. The two organisations need to share data but their respective tools can’t speak to each other.

How can we make it easier to navigate the complex domain of MEL technology and find appropriate tools?

When Aptivate facilitated an open-space meeting for the Bond MEL and Tech4Dev groups one session talked about creating a map of tech tools that can be used at each stage of the planning cycle. I can see a place for a TripAdvisor-style platform where users of MEL tech tools share their experiences, newcomers benefit from reading this, suppliers get feedback, and everyone has the opportunity to connect and collaborate.

How can we enable interoperability between tools and promote data sharing capabilities?

We could adopt an open standard for MEL data. It should be possible to codify MEL frameworks, indicators and monitoring data in a way that can be commonly interpreted by different tools. This could save a lot of time. A colleague who developed the open contracting standard with the World Bank, recently reminded me that a standard is only as good as its adoption. So if there already is an open standard for MEL data then let’s deal with the adoption problem.

How can we make MEL technology attainable and sustainable?

I think there is so much commonality in the needs for MEL technology. It should be possible to pool resources to build and maintain a set of excellent open-source tools that are accessible for poorer organisations. A recent report on tech-enabled MEL said that proprietary management information systems (MIS) often have high costs for training, installation and long-term service agreements; and developing a system in-house can require a large investment.

The same report argued that many organisations don’t have the skills to set up and maintain open-source tools themselves and that existing open-source MIS tools lack functionality and usability. At Aptivate we want to address this with our Kashana project.

Our vision is: a low cost, easy-to-use toolset for MEL that works on low-bandwidth connections; it should take the pain out of planning and monitoring, enabling you to spend more time on evaluation, learning, and delivering better results. The name is derived from the amharic word käshänä, meaning ‘do good work’. I like this because I don’t think you work in development to run IT projects or collate spreadsheets; you do it because you want to do good work, in every sense.

We want to run a Kashana pilot programme to test its viability and get user feedback. We are looking for organisations who use a structured MEL framework, such as a log frame. Would you like to be involved? If so, please email kashana@aptivate.org

George Flatters is Aptivate’s leading expert on technology-enabled monitoring, evaluation and learning and this article is based on an Aptivate blog entry.

Filed Under: Software
More About: , , , , , ,

This Guest Post is an ICTworks community knowledge-sharing effort. We actively solicit original content and search for and re-publish quality ICT-related posts we find online. Please suggest a post (even your own) to add to our collective insight.
Stay Current with ICTworksGet Regular Updates via Email

2 Comments to “3 Systematic Reasons Why M&E Technology Projects Fail”

  1. David McCann says:

    This reminds me a lot of https://xkcd.com/927/: the solution to creating an open-source M&E tool that everyone can use is to start building a new one, rather than contributing to one of the many existing open-source communities out there (RapidPro, Frontline and Kobo Collect just to name a few that are already rather robust)? Sounds like being a part of the interoperability problem, not part of the solution.

  2. Mike Dawson says:

    I find a lot of organizations have taken up very simple user friendly tech enabled MEL in the form of google docs and MS/Open office spreadsheets. I would generally agree with David’s point here.

    When it comes to data portability: as long as the data is either sitting in an open source system or in some kind of format with a defined standard then yes it can be moved: but unless there is sufficient demand there won’t be anyone who has gone and coded a migration tool. How many times do people still get stuck moving email and address books between common programs?

    Yes organizations will have to budget technical expertise to get more complex tasks done: unless they want to budget for typewriters, calculators and photocopiers instead. Adopting open standards is definitely a good thing: but with each project tracking distinctly different data what standard could possibly handle all this variety beyond a standard database format (e.g. SQL)? Perhaps when donor organizations write policies they should write data specs too along with a reference implementation?

    Another concept which is rather confusing is the idea that monitoring and evaluation for the non profit sector is that different to the commercial sector and requires us to throw out all the key performance indicator, survey, etc tools (open source and otherwise) built there.

    What’s very questionable in this piece is the idea that current open source tools offer “limited functionality” with the reference of what two individuals said. OpenERP, OpenBravo, ODK, LimeSurvey etc can hardly be described as having ‘limited functionality’.

    Where I agree: having a tripadvisor like site to compare different tools for different purposes: that makes a lot of sense. There was once a survey tool picking wizard site that compared various tools: but I remember it disappearing. Nethope has their solutions center but I’d find it tricky to compare things quickly for a specific purpose.

    Also: if it’s open source: where’s the code repo? Can’t find it by googling and it’s not in Aptivate Github ( https://github.com/aptivate/ ).