News flash: most nonprofits are terrible at software development, open-sourcing is not a sustainability plan, and constantly re-inventing the wheel is a poor way to spend limited resources.
So why don’t international development projects leverage more off-the-shelf, private-sector technology solutions? After all, the private technology sector has established mechanisms for managing technical talent, sharing development costs across many users with similar needs, and maintaining and supporting technology solutions over time.
Customization Over Efficiency
One common answer is that development projects are poorly served by off-the-shelf solutions, the vast majority of which were designed to meet needs in other sectors. But even presuming that the international development sector has unique software needs, it would seem large enough to sustain a thriving market of small-to-medium-sized software vendors. There is certainly enough money being spent.
Much of the money currently being spent, however, is being diffused on staff, services, and one-off efforts. Even would-be software companies with a preference to serve common needs are inexorably pulled toward focusing on the lucrative business of fulfilling idiosyncratic project needs on a services basis, rather than packaging and selling an off-the-shelf product.
When one project will pay $100,000 for something that is customized to their every whim, for example, it is difficult to focus instead on a product that sells for $1,000 and might meet the needs of hundreds of projects. Particularly in the early stages of a technology business, it is very difficult to resist the $100,000 in favor of the $1,000. The financial incentives are just too great.
For their part, implementing project teams often seem to place a high value on complete control, on the ability to fully customize a technology solution. While it’s not entirely clear whether these teams fully realize the true cost to build, maintain, and support high-quality software (even open-source), they often operate with large budgets very conservatively set at a much earlier proposal stage; in some sense, by the time of implementation, cost is not really a factor – budget is.
Conditional on being able to secure donor funding, in fact, larger budgets and staffs might be strictly preferred to smaller budgets and staffs. This phenomenon that more-expensive can be preferred to less-expensive is one you often see in the public and nonprofit sectors where “other people’s money” is being spent. You see it far less in the private sector, at least when market forces are operating well.
The Role of Donors
This brings us to what seems to be an important factor: donors. Anybody who works in development will readily admit (and probably bemoan) the fact that donor funding dynamics drive incentives all through the development sector. Though it’s less widely recognized, many working on the donor side do try mightily to combat the unintended and undesirable incentives that result from their funding. But in this case of what appears as a preference for staff and services over off-the-shelf software – where hundreds of projects seem to very expensively re-invent the wheel – are the incentives strictly unintended?
Donor egos or fundraising incentives could lead them to focus on bleeding-edge innovation, quick pilots, and splashy conferences over more mundane investment in technology solutions that efficiently meet project needs. They may also have an implicit (sometimes explicit!) bias against the private sector: given the choice between spending $1,000 on a private-sector technology solution or $10,000 on a custom- or nonprofit-built solution, they may actually prefer the latter option. And they may truly believe that by spending more money and focusing it on a nonprofit or open-source solution, they are acting in the public interest.
But by now, donors should be figuring out that publishing a bunch of code that kind of works most of the time as one of 21+ million GitHub repositories is no guarantee that it will be maintained, used, or even looked at ever again. In fact, the odds are very much against any of those things happening. Building – and then maintaining – high-quality software is a difficult, tedious, continuous business. Getting something started and then open-sourcing it is just wishful thinking.
Another Way
In my view, the public interest would be better served by having more development projects spend money more efficiently, by allowing a more rational, free-market dynamic to support high-quality software firms that meet the shared needs of many projects.
While individual projects would give up some degree of control in principle, the reality is that software supported by revenues from many projects would come to serve those projects considerably better than any custom-built solution ever could. Truly idiosyncratic needs might not be met as well (and is that really a bad thing?), but there is far more that is common across projects than what is distinct – and software vendors would undoubtedly release toolkits, API’s, and other avenues for project customization.
I believe the social gains would be considerable. But what do you think? Is the ICT4D marketplace is working well? Or should we have wholesale change?
Christopher Robert is founder and CEO at Dobility, Inc., which derives 95% of its revenue from SurveyCTO, an open-source-based professional product that meets the needs of many hundreds of projects.
Selecting a good software platform is a critical success factor. Some Organizations hire firms to develop software which is not easily customizable or flexible. I highly recommend the Drupal platform which has matured and grown by leaps and bounds. It’s very easy to integrate it with other platforms through its application program interface (API). It has powerful features such as views which can help an organization create custom reports based on different data sets. Map lovers can be blown away by integration with open layers. For a better user experience faceted search can be deployed with apache solr, as long as the content is properly classified and indexed using a combination of tags, taxonomy and terms.
Saying that other organizations shold stop building software is lIke teling them to stop iterating and trying to improve. Many of the great products that exist today today came from people in development organizations who recognized niche problems which the existing set of tools didn’t quite satisfy. Did we have an sms platform before TextIt? Yes, it was called Frontline. Did the world have a web based email client before gmail? Yes it was called yahoo mail. Thanfully the world kept iterating and building better tools after the first groundbreaking products came out.
Yes, everyone should have a keen eye for re-use and people should make better use of APIs from existing tools to build their apps but we also need to encourage people to keep trying and sometimes failing with new tools.
Yes, thanks, those are fair points. I certainly didn’t mean to suggest that people should stop demanding better solutions and building them as necessary. My own company and product were built purely out of dissatisfaction with what was out there combined with a belief that we could do better. I also believe that competition is healthy and produces better results. My argument was more against one-off solutions and/or non-one-off solutions without any kind of realistic sustainability plan.
Great post Christopher and I couldn’t agree any more. In my minimal experience, donors want to do the right thing with software development and leverage SaaS or open source tools. However, many human and organizational barriers exist that you touched on. The effort to make a splash and focus on the less sexy elements is hard, but the right thing to do. Taking into account total cost of ownership and what happens after the pilots is less of a focus in the ICT4D world. I’d also note that because there are 21M GitHub repositories, it is hard to find projects that were done well and documented well.
Nevertheless, I believe the industry recognizes that and is moving forward, but at a pace that is slower than anyone would like. In an ideal world, every software development project that is ICT4D would have a landscape of previous solutions/tools conducted with honest estimates about what it will take to extend or modify those tools.
There are many challenges that remain, but we are all moving forward. Thanks for these great thoughts.
Thanks for your comments! I realize that we SaaS folks present particular challenges for organizations with grant and procurement systems not at all prepared to deal with recurring fees for software services — so that surely doesn’t help.
And when it comes to finding open-source projects that are done and documented well, it can be so darn hard to even figure out the extent to which a project has become, for all practical purposes, defunct. Many times, there will be some well-funded dissemination/communications team that produces a flashy website and the project repo will be all set up and looking nice, but then it can be very hard to know whether the core team has already scattered or if even the 1.0 bugs were ever truly pounded out. Maybe we’ll get better about aggregating information about truly active, high-quality projects, so that discerning those will become less difficult.
Chris
Totally agree with 90% of this – we produce tech for the Development sector and most of the time our customers (NGOs, donors etc.) do want something custom-built, and start off thinking their scenario is unique and they need every feature they want.
We use an Agile approach which helps to get them to understand the costs and trade-offs, but they usually still think they need “their own tool”.
Where I think you miss something tough is that this is not a private vs open-source issue, it’s more fundamental than that. There are tons of great open-source tools that people could download and use for free… And actually they are generally a better idea than proprietarty ones as they at least make possible the idea that “oh you have ONE unique weird thing, sure we can build THAT bit on top for you”, rather than a choice between doing it all or using something not fit for purpose.
Where possible, we do this – start with an existing open source tool/framework and customise this.
However it’s interesting that a lot of the time this just doesn’t seem to meet an organisation’s needs, and we often find ourselves being asked to build a system from scratch. It seems Pareos 80/20 rule hasn’t hit the Development sector yet unfortunately – usually the additional 80% of time and effort and cost is spent building 20% (if that, maybe 5%!) of additional functionality, which seems a shame and a massive waste of resources.
I’d love to hear thoghts on how to combat this – NGOs are notoriously bad at comign together to co-create solutions; the existing off-the-shelf solutions don’t work quite well enough; and building from scartch is a collosal waste of money….. So what do we try? 🙂
Thanks, Matt. You raise some excellent points.
For our platform, I’ll tell you what we’ve tried — but I should say up front that it hasn’t been terribly successful. For the people who say “this does 95% of what I want but I need this one thing added or changed,” we say, sure, we’ll do that for you. We then offer two options: (1) we change the platform for everybody and share the cost of development (if we think that it will be valuable for other users), or (2) we change the platform and also open-source the change so that it becomes part of the Open Data Kit open-source platform. We charge essentially “full freight” for option 2, but for option 1 we only ask for a subsidy that declines in price depending on how much value we think the change adds to the core platform. We’re an ultra-lean, ultra-efficient development shop (still quite small) and we know our platform extremely well, so we can promise most changes within 2-4 weeks and we charge an absurdly small amount.
Even when we charge the “full freight” cost, it’s generally a small fraction of what it would cost for the user to develop the “5% extra needed” in-house, and it’s orders of magnitude less than what it would cost them to develop their own custom solution (i.e., to develop the full 100% instead of just the 5%). And ongoing maintenance costs for us building something into the platform are essentially zero for the user: we document and support the change, extend and adapt it over time as necessary, and generally keep it part of a living, supported platform. Contrast that even with the case of open source, where if you forked the 95% and added/changed the 5%, you’d be forever responsible for maintaining that fork. To me, what we’re offering users is a huge improvement in value-for-money — but still development users seem to prefer rolling their own far more often than not.
As an economist, I see so much potential for “gains from trade” here, all being left on the table. It seems to me that there’s something more fundamental going on, like perhaps a lack of trust that we’ll keep supplying the platform at a reasonable cost, that we’ll keep things running smoothly, etc. Maybe organizations just really want full control and they’re willing to pay almost anything to have it; maybe they just won’t trust a private-sector provider because there are too many shady providers or too much churn in the industry. I’m not sure.
If the issue is one of trust, is open-source the best solution?
Thanks, Matt. You raise some excellent points.
For our platform, I’ll tell you what we’ve tried — but I should say up front that it hasn’t been terribly successful. For the people who say “this does 95% of what I want but I need this one thing added or changed,” we say, sure, we’ll do that for you. We then offer two options: (1) we change the platform for everybody and share the cost of development, if we think that it will be valuable for other users; or (2) we change the platform for them only and also open-source the change so that it becomes part of the Open Data Kit open-source platform. We charge essentially “full freight” for option 2, but for option 1 we only ask for a subsidy that declines in price depending on how much value we think the change adds to the core platform. We’re an ultra-lean, ultra-efficient development shop (still quite small) and we know our platform extremely well, so we can promise most changes within 2-4 weeks and we charge an absurdly small amount.
Even when we charge the “full freight” cost, it’s generally a small fraction of what it would cost for the user to develop the “5% extra needed” in-house, and it’s orders of magnitude less than what it would cost them to develop their own custom solution (i.e., to develop the full 100% instead of just the 5%). And ongoing maintenance costs for us building something into the platform are essentially zero for the user: we document and support the change, extend and adapt it over time as necessary, and generally keep it part of a living, supported platform. Contrast that even with the case of open source, where if you forked the 95% and added/changed the 5%, you’d be forever responsible for maintaining that fork. To me, what we’re offering users is a huge improvement in value-for-money — but still development users seem to prefer rolling their own far more often than not.
As an economist, I see so much potential for “gains from trade” here, all being left on the table. It seems to me that there’s something more fundamental going on, like perhaps a lack of trust that we’ll keep supplying the platform at a reasonable cost, that we’ll keep things running smoothly, etc. Maybe organizations just really want full control and they’re willing to pay almost anything to have it; maybe they just won’t trust a private-sector provider because there are too many shady providers or too much churn in the industry. I’m not sure.
If the issue is one of trust, is open-source the best solution?
Sounds very sensible.
Interestingly we recetly had a clash with a client over IP rights. Initially they were adamant that they couldn’t possibly pay for something and not own all the IP.
On sharing our thoughts as to why open-sourcing it was a better idea, they surprisingly came round entirely and dropped the IP clause completely.
The reasoning we shared is below, in case anyone finds it helpful.
• All our work involves adapting existing open-source frameworks and re-using our own code and libraries, as well as writing new material for a specific project. In this context it is difficult, perhaps impossible, to grant IP to the entire deliverable. While we could, in theory, grant the IP to the client on the new code, we would be unwilling to grant the IP on our pre-existing libraries and unable to grant the IP on other people’s libraries and frameworks.
• In fact many open-source frameworks demand in their licensing agreements that any software created using their code must also be open-sourced. While this is covered in the licensing clause, it is potentially at odds with granting IP over the new product although I suspect some re-wording could clarify this.
• As open-source developers, it is also important to us from an ethical position that we make as much of our work as possible available to the wider open-source community, especially given we work in the Development sector and are usually dealing with public funds. The way our current contract is worded allows us to do this where handing over IP fully to a 3rd party would restrict us from doing so.
Cheers
Matt
PS.
Chris (and anyone else interested) – please drop me an email ([email protected]) there’s a survey I’m doing on charging models I’d like to share with you 🙂