Sadly, countless health chatbot projects crash and burn across low- and middle-income countries each year.
It’s not about artificial intelligence limitations, insufficient funding, or even poor internet connectivity. The truth is that most chatbot implementations fail because we’re building sophisticated technology solutions backwards.
The Frontier Technologies Hub recently released findings from four health chatbot pilots across Peru, Kenya, and Nigeria that should make every development professional pause.
These real-world implementations covered vaccine uptake, sexual health education, chronic disease management, and post-surgical care. They revealed issues with reliability, mixed impacts on workflows, poor user friendliness and lack of adeptness with local contexts.
The central insight? Chatbot innovation requires a whole systems approach rather than a technology-first mindset. Let me break down the four critical lessons that separate successful chatbot deployments from expensive failures.
Lesson 1: User Needs Must Drive Everything
This is where most projects go wrong from day one. We get excited about what ChatGPT can do and immediately start designing solutions around AI capabilities rather than user realities.
The FT Hub pilots discovered that despite all participants having mobile phones and data access, users lacked the digital skills required to interact with the chatbot.
- In Peru’s EmpatIA pilot, two participants required caregivers to help them use smartphones to access the healthcare chatbot.
- In Nigeria, even though English is an official language, a significant proportion of the population do not speak English as their first language and cannot read and write in English.
The pilots found that users struggled with lengthy text responses and technical medical language, requiring solutions to provide shorter responses in plain language and even integrate speech-to-text functionality.
The Kenya SRHR chatbot pilot provides a perfect case study in user-centered design done right.
Rather than assume young adults wanted sophisticated AI conversations, the team conducted extensive focus groups and discovered that users valued the chatbot as a confidential and judgment-free source of trustworthy, on-demand SRHR information. This insight drove them toward a structured, decision-tree approach rather than complex generative AI.
The lesson is clear: spend more time in communities understanding actual user behaviors, literacy levels, language preferences, and cultural contexts before writing a single line of code.
Lesson 2: Partnerships Determine Success
We cannot build effective health solutions without deep partnerships with local health systems. The most sophisticated chatbot is useless if it can’t connect users to actual care.
The EmpatIA pilot’s critical partnership with the Detecta Clinic enabled them to test and adapt the chatbot solution, both through trials with patients and via engagement and consultation with clinicians. This partnership was instrumental in navigating regulatory requirements and ensuring clinical accuracy.
However, the report reveals a frustrating pattern.
Pilots struggled to secure the partnership with public or public-private partnership institutions, who typically serve the most vulnerable individuals because approaches to decision making around digital initiatives are complex and emphasise the need for rigour and certainty that innovation pilots cannot provide upfront.
This creates a catch-22 that we need to solve systematically.
Public health systems need evidence of effectiveness before partnerships, but you need partnerships to generate that evidence. The most successful pilots established relationships with champions inside health systems who could advocate for piloting new approaches.
What’s particularly valuable is the finding that implementers wanted partnerships with those operating in similar contexts, who were likely faced by similar health system and contextual challenges rather than partnerships with technologists from High-Income Countries.
This suggests we need more South-South knowledge sharing and peer learning networks.
Lesson 3: Operational Integration to Scale
Most chatbot pilots treat operational integration as an afterthought, when it should be the primary design constraint.
The FT Hub pilots learned that successful chatbot deployment requires a clear vision for how their solution would fit within the context of wider operational systems and processes, including identifying what (if any) transformation might be needed to accommodate the new chatbot.
Consider the digital skills challenge for health workers themselves.
Pilots identified the need to consider the digital skills of health actors who might engage directly with the chatbot, or with data flowing from the chatbot, and the need to provide support and training on the use of technology in their working roles.
The most sophisticated finding involves the balance between AI efficiency and clinical responsibility.
None of the pilots used the chatbots to provide patient-specific health advice, or to diagnose patient conditions. Instead, the chatbots were used to provide access to general (non-patient-specific) health information. They established clear handoff points between chatbots and existing health system actors when patients needed expert care.
This operational thinking extends to data quality and clinical accuracy.
All pilots took care to engage experts and develop curated and accurate datasets that their chatbot could draw on requiring engagement of clinical experts to help assess and determine which data items (for example journal articles or reports) a chatbot should be trained on.
Lesson 4: Regulatory Environment as Strategic Enabler
Most technology projects treat regulation as a barrier to navigate around. The most successful health chatbot implementations treat regulatory frameworks as strategic advantages that enable trust and sustainability.
Those pilots with relationships to Institutional Review Boards (IRBs) felt that they helped them to test solutions and have confidence they were working in an ethical, responsible and inclusive manner, while those who did not use IRBs felt this inhibited their ability to make progress.
The global regulatory landscape is rapidly evolving.
What’s particularly relevant for LMIC contexts is that the relative lack of stringent data regulations in these regions creates a flexible environment that enables innovative approaches to data management and patient care while noting there is an opportunity for industry stakeholders and others to assist countries in tailoring their approaches.
The WHO’s recent guidance on AI ethics provides a framework that LMICs can adapt, emphasizing that governments from all countries must cooperatively lead efforts to effectively regulate the development and use of AI technologies.
The Risks of Getting This Wrong
When chatbot implementations ignore systems thinking, the consequences extend far beyond wasted resources. The FT Hub report identifies several critical risks that should concern every implementer:
- Digital Divide Amplification: Chatbots not implemented with due care run the risk of exacerbating inequities in access to healthcare, rather than addressing these inequities. If solutions only work for digitally literate, English-speaking, smartphone users, they systematically exclude the most vulnerable populations.
- Clinical Safety Failures: Without proper operational processes, there are significant risks that chatbots can provide inaccurate information and guidance—and lead to consequences, including individuals failing to seek the care they need. This is particularly dangerous when generative AI is assumed to be self-aware and capable of distinguishing between accurate and inaccurate information.
- Algorithmic Bias: If training data are not representative of different groups of users, and the different needs each may have in relation to health, there is a risk of ‘algorithm bias’ whereby the chatbot provides guidance that is accurate for some users but provide less relevant or potentially dangerous guidance for other groups.
- Healthcare Worker Displacement: AI implementation can create risks of dependency and in the longer-term lead to a ‘deskilling’ of the existing workforce, and/or send signals that there is less demand for skills.
Where To Go From Here?
The evidence is clear: successful health chatbot implementation in LMICs requires abandoning technology-first thinking in favor of systems-first design. This means starting with user research, building genuine partnerships, designing for operational integration, and leveraging regulatory frameworks as enablers rather than obstacles.
As the report concludes:
“By fostering shared learning, investing in enabling policy environments and grounding innovation in local realities, the next generation of health chatbots can advance equitable and resilient health systems.”
The choice is ours: continue building impressive demonstrations that fail at scale, or start doing the harder work of systems thinking that creates lasting health impact.
