⇓ More from ICTworks

9 Considerations for Seamless Mobile MERL Solutions

By Praekelt Org on January 2, 2017

With mHealth a fast growing component of health delivery systems, many organisations are developing mobile and digital solutions to tackle health problems around the world. Presentations at MERL 2016 Conference showed that many are also using mobile and digital tools to analyze and monitor programs.

Unfortunately, there is often a disconnect between the solutions themselves and the MERL. Frequently, programs still evaluate apps, websites, and SMS services through traditional door-to-door surveys or phone calls. But there is value in doing MERL through the very technology that you are evaluating – or as we call it Seamless MERL.

Seamless MERL means that product and evaluation are tied directly together, allowing direct access to and providing a smooth experience for users, where services and research are combined.

Our Seamless MERL experience

The Institute for Reproductive Health at Georgetown University (IRH), with partner Cycle Technologies, has developed two mobile products for family planning:  CycleBeads® app and Dot™ app. Praekelt.org has developed two mobile and digital products for family planning, maternal health and learning: MomConnect and Girl Effect Mobile. All four products demonstrate the potential of Seamless MERL to address unique research questions:

  • CycleBeads app: It has already gone through extensive testing, but as user numbers grows and it expands into new geographic markets, IRH wants to find out more about who are users are and how they engage with the app.
  • MomConnect: Since launch, more than a million new and expectant mothers have signed up to MomConnect receive stage-based, personalised health messages delivered via SMS. The message sets have been created to drive behaviour change and Praekelt need to learn what combination of content, format and frequency is most successful.
  • Dot app: To assess the efficacy of Dot to prevent pregnancy, data collection must integrate low burden processes to capture personal, and often sensitive, data. Engagement with both the app and data collection tools must be secure and interesting to retain participants for >13 months. The method and results must meet high standards.
  • Girl Effect mobile: Girl Effect has created an entertaining, informative and free-to-access website for girls in over 40 countries and in more than 15 languages. Conventional tracking mechanisms, like Google Analytics, track how many users are visiting the site, how long they’re staying and what pieces of content are most popular. But these mechanisms do not allow us to understand why girls (and boys) are visiting or to track changing levels of knowledge.

In Seamless MERL design, consider this:

There’s no single way to create more seamless MERL experiences for users. But, by comparing our experiences across these four case studies, we’ve developed a list of essential considerations

  1. Set a research/ monitoring goal upfront: There are many reasons why you may want to do research or monitoring on your digital product. Refining the questions you want answered and the purpose of your research is the first step. MAke it a living document that you revisit as data collection proceeds.
  2. Be smart about interaction frequency: It can be a difficult balance. Too infrequently, and you don’t get good data.Too often, and you risk annoying your users (which can reduce interactions, and reduce the chances of good data). Look at usage patterns and sequencing; most digital products see a steep drop-off of users after the first month, so ask the most important questions at the start.
  3. Consider ethics: Seamless MERL doesn’t avoid the normal ethical research questions, and in some ways, it creates more. For example, once a question is sent, you can’t control who has the phone or the user’s privacy settings, which is particularly important for sensitive topics, such as family planning usage or HIV/ AIDS status.Identify the possible risks, mitigate them where you can, and disclose them to the user.
  4. Choose what data you need: Questions that are short, engaging, and easily understood in the local context are the most successful–that’s true with any MERL. However, with digital MERL, expect to ruthlessly decide what data is most important. Response rates tend to drop after a certain number of questions.
  5. Protect user data: Seamless MERL can create vulnerabilities in data collection and storage. While hacking is fairly rare, the larger the project, the greater the risk. Beyond the risks posed by malicious external parties, employ passwords and different levels of access to protect data within organisations too. Every piece of the system, from the user’s device to data storage to final analysis must be protected carefully
  6. Know the rules: MERL must follow the appropriate laws and regulations in every country. For example, patient privacy in the US is governed by HIPAA rules, while other countries mandate that data is stored in country, on local servers.
  7. Timing counts: A longer monitoring effort may give more more insights on use over time, but engaging users over a period of months or years can be difficult. Strategies like gamification and providing offline user touchpoints can increase engagement and response rates. No matter what you do, users will drop off over time, so collect the more important answers first!
  8. Make it easy for your respondents: Technology is not always intuitive to use. If your survey requires users to navigate through complicated menus or use unfamiliar features, you’re going to be limited to only the most tech-savvy respondents. Accurately assess the capabilities of your respondents through piloting or focus groups and design accordingly, keeping in mind that many (most?) people have a low tolerance for technology-induced frustration.
  9. Choose your channel: SMS, through your app, online? This choice is especially relevant for services and products that engage with end users via multiple channels. Think about the potential burden on the user and the potential richness of the engagement to decide when you use which channel and for what purpose.

Mobile technologies can overcome many obstacles to reach new audiences. But with each additional point-of-contact and every subsequent request for information, we risk losing our users, limiting not just our MERL, but service engagement too.

It’s critical that we reduce the friction users experience as they consume and use our products. Our ultimate goal should be a seamless experience in which services and research are intertwined. The result, hopefully, are products that collect larger volumes of better quality data than ever before.

Authors: Nicki Ashcroft, Program Officer (IRH), Jonathan McKay, Director of Partnerships, Praekelt.org. Missed our session? Download our slide deck and case study notes here.

 

Filed Under: Healthcare
More About: , , , , , , , ,

Written by
We use open source technologies to deliver essential information and vital services to more than 100 million people in over 54 countries. Partnering with governments, NGOs and social enterprises, we provide our users with information, inspiration, and access to improve their health and wellbeing.
Stay Current with ICTworksGet Regular Updates via Email

One Comment to “9 Considerations for Seamless Mobile MERL Solutions”

  1. Veronica Olazabal, The Rockefeller Foundation says:

    Thanks for sharing your point of view on seamless MERL. I agree with your points about integrating data collection into the delivery of the program work itself for MRL. I do wonder though about the “E,” particularly around the piece on validation which is not addressed in your post. I think this is critical and often in MERL discussions there is not enough distinction made about the different strategies one would take to not only monitor progress but also validate (evaluate) the intended and unintended impacts of delivery. What are your thoughts?