Helping USAID ensure accountability and results through monitoring, evaluation, and learning (ME&L) presented a multitude of challenges this year. The COVID-19 pandemic required physical distancing, travel restrictions, and mandatory telework for all USAID ME&L contractors.
For example, the Feed the Future Global Program Evaluation for Effectiveness and Learning (PEEL) project – funded by USAID as part of Feed the Future, the U.S. Government’s global hunger and food security initiative- frequently works in fragile and conflict-affected areas with security risks and stakeholder access limitations. Yet a COVID-19 digital response required PEEL to rethink it’s remote performance evaluation in the South Kivu Province, DR Congo.
The country has poor Internet connectivity, especially in rural areas, and many farmers, particularly women, don’t have mobile phones. PEEL had to integrate new kinds of data collection methods with local consultants relying mostly on remote methods.
6 Lessons Learned in COVID-19 Remote Data Collection
Below are six lessons learned from the largely remote DR Congo evaluation. We also included learnings from two performance evaluations in Mali that had to be adapted for remote supervision due to political violence. Like the DRC, Mali also has extremely low Internet penetration by international standards.
1. Be flexible and open to change with contingency plans.
Largely remote data collection requires rethinking the entire workplan and why and how you do everything. It also requires planning a series of reflect-and-adapt moments to adjust and switch to contingency plans depending on how host country public health and safety mandates evolve. PEEL’s development of multiple contingency plans enabled the evaluation teams to pivot quickly to changing guidance and at times unexpected initial results.
2. Find workarounds to obtain the data you need.
Since March 2020, PEEL has primarily relied on online surveys and computer-assisted telephone interviewing (CATI) to gather quantitative data to limit in-person data collection in accordance with strict social distancing and safety guidelines. PEEL also has relied on Google Meet, Zoom, Skype, WhatsApp, and telephone interviews for one-on-one key informant interviews for qualitative data since in some cases stakeholders cannot meet in groups for focus group discussions and in most cases facilitators cannot travel to remote areas due to COVID-19 restrictions.
3. Remote data collection may affect the timeline.
PEEL found that no or limited need to travel doesn’t necessarily save time. Under workplans for traditional in-person evaluations, teams are in the field for data collection for a set, limited amount of time. Without this tight window for in-person data collection, it can be harder to get key informants to make themselves available on the schedule you seek. Moreover, it takes longer to interview stakeholders individually versus in groups, and additional time is also needed for training and overcoming communications technology challenges, including having to reschedule interviews if technologies fail.
4. Identify stakeholder groups to overcome digital divides.
In remote parts of DRC or Mali, it is critical to identify ways to include women, the less educated/lacking in digital literacy skills, and stakeholders who do not own phones in mobile data collection. PEEL’s solution was to identify leaders of producer or other community-based organizations, such as farmer cooperatives or village savings and lending associations, and work through them to identify and arrange interviews for otherwise hard-to-reach populations.
5. Invest heavily in upfront training for local consultants.
Even before COVID-19, PEEL relied on local consultants who speak local languages and are often best equipped to interview stakeholders. Travel restrictions and social distancing requirements, however, necessitate transitioning a larger share of responsibility for in-country data collection to local consultants. Without an expatriate team leader on the ground managing the evaluation in-country, PEEL invested extensively in training the local consultants remotely on workplan strategies, data collection instruments and technologies, reflect-and-adapt moments, safety guidelines, and potential contingencies before sending them out to the field.
6. Be open to new data sources in whatever ways are possible.
PEEL found that because you can’t do what is traditionally done, you have to be agile and open and prepared to do something entirely different. Each solution depends on the situation and options and technology available in that locale. For example, focus groups are great to engage people in discussions and flush out issues in participatory and comprehensive ways. In pre-COVID-19 times, focus groups also save time and can be more cost effective with each focus group reaching six to 10 people at a time.
PEEL saw that focus groups were the biggest challenge in doing remote data collection. Post-COVID-19, focus groups were a significant logistical challenge. They and other data collection processes need to be redesigned to avoid tarmac bias – focusing data collection in more easily accessible locations. PEEL has conducted focus group discussions by speaker phone when the facilitator couldn’t travel to remote areas.
PEEL is also exploring new methods of data collection for future use – such as application data, sensor data, biometrics, and data from drones – to provide more access to larger volumes of remotely collected data than ever before. Innovations coming out of COVID-19 digital response can complement and potentially replace older data collection methodologies once the pandemic is over.
By Monica Jerbi, Communications and Knowledge Management Consultant to ME&A, Inc. The Feed the Future Global Program Evaluation for Effectiveness and Learning project is made possible by the generous support of the American people through the United States Agency for International Development (USAID) under the Feed the Future initiative. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of USAID.
Am glad to have associated with you