Internews Center for Innovation & Learning

Internews Center for Innovation & Learning
Twitter icon
Facebook icon
RSS icon

Dadaab: using mobile technology for large surveys in emergency settings

In August 2011, Internews led a joint communication and information needs assessment with Radio Ergo / International Media Support (IMS) and Star FM of Kenya, with significant support from the Norwegian Refugee Council (NRC). This assessment aimed at understanding the information needs of refugees in Dadaab and exploring ways to improve the flow of communication between refugees, aid agencies and host communities.

Understanding the information ecology in any country is vital to Internews’ work, and assessments like this one form an important component of our humanitarian information projects. In the aftermath of a disaster, understanding how the information ecosystem has changed can make aid delivery much more effective. If humanitarian actors understand how information flows and how it is accessed, shared and consumed, they can provide better services, engaging with affected populations through the most appropriate channels and platforms and fulfilling the right people have to know about what is happening as well as what services are available to them. People affected by crisis have a right to ask questions and get answers, therefore becoming active agents in their own relief and recovery.

To address this issue in the Dadaab camps, the Internews-led assessment team trained a group of local volunteers from the NRC on using smart phones with data collection software designed by Episurveyor to conduct interviews. Internews’ goal was to better understand and document the existing local information ecosystem in Dadaab: how information flows, how it is accessed, shared and consumed including the reliability of sources, what are the existing initiatives, and what is the local media scene. Based on this, Internews provided recommendations to humanitarian agencies on how to establish better two-way communication exchanges with local populations. It also reviewed this information internally in order to improve its own work. 

The Assessment

From August 7-14, 2011, the Dadaab assessment team conducted an extensive survey of over 630 refugees in all three of Dadaab’s refugee camps: Ifo (including the informal Ifo Extension), Dagahaley, and Hagadera. Surveys targeted both new arrivals (those who had been in the camps for less than nine months) and long-term residents. The sample had strong representation of both men and women. The assessment team also interviewed a number of humanitarian workers and a follow-up team visited Dadaab from August 22-27, 2011 to present preliminary findings of the assessment, work on immediate next-steps, and continue engaging with several humanitarian agencies. This included initial planning for the construction of a local radio station to be built and managed by Star FM.

The assessment piloted the use of smartphones (Android) to record respondents’ answers in the field, in combination with paper surveys. This meant answers could be uploaded in real-time, allowing for immediate data reviewing and analysis. The platform for uploading and managing the survey data was EpiSurveyor (www.episurveyor.org), supported by the mobile app developed by Open Data Kit (www.opendatakit.org).

The benefits of mobile data collection

This was Internews’ first experiment using mobile technology to conduct these types of assessments. This survey methodology is now being further tested by Internews in another research pilot in Haiti. Based on Internews’ Dadaab experience, the organization is keen to further take advantage of the benefits afforded by mobile data collection.

These include:

  • FASTER DATA GATHERING AND ANALYSIS: The real-time nature of the data uploading reduced the time-lapse between data collection and analysis.
  • ABILITY TO COLLECT ADDITIONAL GEOGRAPHIC DATA: The GPS capabilities of the smart phones added a geo-referenced layer to the data itself, which was useful both for data quality and for data visualisation.
  • LESS FUSS: Interviewers on the ground underlined that the use of mobile phones made it easier to conduct interviews without attracting too much attention in the camps.
  • REMOTE QUALITY CONTROL: The time-stamp and geo-location tags of uploaded data were critical for quality control. They permitted managers at headquarters to ensure that researchers on the ground were covering significant areas while also taking sufficient time to conduct interviews. The real-time uploading of the reports also allowed research managers to identify and flag mistakes and correct them immediately. 

It was clear from Dadaab that the possibility of instantaneous data collection and analysis holds great promise, both to provide real-time data for fast response and as another tool for ongoing monitoring and quality control of field research activities themselves.  Having said that, the project did encounter a number of challenges.

Challenges and Lessons Learned

We learned a lot in Dadaab, including several things about how to improve the use of mobile survey methodologies. In general, field monitoring and other quality-checks identified a few instances in which proper methodology was not being carried out; for example, questions were occasionally incorrectly asked or interviewee samples were inadequately demographically randomized. All suspect data was deleted, which left the team with a total vetted sample size of 492 respondents out of the original 632 interviews conducted.

In the end there were a number of things that made it harder to quickly capture and analyze all the information Internews sought and the lessons learned were important. With them, there is hope for greater understanding of information needs and an opportunity to make a difference in humanitarian aid through information.

Overall, we learned the following lessons:

  1. Survey methodology needsto be adapted to the situation on the ground and refined in order to have an adapted and yet flexible system in place
  2. The use of mobile technology is key for geo-location and real-time but it always needs to be implemented with paper backup
  3. Cultural and social issue needs to be considered when deciding to use mobile or any other technology in humanitarian emergency settings
  4. Preparation is key in large deployments of technology based tools in humanitarian settings, both in terms of equipment and technology knowledge
  5. Analysis and data scrambling is a secondary and totally different task that needs to be done and thought through with the support of experts and dedicated methodologies

The survey that was created for this assessment was fist designed by the Director of the Humanitarian Media Project and then refined by the project manager on the ground. The survey design and the methodology used for the upload of the survey on the phones and the double language (English and Somali requirements) created the following issues:

  • The survey was too long and with too many open questions. This created complications when it came to analyze the data.
  • The survey was not designed using coded responses, which meant that the coding was done manually once the survey was done, with a dispersion of energy and time.
  • The survey was created as 2 separate files, one for Somali and one for English languages. This means that also the two sets of data were separate and had to be aggregated manually at the end of the survey
  • The survey was changed several times from the first design, which lead to multiple master files and with some errors in the translation into Somali
  • The software did not allow the project manager on the ground to change the survey directly, so all changes needed to pass through the Innovation Advisor in Nairobi.
  • The fact that mobile phones were used for the first time by Internews in this instance also created some issue, mainly due to the lack of previous knowledge of the tool and the time taken to familiarize with its features. The training conducted in the first day was considerably longer than usual due to the fact that the trainers had to be trained on using the phones and the forms in addition to the survey methodology itself.
  • Batteries and security issues were also raised in relation to the use of mobile phones; an issue that can be solved by providing the researchers on the ground with extra batteries and using paper based surveys as a back up system. 

Recommendations

  1. BE REALISTIC: The design of the survey was over ambitious. The organization asked too many questions given the capacity and resources on the ground and the ambitious timeline to deliver the final report. When you think you are asking too many questions, you actually are asking too many questions.
  2. DO YOUR HOMEWORK (AND UNDERSTAND THE BACK END OF YOUR INITIATIVE): It would have been useful to have tested Episurveyor in a reduced and manageable environment prior to the full pilot. The organisation also failed, from a human perspective, to better understand the flexibility and the level of analysis that the tool could provide. Preparation is key in any large deployment of technology-based tools in humanitarian settings, both in terms of equipment and technology knowledge. Spend more time on planning and thoroughly investigating a new tool.
  3. HAVE A PLAN FOR YOUR DATA (AND DON’T WASTE DATA): When conducting a survey, do not ask open-ended questions unless you have the capacity to analyse all that data. In a number of questions, despite providing virtually all-possible options to the questions, there was also an "others" category in each question. Internews didn’t have the capacity to analyse all this data. Just because we can collect more data, at faster speeds doesn’t necessarily mean it is better.
  4. LOST IN TRANSLATION: The survey was done in both English and Somali languages, which meant that the team had to create two separate files on the smart phones, one for Somali and one for English. This resulted in the collection of the two different sets of data that had to be aggregated manually at the end of the survey. Factoring in language issues is critical to the success of valid survey information.
  5. IT TAKES A VILLAGE: Doing a survey of this magnitude takes a village of people in the field; administrators, analysts, and writers. Surveys require a whole range of inputs, including before and after the data collection and initial analysis. For example, analysis and data scrambled is a second and totally different task that needs to be done and thought through in advance. Additionally, for this project Internews had a research team in Dadaab, a research advisor in Haiti, its main data analyst in the USA, and its lead report writer in Australia. This represented fantastic capacity, but geography was problematic. Don’t overstretch your team, coordinate your roles, plan the entire process in advance, and fine-tune your methodology.
  6. GO BACK TO BASICS (NEVER UNDERSTIMATE THE HUMAN COMPONENT): Logistics, batteries, human capacity, training time, local and religious holidays, politics, securit -- these are some of the incidental factors that have real effects on the implementation of any project. Even with the best new tools, factors like these can directly determine the success or failure of any survey initiative. Survey methodology needs to be adapted to the situation on the ground and refined in order to have a tailored but flexible system in place. Keep it real. At the end of the day, everything goes back to basics: human interaction.

 (Images: Meridith Kohut/Internews)

This case study was produced for the CDAC Network Media and Tech Fair held in London - 22 March 2012 

Global Issues: 
Region: