A custom CAPI application
First – a bit of history. For just over ten years I have been interested in using computers to conduct interviews. Instead of recording the responses on a paper questionnaire, the interviewer uses a hand-held computer. When we first started in 2007 there were no Android tablets or iPads. PDAs such as the Palm pilot were available but they were not well suited to complex, multi-topic questionnaires of the type with which we were working . However, it was possible to buy a device called a UMPC – an ultra-mobile personal computer.
These were full Windows PCs but with a 7″ touch screen and stylus. Compared to modern tablets they were quite bulky, had limited battery life and were a bit more vulnerable (by having ‘spinning’ hard drives, for example). But they provided a real opportunity to start conducting electronic surveys. When we began this work the main problem was a lack of suitable software to implement the questionnaire. (By the way, the ‘we’ in this case is a survey company, EDI Africa, based in Bukoba, Tanzania, for whom I did some consultancy work between 2006 and 2010.) To address this, we used MS-Access to develop the application which could then be deployed using the free run-time system. I had a lot of experience in using Access and was able to use code that I had written for other purposes.
This approach was broadly successful. As well as delivering a lot of good quality survey data, it provided a valuable prototype from which EDI developed its own survey software called SurveyBe. Between 2007 and 2010, EDI used this MS-Access application to conduct 9 separate surveys in Tanzania collecting data from approximately 14,200 household interviews and 220 community/group interviews. Many of the surveys were quite complex (broadly based on the World bank’s Living Standard Measurement Survey format). The software itself had a range of features (some rather experimental) such has listing and sampling functions, multi-language support, conditional enabling (routing) of questions, comprehensive data validation with automatic and user-defined rules, use of photographs for reporting measurement units, built-in file transfer capability, integrated GPS capture, support for panel surveys, automatic generation of data transfer scripts for Stata and SPSS, flexible case management features and many more.
Ten years on, both the hardware and software are much better and CAPI has become much more widely used. There are still challenges and I have often seen poor implementations of the technology, failing to deliver the benefits which are available. More on that in a later blog. For those wanting to implement complex, multi-topic surveys there are now a number of solutions available. As well as EDI’s SurveyBe, two others spring to mind. The World Bank has created its Survey Solutions product to cater specifically for those conducting surveys in low-income countries. And CSPro, which has long been the most popular system for processing paper questionnaires in surveys of this kind, now supports data collection on Android devices. There are others of course and it is not my intention to provide any kind of review or endorsement. I offer this information to provide some context for readers engaged in this type of work and familiar with the resources available.
A Learning Experience
Let me now turn to the main subject of this blog; the development of a bespoke CAPI application. The main reason for embarking on this little project was to teach myself how to develop apps that could run on a tablet or smart phone. I concluded that the best approach was to take a real world example; one with which I was familiar that I could share with friends and colleagues to get their views and feedback. My choice was the proxy-means test (PMT) assessment that is used in Mongolia to identify households eligible for a number of targeted benefits. Between 2011 and 2013 I had worked in Mongolia on this programme and so was quite familiar with the assessment methodology which is currently paper based. The assessment questionnaire is quite simple, not dissimilar to a population and housing census form, and so was an ideal candidate for me to try writing my first app. The actual questionnaire I used was an early version of the questionnaire for the nationwide assessment which took place in March/April 2017. In total, almost 565,000 households (about 73% of the population) were assessed using a pen and paper approach. The assessment data were entered into a database which was developed and maintained by the Mongolian Ministry of Labor and Social Protection (MLSP).
At this point I should perhaps digress with a few caveats. Most importantly, I want to stress that this work has no formal connection with or endorsement by the Govt of Mongolia. I do not have access to any of the proxy-means test data that have been collected. The only resources that I have used are the blank questionnaires (both Mongolian and English) which are widely known in Mongolia, and the list of administrative areas (Aimags, Soums, Bags etc.). Secondly, to demonstrate the multi-language capability of the app, I have used the questions from the Mongolian questionnaire for some of the captions but for others I have resorted to Google Translate. I offer my apologies to any Mongolians puzzled by the strange text they may encounter. Finally, I am well aware that the proxy-means test methodology for targeting benefits is not without controversy but that is not the subject of this blog.
A simple questionnaire like this could have been implemented using one of the survey software applications mentioned above. However, this would not have satisfied my primary objective of learning how to develop apps. But there are other reasons why a bespoke implementation of this type may have some advantages over one of the specialist CAPI packages. Here are just a few.
- The first is integration with existing information systems. An application such as this could be fully integrated with the current system used in the MLSP. This system uses MySQL and PHP. The app currently uses these to send data to a remote server and store the data in some MySQL tables. It would be relatively simple to hook up this app with the MLSP system.
- Another benefit is highly intuitive design and layout. The PMT assessments are done by a large number of people with varying levels of ICT literacy. Training needs to be focused on conducting the assessment; how to ask the questions, record responses etc. We don’t want to spend too much time on how to use the app. The best approach is to make it as simple and intuitive as possible. My own guidelines are simple; people use email, web browsers and a whole range of apps on their phones without going on training courses. The software designers know this and so put a lot of effort into making the software very intuitive. I have attempted to apply this principle here. Please note that I am not suggesting that the other CAPI system are not user-friendly or intuitive. It’s just they may not be suitable for such a wide user base.
- The software I have used allows fine-grained design features such as inserting custom code that runs when certain values are changed. The architecture is very open and offers all kinds of possibilities that are not possible with other systems.
A description of the software with instructions on how to use it (including installation on an Android tablet) is found here.
In future blogs I will talk about some of the design features, such as why I am still a fan of SQL, as well as the challenges and frustrations that I encountered when creating the app.
Comments and questions are very welcome.