Module 5

 

Survey Design

In many studies in development there is a need to collect data from either households or individuals. If we wish to make statistical inferences we need to use sound statistical principles in our design. Since most surveys are considered observational data the data can be used to describe the communities.  It is not appropriate to imply any cause and effect with observational data, although if we carefully design for comparing data from an intervention, it may be possible. If we use simple random sampling principles in selecting the survey units the estimates from these sample surveys have a broader applicability, as they will avoid bias.

 Aims and objectives of module 5

  • Introducing surveys with a simple overview including definitions and notation
  • To understand the basics of simple random sampling (SRS)
  • To understand stratified sampling
  • To develop research questions and link them to questions of interest on the survey
  • To use objective methods to preparing wording of questions to suit your objectives
  • To consider the report structure at the beginning to inform your questionnaire design
  • To consider issues of resource use, and training of enumerators
  • To use descriptive statistics to summarise the study
  • To calculate and understand measures of reliability of estimates

Introducing surveys with a simple overview

Here is a presentation that leads you through many aspects of survey design, make sure you understand the terminology used. Click to open.
All about Surveys

Some definitions and notation

A census – this consists of collecting data on the whole population. In this case there is no sampling or estimation as the population parameters can be calculated. Many countries conduct a census once every five or ten years.

Population  – this consist of all the units of interest. We  distinguish between the target population which would be the whole aggregate population and the sampled population.  For example the target population may be everyone (over 18 years old) in a State or district. However if we are conducting a phone survey the sampled  population consists of  those households with a phone. We aim to have the sampled population  close to the target.

Sample – this is the units randomly selected to be in our sample. We use the sample to make estimates (such as means, totals or proportions from our response variables) with which to make inferences about the population.

Sampling Frame – this consists of an actual list of units within the population.  So it  could be a data frame of names and addresses, it could be a list of businesses, or it may be a map with houses marked on spatially. They should be extensive and up to date or current

Sampling fraction (f) – this is defined as  n/N  which is the ratio of the size of the sample to size of the population. N is the population size and n is the sample size.

Response rate – the percentage of completed questionnaires. There are some differences between types of surveys on how these are calculated. This is affected by the mode of the survey (face-to-face,  online or telephone), the currency of the sampling frame, number of recalls, silent numbers and so on.

Estimates There are five characteristics of the population that are typically of interest:

  1. Mean
  2. Total
  3.  A ratio of two totals or means
  4.  Proportion of units in a defined class
  5.  A percent of units in a defined class

Check out other definitions under the Technical asides and definitions.

The basic principles of simple random sampling (SRS)

When we wish to undertake a simple representative survey, we should aim for simple random sampling.

This means that every individual within your target population has an known chance of being selected.

 

What is stratified sampling?

If we wish to  survey within groups – such as large, medium and small businesses. We would need to have the size of each of the businesses on our sample frame before we start.

Another strata that has been used is  remoteness, again one would need to have the frame classified in the strata before you sample.

There is a known probability of sampling- we can sample using the same number in each strata. In some cases we could sample all the large businesses – as within  a region there may be only  a few, however large businesses would affect our estimates. If we sample with start we can produce useful estimates within each strata.

Developing research questions and the survey questionnaire

 

Objective methods to prepare wording of questions to suit your objectives

 

Consider the report structure at the beginning of the project to inform your questionnaire design

A simple but useful way to start planning a survey is to sketch out (in dot points) what sections you will need to report on. An example could be
Executive Summary
Outcomes by area, and have you stratified by area or region, or even rural, urban and remote.
Outcomes by demographics (such as age groups, gender, or employment status
If reporting on resources- what level of detail do you require (farm level, or fields within a farm)
If you are reporting on any of these factors you need to ask the appropriate questions.

Are you planning to use graphics ? Plotting provides a nice summary. remember to record the sample size, and  an estimate of reliability.

 

Issues of resource use, and training of enumerators

The training required for your team, will depend on their experience. However questions should be piloted with a small group, and changes made on any poor phrasing. Closed questions are often easier to collect and lists of potential answers can assist enumerators to be aware of most options. Collecting data can be costly and so a focused survey with closed questions are often most efficient.

Descriptive statistics to summarise the study

 

Calculating and understanding measures of reliability of your estimates

For examining tests of association, see Module 4

 

 

Leave a Reply