Chapter 2 - Formulation of the Statement of

Objectives
2.0 Introduction
The first task in planning a survey is to specify the objectives of the survey as thoroughly and as clearly as
possible. A clear Statement of Objectives guides all subsequent survey steps. These steps should be
planned to ensure that the final results meet the original objectives.

Suppose that a survey on poverty is to be conducted. It is not enough to indicate that the purpose of the
survey is to provide information on, for example, housing conditions of the poor. Such a vague
statement may serve as a broad description of the surveys general theme, but ultimately it must be
expanded into more specific language. What is meant by housing conditions? Does it refer to the
building, its age or need of repair, or does it refer to crowding (e.g., the number of people per square
metre)? What precisely is meant by poor? Is poverty to be measured in terms of income, expenditures,
debts, or all of these?

The statistical agency, in consultation with the client, first needs to define the information needs and
primary users and uses of the data more completely and precisely. Broadly, what information is required
on housing conditions for the poor? Who needs the data and for what purpose? Suppose that the client
requesting the survey is City Council. City Council suspects that housing conditions of the poor to be
inadequate and expects to have to build new subsidised housing. City council may want to know how
many new houses would be are required and how much they would cost. It may want to ask the poor
where they would want the new housing. The city may also expect to vary the subsidy depending on the
poverty of the family, thereby requiring data for different levels of poverty.

Next, specific operational definitions need to be developed, including a definition of the target population.
These definitions indicate who or what is to be observed and what is to be measured. In the case of poor,
the definition might be all families with a gross income below a certain level. The terms family and
income must also be defined. Coverage of the population needs to be defined: what geographic area is
the client interested in which areas within the city? And what is the reference period last week, last
year?

The statistical agency also needs to know what specific topics are to be covered by the survey. Does the
client want information by income group, the type of dwelling (e.g., apartment building, single detached
house, etc.), the age of the dwelling, the number of people living in the dwelling, etc.? What level of
detail is necessary for each topic and what is to be the format of the results? This usually takes the form of
proposed analysis tables. For a sample survey, the level of detail possible is a function of the size of the
sampling error in the estimates as well as operational constraints such as available time, budget, personnel
and equipment. These quality targets and operational constraints will heavily influence the scope of the
survey.

The Statement of Objectives may be revised many times by the statistical agency, in consultation with the
client, during the planning, design and development of the survey.

The purpose of this chapter is to illustrate how to formulate the Statement of Objectives.
2.1 A Multi-Step Process for Developing the Statement of Objectives
Developing the Statement of Objectives is a multi-step, iterative process involving the statistical agency,
the client and the users (if different than the client). The steps in the process are to determine:
- the information needs;
- the users and uses of the data;
- the main concepts and the operational definitions;
- the survey content;
- the analysis plan.

To illustrate these steps, consider the following example. The citys Regional Transit Authority (RTA)
has been instructed by the City Council to take steps to facilitate the use of public transit by senior
citizens (referred to as seniors). Since the RTA does not have current information on seniors needs or
travel habits, it has approached the statistical agency to help gather new data. The following paragraph is
the RTAs initial statement of the situation:

To facilitate the use of public transit by the citys seniors, the RTA is considering modifying its
existing service. Possible changes include, for example, purchasing special buses, modifying
existing buses, adding new routes, or possibly subsidising fares. Before proceeding with
expensive purchases and alterations, the RTA requires information on the transportation needs of
seniors so that it can tailor its budget and improvements to their needs.

2.1.1 The Information Needs (State the Problem)

The first step is to describe in broad terms the information needs of the client. The statistical agency
should begin by identifying the problem and stating it in general terms. Why has a survey been
suggested? What are the underlying issues and in what context have they been raised?

In the RTA example, the City Council first instructed the RTA to take steps to facilitate the use of public
transit by seniors. In the initial statement, the RTA interpreted this as a need to modify existing service to
facilitate the use of public transit by the citys seniors. But what is the objective that must be directly
addressed to help the RTA achieve that goal?

The RTA requires information on the transportation needs of seniors and if and how they are
currently being met.

The overall information needs of the survey have now been identified. It is important to return to this
statement at every step of the survey to ensure that the survey objectives are met.

2.1.2 The Users and Uses of the Data

The next questions to ask are: Who are the main users of the data? What will the information be used for?
The statistical agency needs to know who the users are because their input is very important during the
planning phase of a survey. (The ultimate users of the data are not always the client, although this is often
the case.) The uses of the data must be identified to specify more precisely the information needs. This is
done in consultation with the client and data users. What types of policy issues are to be addressed? Will
the survey information be used to describe a situation or to analyse relationships? What type of decisions
might be made using the data and what might be the consequences? If possible, potential respondents
should also be consulted since they could identify issues and concerns that are important to them which
could affect survey content.

According to the RTA, the RTA requires information on the transportation needs of seniors so that it can
tailor its budget and improvements to the needs of seniors. Specifically, the information may be used by
transportation planners at the RTA to:
- purchase special buses;
- modify existing buses;
- add new routes;
- subsidise fares.

The information needs of the survey have now been identified, along with who will use the data and what
it will be used for; this is particularly important. For example, suppose that the RTA expects to have to
add new routes, then the RTA may want to ask seniors where they would like these routes. If the RTA
expects to have to modify existing buses, then it may want to know what modifications seniors would
prefer. If the RTA expects to have to purchase special buses, it may want to know what special buses
seniors require. If the RTA is considering subsidising fares, it may want to ask seniors what they would
consider to be an affordable fare. The results expected, therefore, and the consequences of the results
determine the survey content.

2.1.3 Concepts and Operational Definitions

In order to identify the data required to meet the surveys objectives, the statistical agency needs clear and
precise definitions. These definitions may specify exclusions, such as homeless individuals or individuals
living in institutions, etc. To the extent possible, recognised standard definitions should be used. This will
facilitate communication with the data users and respondents, as well as ensure consistency across
surveys. The statistical agency may be required to develop some standard definitions, for example, for
dwelling, household, family, etc.

In order to determine the operational definitions, there are three questions that should be asked: Who or
What? Where? and When? One of the first concepts to be defined is the surveys target population. The
target population is the population for which information is desired. It is the collection of units that the
client is interested in studying. Depending on the nature and the goal of the survey, these units usually
will be individuals, households, schools, hospitals, farms, businesses, etc. Returning to the RTA example,
the following questions should be asked when defining the surveys target population:

i. Who or what is the client interested in?

Here, the statistical agency needs to consider the type of units that comprise the target population and the
units defining characteristics. For the RTA survey, it has been stated that the client is interested in the use
and needs of public transit by seniors. Explicit definitions of seniors, public transit and use are required.
Suppose that seniors are defined as persons aged 65 years or over. (The client should clarify with the
RTA what the RTAs own definition of seniors is for the purposes of urban transportation.) There may be
several forms of public transit: buses, trains, subways and special needs vehicles. Suppose that the client
is only interested in buses. Another question is whether the client is only interested in seniors who
currently use buses, or all seniors? The client may be interested in all seniors.
ii. Where are the units of interest?

This refers to the geographical location of the units (i.e., seniors). The client may only be interested in the
use of public buses operating in the city metropolitan area (as defined by a recent census, for example;
again, a clear definition is required), or perhaps the area over which the RTA has jurisdiction (i.e., the
area served by the existing network of public bus routes). So, the client must decide if it all seniors are
part of the target population, or just those living in a particular region.

iii. What is the reference period for the survey? (When?)

What time period do the data refer to? (When?) The answer appears to be now since the RTAs
statement refers to current needs. In practice, this could mean that seniors are asked about their use of
public buses during a recent reference period (week, month, etc.). Should seniors be surveyed for more
than one period or asked about several different reference periods?

An important consideration with respect to the reference period is seasonality. Certain activities are
related to the time of the week, month or year. Therefore, conclusions that refer to a specific time frame
may not be valid for other time frames. For example, if the RTA questionnaire asks seniors about their
use of the transit system during the week, the survey results may not be valid for weekends.

In addition to the target population, many other concepts must be defined. The following are examples of
three related concepts commonly used by household surveys at Statistics Canada:

A dwelling is any set of living quarters that is structurally separate and has a private entrance
outside the building or from a common hall or stairway inside the building.

A household is any person or group of persons living in a dwelling. A household may consist of
any combination of: one person living alone, one or more families, a group of people who are
not related but who share the same dwelling.

A family is a group of two or more persons who live in the same dwelling and who are related
by blood, marriage (including common-law) or adoption. A person living alone or who is
related to no one else in the dwelling where he or she lives is classified as an unattached
individual.

For more details on how to define the target and survey populations see Chapter 3 - Introduction to
Survey Design.

2.1.4 Survey Content

A clear Statement of Objectives ensures that the survey content is appropriate and clearly defined. Having
determined the overall information needs, the users and uses, and the operational definitions, the
statistical agency next needs to know what specific topic areas are to be covered by the survey. This is
often an iterative process. The process of specifying the survey content often leads to revelations about
incompleteness in the information needs and uses, or conversely, to the revelation that some needs cannot
be met due to operational or definitional reasons.

Returning to the RTA example, the information that is required at a fairly general level has been
identified. Now the statistical agency needs to expand on this
For seniors, the client may also wish to determine various characteristics such as:
- age;
- sex;
- disabilities;
- household income;
- geographic location (are seniors mainly living in small areas within the city, such as retirement
homes, or are they spread out throughout the city?);
- dwelling type (e.g., retirement homes, apartments, houses);
- household composition (who are they living with?).

To determine transportation needs, the client may require information on:

- number of trips last week;
- frequency of travel (by time of day; weekday versus weekend);
- modes of transportation used;
- problems using public buses;
- amount of local travel.

A desire for information on trip characteristics may lead to questions about:

- the reason for the trips;
- the geographic origin and destination of the trips;
- limitations on travel;
- special aids or assistance needed;
- the number of trips cancelled due to lack of transport.

To determine whether or not needs are currently being met, the client may have to understand such issues
as:
- accessibility (How many seniors own a car, bicycle, etc.?);
- use of public buses;
- amount of money spent on public buses;
- ways to improve service;
- ways to encourage seniors to use (or more frequently use) public buses.

Note that definitions are required for all concepts not already defined. For example, what is meant by a
disability? What is a trip?

The specific topics to be covered determine the variables to be collected, the questionnaire design and
even the sample design. These in turn affect the choice of data collection method, for example whether or
not to use interviewers, and therefore the cost of the survey.

The statistical agency must cover all aspects of the information needs, but to avoid incurring unnecessary
costs or imposing undue response burden on the survey population, it should eliminate all irrelevant items
that do not directly relate back to the surveys objectives.

In a later step, this description of survey content must be transformed into questions and a questionnaire.
For more details, see Chapter 5 - Questionnaire Design.

2.1.5 The Analysis Plan (Proposed Tabulations)

Once all of the items to be measured have been identified, the next task is to determine how much detail
is required for each item and the format of the results. What measures, counts, indices, etc. are needed?
Are estimates for subpopulations required? The detailed plan of the way the data are to be analysed and
presented is referred to as the analysis plan and, in addition to planned analyses, requires the creation of
proposed tabulations. An analysis plan greatly facilitates the design of the questionnaire.

For example, with respect to the details of the final results, is it necessary to be able to distinguish
between different age groups within seniors? Does the client need to distinguish between men and
women, or between different types of transportation (bus, car, bicycle, etc.)? Should continuous or
categorical data be used? For example, does the client need to know what a seniors exact income is, or is
a range of incomes adequate? (If the client is interested in calculating averages, then exact income is more
appropriate.)

Note that the analysis plan can involve going back to and refining the operational definitions and survey
content. For the RTA example, here are some possibilities for the level of detail of the results, in
increasing order of detail:

Household income:
- household income ranges (e.g., less than $15,000; $15,000 to $29,999; $30,000 to $49,999; etc.);
- the exact number representing total household income;
- the exact income from each source (wages or salary, pensions, investments).

Disabilities:
- a single question on whether the senior has a physical condition that limits his/her ability to take local
trips;
- a single question on each of several distinct disabilities;
- a series of questions to be used to determine the presence, nature and severity of each disability.

Household composition:
- senior lives alone / does not live alone;
- number of people in the household;
- categories of households (single, couple, two related adults other than couples, three or more related
adults, etc.);
- each adults age and relationship to a reference person, in order to derive the exact household
composition.

Number of trips last week:

- ranges (e.g., 0-3, 4-6, etc.);
- the exact number;
- the exact number by day and time of day.

Frequency of travel:
- the percentage of trips on weekdays or weekends;
- the exact number of trips taken on each day of the week.

Modes of transportation used:

- mode used most often during the reference period (e.g., last week);
- all modes of transportation used (public and private);
- number of trips on public buses only;
- for each trip taken, the mode of transportation that was used.

Problems using public buses:

- factor causing the most difficulty;
- all factors causing difficulty;
- a ranking of the factors according to difficulty caused;
- for each factor, a measure of how much difficulty it causes.

In the cases presented above, the first, least detailed, breakdown may be sufficient, or it may not contain
enough detail to meet the clients needs for information. The last, most detailed breakdown may contain
exactly the right amount of detail, or it may be excessively detailed, and in fact be too difficult to answer.
While detailed information provides greater flexibility for analysis and may permit comparison with other
information sources, the statistical agency should always try to ask for information that is detailed enough
to meet the analysis needs, but no more, to avoid burdening respondents.

It is a good idea to prepare a preliminary set of proposed tabulations and other key desired outputs.
Determining how the results are to be presented helps define the level of detail and indeed the whole
scope of the survey. Without a clear analysis plan, it may be possible at the end of the survey to generate
hundreds of analysis tables, but only a few may relate directly to the surveys objectives.

The proposed tabulations should specify each variable to be presented in a table and its categories. The
purpose of this step is to create and retain mock-ups of those tables that will form the analysis.
Specification at this level allows the statistical agency to begin drafting questions for the survey
questionnaire.

For example, for the RTA survey, the population could be partitioned into two or more groups (e.g., to
compare seniors with a disability to seniors without a disability).

Single item summaries (frequency distributions, means, medians, etc.) could be produced, such as:
- percentage of trips taken on each day of the week (Table 1);
- average number of trips taken on public buses;
- average amount of money spent on transportation last week;
- percentage of seniors by most frequent reason for trip.

Table 1: Trips Taken by Day of the Week

Day of Week Number of Trips % of Total Trips
Sunday
Monday
Tuesday
Wednesday
Thursday
Friday
Saturday
Total

Cross tabulations of possible interest could include:

- number of trips by mode of transportation (Table 2);
- number of buses taken by starting and ending points;
- distribution of reasons for not using public transportation by characteristic of person (e.g., disabled,
etc.).

Other relationships that could be investigated include:

- average amount spent on transportation for each income group;
- median income of housebound seniors.
 Table 2: Number of Trips by Mode of Transportation
Mode of Transportation Number of Trips % of Total Trips
Public transportation
Bus
Subway
Other
Private transportation
Car/truck Bicycle
Walk
Other
Total

2.2 Constraints that Affect the Statement of Objectives

There are many requirements and constraints that can affect a surveys Statement of Objectives. One
relates to the quality of the estimates. How precise should the survey results be? This refers to the
magnitude of sampling error that is acceptable for the most important variables. Precise, detailed results
often require very large samples, sometimes larger than the client can afford. As a result, the client may
decide to relax the precision requirements or produce more aggregate, less detailed data.

Factors affecting precision, and therefore the sample size, include the following:
- the variability of the characteristic of interest in the population;
- the size of the population;
- the sample design and method of estimation;
- the response rate.

In addition, operational constraints influence precision. Sometimes, these are the most influential factors:
- How large a sample can the client afford?
- How much time is available for development work?
- How much time is available to conduct the entire survey?
- How quickly are the results required after collection?
- How many interviewers are needed? How many interviewers are available?
- How many computers are available? Are computer support staff available?

Precision is discussed in more detail in Chapter 3 - Introduction to Survey Design, Chapter 6 -

Sample Designs, Chapter 7 - Estimation and Chapter 8 - Sample Size Determination and Allocation.

Other factors that impact the Statement of Objectives include:

- Can the required variables be measured with the available techniques?
- Will acquiring the desired results be too much of a burden on the respondents?
- Could confidentiality of the respondent be compromised given the level of detail of the disseminated
results?
- Will the survey have any negative consequences on the reputation of the survey agency?

These considerations are all aspects of planning a survey. See Chapter 13 - Survey Planning and
Management for more details.
2.3 Summary
Without a clear idea of the information needs, the statistical agency risks tackling the wrong problem,
producing incomplete or irrelevant results and wasting time and resources; the surveys activities could
simply annoy or inconvenience many respondents, without producing any useful information. For these
reasons, the surveys objectives must be clearly defined during the planning phase.

The following list summarises the most important questions and items to be considered when developing
a surveys objectives and information needs:
- What are the overall information needs of the survey?
- Who will use the data and how will they use it?
- What definitions will be used by the survey?
- What are the specific topic areas to be covered by the survey?
- Has an analysis plan with proposed tabulations been prepared?
- What is the required precision of the estimates?
- What are the operational constraints?

Formulation of the surveys objectives may continue to be refined during the design and development of,
particularly, the questionnaire (see Chapter 5 - Questionnaire Design).

Bibliography

Brackstone, G.J. 1991. Shaping Statistical Services to Satisfy User Needs. Statistical Journal of the
United Nations, ECE 8: 243-257.

Brackstone, G.J. 1993. Data Relevance: Keeping Pace with User Needs. Journal of Official Statistics, 9:
49-56.

Fink, A. 1995. The Survey Kit. Sage Publications, California.

Fowler, F.J. 1984. Survey Research Methods. 1. Sage Publications, California.

Kish, L. 1965. Survey Sampling. John Wiley and Sons, New York.

Levy, P. and S. Lemeshow. 1991. Sampling of Populations. John Wiley and Sons, New York.

Moser C.A. and G. Kalton. 1971. Survey Methods in Social Investigation. Heinemann Educational Books
Limited, London.

Satin, A. and W. Shastry. 1993. Survey Sampling: A Non-Mathematical Guide Second Edition,
Statistics Canada. 12-602E.

Statistics Canada. 1998. Policy on Standards. Policy Manual. 2.10.