Definition Of Research The word research is derived from the Latin word meaning to know.

It is a systematic and a replicable process which identifies and defines problems, within specified boundaries. It employs well designed method to collect the data and analyses the results. It disseminates the findings to contribute to generalizeable knowledge. The five characteristics of research presented below will be examined in greater detail later are: 1. 2. 3. 4. 5. systematic problem solving which identifies variables and tests relationships between them logical, so procedures can be duplicated or understood by others empirical, so decisions are based on data collected reductive, so it investigates a small sample which can be generalized to a larger population replicable, so others may test the findings by repeating it

Basic and Applied Research Basic (a.k.a. fundamental or pure) research is driven by a scientist's curiosity or interest in a scientific question. The main motivation is to expand knowledge, not to create or invent something. Therefore, in this case, there is no obvious commercial value to the discoveries that result from basic research. Basic science investigations probe for answers to questions such as: How did the universe begin? What are protons, neutrons, and electrons composed of? Applied research is designed to solve practical problems of the modern world, rather than to acquire knowledge for knowledge's sake. One might say that the goal of the applied scientist is to improve the human condition. For example, applied researchers may investigate ways to improve transportation systems . . Distinction Between Basic and Applied Research The distinction between basic and applied research isn't always clear. It sometimes depends on ones perspective. One way to look at it is to ask the following question: "How long will it be before some practical application results from the research?" If a practical use is only a few years away, then the work can be defined as strictly applied research. If a practical use is still 20-50 years away, then the work is somewhat applied and somewhat basic in nature. If a practical use cannot be envisioned in the foreseeable future, then the work can be described as purely basic research. In other words, basic research lays down the foundation for the applied science that follows. If basic work is done first, then applied spin-offs often eventually result from this research.

Rules for Research Questions First Rule: Start With a Simple Question A simple question has one stem and one topic. You may recall in the last section it was recommended that beginning researchers should start with a simple question that means a question with one stem and one topic. In this question Who drove the train? The stem is who and the topic is drove the train. Second Rule: Use Action Questions As you start to write questions you will note that some do not require action. Any question that can be answered by a yes or no is not action oriented. These questions are stoppers. The question has been answered, excluding the need to do any research. Questions that begin with should or could are stoppers; they elicit opinions, not facts. Therefore the next most important thing about research questions is that they must be: action-oriented and demanding some activity on your part Third Rule: Use Active Questions The type of question you ask about your topic is the basis for the design of your research plan. Whether you go to the transportation libraries or whether you observe traffic flow, your particular activity is inherent in the question you have asked. At the beginning of the research plan you need something that will provide direction. You are concerned with the planning phase of the research with dealings in the future, for this reason:

ensure to ask an active question active questions imply that the researcher will have to measure or observe something

Fourth Rule: Do not Elicit Opinions A statement declaring a fact (Speeding micro-buses have an effect on accidents) require no action on anyones part. The question, on the other hand, (What is the relationship between speed and accidents?) demands an answer. If your question can be answered by a simple yes, no, or I dont know, then you dont need to do research to find the answer for this reason:

ensure that you are not eliciting opinions ensure that you are ending up with yes, no answers

Everyone has an opinion on each of the above questions. Try rewriting each of your should questions into action questions that require some investigation to find the answer. You will notice a great difference between action and opinion questions. Fifth Rule: Avoid Use of Inactive Verbs As you write your initial questions, try to write questions that begin with what, for example what is the relationship., andwhy. Avoid using inactive verbs such as do at the beginning of your question. Questions beginning with do, like questions that begin with should, can be answered by yes, no, maybe, or I dont know, and are considered stoppers. They elicit an opinion rather than some activity directed toward research. Notice how questions that begin with do look. Do divers over speed? Do drivers respond to traffic lights in the same way? A Hypothesis 2

A hypothesis is a specific statement of prediction. It describes in concrete terms what you expect will happen in your study. Not all studies have hypotheses. Sometimes a study is designed to be exploratory If there is no formal hypothesis, then perhaps the purpose of the study is to explore some area more thoroughly in order to develop some specific hypothesis or prediction that can be tested in future research. A single study may have one or many hypotheses. Hypothesis test is formulated by two statements, one that describes the prediction and one that describes all the other possible outcomes with respect to the hypothesized relationship. The prediction is that variable X and variable Y will be related whether it's a positive or negative relationship. One-tailed Hypothesis. If your prediction specifies a direction, and the null therefore is the no difference prediction and the prediction of the opposite direction, we call this a one-tailed hypothesis. For instance, let's imagine that you are investigating the effects of a new road tracking system for haulers and that you believe one of the outcomes will be that there will be less travel time as the drivers will be monitoring on-line. Your two hypotheses might be stated something like this: The null hypothesis for this study is: h0: As a result of the new road tracking system for Haulers Company, there will either be no significant difference in travel time or there will be a significant increase in travel time. Which is tested against the alternative hypothesis: ha: As a result of the new road tracking system for Haulers company, there will be a significant decrease in travel time. The alternative hypothesis -- your prediction that the new system will decrease travel time -- is shown below. The null must account for the other two possible conditions: no difference or an increase in travel time. The figure shows a hypothetical distribution of travel time differences. We can see that the term "one-tailed" refers to the tail of the distribution on the outcome variable. Two-tailed Hypothesis. When your prediction does not specify a direction, we say you have a two-tailed hypothesis. For instance, let's assume you are studying new packaging system. In this case, you might state the two hypotheses like this: The null hypothesis for this study is: h0: As a result of new packaging system of the, there will be no significant difference average weight from the old system. Which is tested against the alternative hypothesis: ha: As a result new packaging system, there will be a significant difference in average weight. The term "two-tailed" refers to the tails of the distribution for your outcome variable. Hypothetical-Deductive Model The important thing to remember about stating hypotheses is that you formulate your prediction (directional or not), and then you formulate a second hypothesis that is mutually exclusive of the first and incorporates all possible alternative outcomes for that case. When your study analysis is completed, the idea is that you will have to choose between the two hypotheses. If your prediction was correct, then you would (usually) reject the null hypothesis and accept the alternative. If your original prediction was not supported in the data, then you will accept the null hypothesis and reject the alternative. The logic of hypothesis testing is based on these two basic principles: 1) the formulation of two mutually exclusive hypothesis statements that, together, exhaust all possible outcomes 2)the testing of these so that one is necessarily accepted and the other rejected 3

Hypothesis testing seems a convoluted and formalistic way to ask research questions. However, it encompasses a long tradition in statistics called the hypothetical-deductive model, and sometimes we just have to do things because they're traditions. And anyway, if all of this hypothesis testing was easy enough so anybody could understand it, how do you think statisticians would stay employed? In logic, we often refer to the two broad methods of reasoning as the deductive and inductive approaches. Top Down Approach Deductive reasoning works from the more general to the more specific. Sometimes this is informally called a topdown approach. Bottom Up Approach Inductive reasoning works the other way, moving from specific observations to broader generalizations and theories. Informally, we sometimes call this a bottom up approach. In inductive reasoning, we begin with specific observations and measures, begin to detect patterns and regularities, formulate some tentative hypotheses that we can explore, and finally end up developing some general conclusions or theories. These two methods of reasoning have a very different "feel" to them when you're conducting research. Inductive reasoning, by its very nature, is more open-ended and exploratory, especially at the beginning. Deductive reasoning is narrow in nature and is concerned with testing or confirming hypotheses. Even though a particular study may look like it's purely deductive most research involves both inductive and deductive reasoning processes at some time in the project. Even in the most constrained experiment, the researchers may observe patterns in the data that lead them to develop new theories. What is a Research Design? The design is a set of instructions to the researcher to gather and analyze data in certain ways that will control who and what are to be studied. Unwanted or extraneous variables can always be controlled, the variance of specific variables is enhanced, other words, the design makes it possible for you to isolate the variables you are interested in from all other variables and to measure them accurately so that your data are reliable and valid. Choice of a Research Design The design chosen must be the best way to answer the question; it must fit the level of the question. Variables about which nothing is known need to be questioned at a basic level, and the answer is a description of those variables. The design always builds on previous findings, except in replication studies. If you feel the results of previous research require further support, your study may replicate the previous research, following the design exactly as it was in the original study. Descriptive Designs Descriptive designs result in a description of the data, whether in words, pictures, charts, or tables, and whether the data analysis shows statistical relationships or is merely descriptive. If the topic is for Level I or II, the design will be descriptive. What questions invariably lead to descriptive designs. No matter what method is chosen to collect the data, all descriptive designs have one thing in common: they must provide descriptions of the variables in order to answer the question. The type of description that results from the design depends on how much information the researcher has about the topic prior to data collection. Look at the design in the same way that you looked at the question. Level I questions, with little or no prior knowledge of the 4

topic, lead to exploratory descriptive designs. Level II questions, where the variables are known but their action cannot be predicted, lead to descriptive survey designs. Exploratory Descriptive Designs When the purpose of a study is exploration, a flexible research design that provides an opportunity to examine all aspects of the problem is needed. As knowledge of the variables increases, the researcher may have to change direction. Ideas occur as data are collected and examined. The key to a good exploratory design is flexibility. The word exploratory indicates that not much is known, which means that a survey of the literature failed to reveal any significant research in the area. Thus, you cannot build on the work of other; you must explore the topic for yourself. Even though we talk about the exploratory study as an entity in itself, it should be remembered that it is an initial step in the development of new knowledge. Because of the flexibility of this type of design, very few, if any, variables are under the researchers control. They are said to be under the control of the situation in other words, observed as they happen or as the researcher comes upon them. As a result, no inferences can be drawn from the data. The data may lead to suggestions of hypotheses for further study or to an idea for a conceptual framework to explain the action of the variables, but the exploratory question must be followed by higher-level questions if new knowledge is to be gained. Your methods and questions will change depending on what you find out as you go along. The results of the study will provide detailed descriptions of all the observations made by the researcher, arranged in some kind of order. Conclusions drawn from the data include some educated guesses or hypotheses for further study. A relationship between the observations made and a concept might be proposed. Further research would be required to test these proposals. This is the purpose of exploratory research. Descriptive Survey Designs Questions at Level II ask, What is the relationship between or among variables? You know what the variables are, and you know how to measure them, so you are beyond the scope of an exploratory study. The variables you are interested in have been studied before, either independently, as in an exploratory study, or with other variables, so that there is sufficient information to ask a question about the relationship between them. You are able to relate the variables in your study to a concept or conceptual framework so that the study does build on previous work. The major consideration is accuracy in the measurement of the variables. Designs for studies at Level II require a descriptive survey. The design dictates how the variables are to be measured in testing their relationship. In this type of design, the variables are partly controlled by the situation, as they are in exploratory design, but they are also partly controlled by the investigator, who chooses a sample for the study. You are not interested in controlling the other variables, merely in observing them. For example, in a study of the relationship between educational level of drivers and ability to make sound judgments about traffic signs, the investigator controls the first variable by selecting a sample of drivers with all types of educational backgrounds. The judgments of these drivers are then assessed. The drivers judgments will be examined and evaluated as they occur. The purpose of the study will be accomplished by seeing if the occurrence of sound judgment is related to educational background. Many research questions ask about variables that cannot be subjected to experimental manipulation, either because the variables cannot be manipulated or because to look at them outside their natural setting would be meaningless. For example, in looking at factors leading to mental illness, it would be unethical to isolate a single factor, such as poverty, and manipulate it to see if it results in mental illness. An experimental design would require that subjects be assigned to groups placed at different levels of poverty. After the specified length of time had passed, the groups would be examined to see if mental illness had developed. The absurdity of such an approach is obvious.

Rather than using experimentation to discover the causes of mental illness, you start with the effect and select a sample of mentally ill patients. Then you look for variables that might be related to mental ill patients. You might find a significant relationship between poverty and mental illness. You might also establish that poverty precedes mental illness in time. However, you might discover that well-to-do persons are less likely to be diagnosed as mentally ill even when they have the same symptoms as person at the bottom of the poverty scale. Alternative explanations are always possible in descriptive surveys. Subjects differ on many factors, only a few of which can be controlled. Although absolute proof of causality cannot be established in a descriptive survey, it is possible to accumulate extensive evidence to support causality. No experimental research has been done with human subjects to see if road accidents can be induced by introducing slippery road surface. But, by showing that slippery road surface is the one variable preponderant in road accidents, support grows for the theory that the accidents can be caused by slippery road surfaces. Many variables of interest to transportation researchers cannot be experimentally manipulated. Attitudes, beliefs, or behaviours are concepts that are often thought of as causal in health, illness response to treatment and other effects. The descriptive survey can be of great value in the study of these variables. Experimental Designs All experimental designs have one central characteristic: they are based on manipulating the independent variable and measuring the effect on the dependent variable. Experimental designs result in inferences drawn from the data that explain the relationships between the variables. If your study is at Level III, an experimental design is best. Why questions are always experimental design. The classic experimental design consists of the experimental group and the control group. In the experimental group the independent variable is manipulated. In the control the dependent variable is measured when no alteration has been made on the independent variable. The dependent variable is measured in the experimental group the same way, and at the same time, as in the control group. The prediction is that the dependent variable in the experimental group will change in a specific way and that the dependent variable in the control group will not change. Experimental Group Control Group Independent Variable Changed Unchanged Dependent Variable Measured Measured

For the classic experimental design, then, there are two groups (experimental and control), two variables (independent and dependent), and one measurement of the dependent variable. When data are analyzed, only the measurements on the dependent variable for the two groups are contrasted. Controlling Unwanted Influences To obtain a reliable answer to the research question, the design should eliminate unwanted influences. The amount of control that the researcher has over the variables being studied varies, from very little in exploratory studies to a great deal in experimental design, but the limitations on control must be addressed in any research proposal. These unwanted influences stem from one or more of the following: extraneous variables, bias, the Hawthorne effect, and the passage of time. These four will be discussed in turn, along with some suggestions for controlling their effects. You will need to identify those that seem relevant to your question and show how you will control their effects.

Extraneous Variables Extraneous variables are variables that can interfere with the action of the ones you are studying. They could as easily have been chosen as independent variables had you been interested in them because of their known effect on your variables. Since they are not part of your study, their influence must be controlled. In the research literature, you will see extraneous variables also referred to as intervening variables, found in Level III experimental designs, directly affect the action of the independent variable on the dependent variables are those that occur in the study setting. They include economic, physical, and psychological variables. Extraneous variables usually are not a problem in Level I studies. Since you are studying only one variable in depth, these variables are assumed to be independent. They could be extraneous variables, as well. That is why an exploratory descriptive study explores in depth: all variables must be accounted for and taken into consideration in order to be described adequately. At Level II, when you do not know which variables are independent and which are dependent, you must assume that there may be variables that have not been accounted for that may be related dependent variable. Some variables have been shown to be related to one of your variables in other studies. These are known extraneous variables and must be considered when designing your study, so that their effect your variables can be controlled. At Level II, it is sometimes difficult to establish which one is the independent variable, even after confirming that there is a significant relationship between two variables. Unless you can demonstrate that one variable precedes the other in time, it may be impossible to determine which the independent variable is. Therefore, it is important to control extraneous variables when you can identify what they are, so that you can isolate the relationship between the ones you are interested in studying. At Level III, where you are predicting the relationship, your must be very careful to control all possible extraneous variables that might intervene in your test. These variables can be identified from the literature on your topic. Methods of controlling extraneous variables include:

randomization homogeneous sampling techniques matching building the variables into the design.

Randomization Theoretically, randomization is the only method of controlling all possible extraneous variables. The random assignment of subjects to the various treatment and control groups means that the groups can be considered statistically equal in all ways at the beginning of the experiment. It does not mean that they actually are equal for all variables. However, the probability of their being equal is greater than the probability of their not being equal, if the random assignment was carried out properly. The exception lies with small groups where random assignment could result in unequal distribution of crucial variables. If this possibility exists in your study, perhaps one the other method would be more appropriate. In most instances, however, randomization is the best method of controlling extraneous variables. Homogeneous Sample One simple and effective way of controlling an extraneous variable is not to allow it to vary. Choose a sample that is homogenous for that variable.

Matching When randomization is not possible, or when the experimental groups are too small and contain some crucial variables, subjects can be matched for those variables. The experimenter chooses subjects who match each other for the specified variables. One of these matched subjects is assigned to the control group and the other to the experimental group, thus ensuring the equality of the groups at the outset. The process of matching is time consuming and introduces considerable subjectivity into sample selection. Therefore, it should be avoided whenever possible. If you use matching, limit the number of groups to be matched and keep the number of variables for which the subjects are matched low. Matching with more than five variables becomes extremely cumbersome, and it is almost impossible to find enough matched partners for your sample. Matching may be used in all research designs (besides Level III) when you are looking at certain outcomes and want to have as much control as possible. Building Extraneous Variables into the Design When extraneous variables cannot be adequately controlled by randomization, they can be built into the design as independent variables. They would have to be added to the purpose of your study and tested for significance along with your other variables. In this way, their effect can be measured and separated from the effect of the variables you wanted to study initially. Particularly in experimental designs, but also in descriptive surveys, the effect of these variables can be subtracted statistically from the total action of the variables. This method adds to the cost of the study because of the additional data collection and analysis required. Therefore, it should be used only as a last resort. In exploratory descriptive studies where the nature of the variables is not known, extraneous variables are said to be built into the design. The purpose in these studies is to identify the relevant variables and assess their relationship in the data analysis. Therefore, it is essential that you treat all variables as independent during the data collection, so that no data that later might point to relationships between variables will be overlooked. The separation of extraneous variables from independent and dependent variables is part of the analysis of data in exploratory research. Bias Bias results from collecting the data in such a way that one answer to the research question is given undue favor over another. Since bias has to do with the evidence you collect to answer your question, it is a concern during the sample selection and data collection phases of the research process. All researchers have biased views of their own questions. You know what you want the results to be and, if you are not careful, you will unconsciously sway the study in that direction. Therefore, when you can take precautions to maintain objectivity you should do so. For example, if you have a choice between randomly selecting your subjects and subjectively picking them out of a group, you should choose random selection. Bias during the data collection means that the researcher either is influencing the responses of the subjects or is selectively recording observations according to conscious or unconscious predispositions. Both of these problems are present in exploratory studies, especially those where the researcher is involved in the setting with the subjects. If data collection is unstructured and the researcher interacts with the subjects to any great extent, it is virtually impossible to eliminate bias from the study. It is possible, however, to plan for maximum objectivity by establishing some checks and balances. Questionnaires and Interviews

Asking questions can provide measurement of many concepts and variables important to transportation research. Transportation practitioners ask questions frequently as part of assessment and evaluation. The important thing to remember when choosing this method is that it must be the most appropriate one to measure the variables as you have defined them. Whether you use the interview or questionnaire method, it must be because your operational definition calls for the subjects self-report. If it does not, or if there is reason to believe that the person cannot give a valid response then these methods are not appropriate. The major difference between questionnaires and interviews is the presence of an interviewer. In questionnaires, responses are limited to answers to predetermine questions. In interviews, since the interviewer is present with the subject, there is an opportunity to collect nonverbal data as well and to clarify the meaning of questions if the subjects do not understand. The written questionnaire has some advantages. For one thing, it is likely to be less expensive, particularly in terms of the time spent collecting the data. Questionnaires can be given to large numbers of people simultaneously; they can also be sent by mail. Therefore, it is possible to cover wide geographic areas and to question large number of people relatively inexpensively. Another advantage of questionnaires is that subjects are more likely to feel that they can remain anonymous and thus may be more likely to express controversial opinions. This is more difficult in an interview, where the opinion must be given directly to the interviewer. Also, the written question is standard from one subject to the next and is not susceptible to changes in emphasis as can be case in oral questioning. There is always the possibility, however, that the written question will be interpreted differently by different readers, which is one reason for carefully pre-testing questionnaires. Interviews have many advantages, the most significant of which is questioning people who cannot write their responses. This category also includes illiterate subjects or subjects who do not write as frequently as they speak. Oral responses from these individuals will contain much more information than would their written responses. Another advantage of the interview method is that is actually results in a higher response rate than does the questionnaire. Many people who would ignore a questionnaire are willing to talk, with an interviewer who is obviously interested in what they have to say. Another advantage of the interview technique is that you can plan to ask questions at several levels to get the most information from the subject. This approach is unique to the interview. The combination of structured and unstructured questions can provide depth and richness to the data and, at the same time elicit data that are comparable from one subject to the next. When looking for a questionnaire or interview schedule to use in your study or when developing your own tool, you will have to consider the various kinds of questions that you can ask to obtain a range of data, and then decide which method is best suited to your variables. The content of the questions must be considered first, then the amount of structure in the format. Question content or the purpose of the question falls into two basic categories: those aimed at facts and those aimed at perceptions or feelings. Factual questions ask subjects for information about themselves or about even or people about which they know something. Criteria for Selecting the Interview or Questionnaire Advantages of the Questionnaire The subject needs be able to This approach is less read or write expensive in terms of time Advantages of the Interview 9

and money The interviewer can observe Subjects feel a greater the responses of the subject sense of anonymity The format is standard for Questions may be clarified if all subjects and is not they are misunderstood dependent on mood of interviewer An-depth data may be obtained Large samples, covering on any subject and are not large geographic areas, dependent on predetermined compensate for the questions expected loss of subjects A greater amount of data There is a higher response and over a broad range of retention rate topics may be collected Degree of Structure in Questionnaires & Interviews Structured questionnaires and interviews are those in which the questions are presented in exactly the same way, with the same wording, and in the same order to all subjects. The questions are standardized to ensure that the subjects answers can be compared. The questions can be asked by an interviewer or can be given to the subject as a paper-pencil test; in either case, the questions are asked in the same order for all subjects so that the order of the questions cannot affect the subjects responses. The most structured question are fixed alternative questions which the subject is asked to choose one of the given alternatives. Some examples of fixed alternative questions are as A. Check the response that best describes how you level of satisfaction with planned road. Very satisfied Satisfied Agree Not Satisfied Neutral

B. Which of the following is your choice of specialty area? Choose only one: 1. Driver 2. Material 3. Passenger Supplier

Questions such as these are the same whether used in a questionnaire or in an interview. They are more commonly used in questionnaires but may be used in interviews, particularly if the subject is unable or unwilling to fill out a questionnaire. In exploratory research, it may not be appropriate to structure the interview questions in advance, other than to decide on the opening statement or question. A flexible interview, properly used can bring out much useful material because it allows the interviewee to pursue whatever seems important to the subjects and thus elicit the subjects values, beliefs, and attitudes. Their responses will be completely spontaneous, self revealing, and personal. The flexibility of the interview is both an advantage and disadvantage to the researcher. The results will not be comparable from one subject to the next because the interview format is never the same. However, the interview is invaluable in exploring the whole range of attitudes, thoughts, and feelings that exist for the topic.

10

In a questionnaire, it is difficult to be unstructured. Some degree of structure is always required because you must set your questions in advance and cannot change them according to the subjects responses. Questions that do not have fixed alternatives, however, are much less structured than those that do, because they require subjects to respond in their own words. The extent of the response that the subject must provide to answer the question will vary from a word to a sentence, a paragraph, or even an essay. The least structured questionnaires are those designed to elicit extensive response from the subject. For example: What is "Level of Measurement"? The level of measurement refers to the relationship among the values that are assigned to the attributes for a variable. Knowing the level of measurement helps you decide how to interpret the data from that variable. It also helps you decide what statistical analysis is appropriate on the values that were assigned. If a measure is nominal, then you know that you would never average the data values or do a t-test on the data. There are typically four levels of measurement that are defined:

Nominal (attributes are only named) Ordinal ( attributes can be ordered) Interval (distance is meaningful) Ratio (absolute zero)

Nominal Measurement In nominal (means naming) measurement the numerical values just "name" the attribute uniquely. No ordering of the cases is implied. For example, gender of class of people is measures at the nominal level as female or male or 1 and 2. A player with number 2 is not more of anything than person with number 1, and is certainly not twice whatever number 1 is. Ordinal Measurement In ordinal (means rank) measurement the attributes can be rank-ordered. Here, distances between attributes do not have any meaning. For example, on a survey you might code Educational Attainment as 0=less than primary school; 1=some primary school; 2=high school certificate; 3= undergraduate degree; 4=maters degree; 5=post master degree. In this measure, higher numbers mean more education. But is distance from 0 to 1 same as 3 to 4? of course not. The interval between values is not interpretable in an ordinal measure. Interval Measurement In interval measurement the distance between attributes does have meaning. For example, when we measure temperature (in Fahrenheit), the distance from 30-40 is same as distance from 70-80. The interval between values is interpretable. Because of this, it makes sense to compute an average of an interval variable, where it doesn't make sense to do so for ordinal scales. But note that in interval measurement ratios don't make any sense - 80 degrees is not twice as hot as 40 degrees (although the attribute value is twice as large) Ratio Measurement In ratio measurement there is always an absolute zero that is meaningful. This means that you can construct a meaningful fraction (or ratio) with a ratio variable. Weight is a ratio variable. In applied transportation research most "count" variables are ratio, for example, the number of drivers in past six months. Why? Because you can have zero drivers and because it is meaningful to say that "...we had twice as many drivers in the past six months as we did in the previous six months."

11

It's important to recognize that there is a hierarchy implied in the level of measurement idea. At lower levels of measurement, assumptions tend to be less restrictive and data analyses tend to be less sensitive and they tend to level I type of research. At each level up the hierarchy, the current level includes all of the qualities of the one below it and adds something new. In general, it is desirable to have a higher level of measurement. Reliability and Validity of Measurement Each step of the research process depends on the proceeding steps. If a step is missing or inaccurate, then the succeeding steps with fail. When developing your research plan, be aware that this principle critically affects your progress. For instance, if you asked your question correctly, you can perform an adequate literature review. A good literature review is basic to the purpose of the study. The purpose of the study is basic to the operational definitions. Once you have operationalized your terms, you can proceed with the design, sample selection, and method of data collection. The concepts of reliability and validity will be discussed now, as they ultimately will influence the data analysis and the outcome of the final report. Reliability and Validity Reliability and validity, in research, refer specifically to the measurement of data as they will be used to answer the research question. In most case, the instrument that measures your variables is the central issue in determining the reliability and validity of the data; however the data analysis process often is also an issue if the data unstructured. Whatever data collection method is used, the intent must be accuracy. The reliability of your data collection refers to its consistency, stability, and repeatability-all of which determine how much you can rely on the results. If you were to measure the same variable in the same subject again, would your result be the same? In addition to reliability, you need to know if the measurement technique used to collect data actually measures what it is supposed to measure; in other words, is it valid? If you use a questionnaire to measure an individuals driving values, did the questionnaire, in fact, measure that concept or something else? Reliability &Validity in Your Research Plan In exploratory descriptive studies, validation procedures frequently reflect self-evident measures or; at most, pragmatic measures; they rely heavily on reliability tests. At the descriptive survey level of design, you need to be concerned with the reliability of your measurements since accuracy in measurement is the key to a reliable Level II study. Instruments must be tested for reliability and validity, and frequently this is done in a pilot study. Even if the instrument has been previously tested in another study, it should be retested as part of your study because it has been shown that both reliability and validity can change overtime. Neither is constant.Level III experimental designs ideally require instruments that have construct validity. Only in this way can there be true confidence in the results of experiments. This factor must be considered in interpreting the results of these studies.Each level of research requires some act of reliability and validity testing of the measurements, whether the research tool is you, the observer, a questionnaire, or a mechanical device. The point is that the results obtained from your measurement should be true results and not due to some error in your instrument. These critical concepts of reliability and validity can make the difference between good research and poor research. Data collection methods

Personal interviews Telephone interviews 12

Mail surveys Internet surveys

We always recommend using the data collection method which, we believe, will work best given the nature of the data to be collected, and the time-constraints which may be in place, and of course using a well-designed survey instrument. Regardless of what data collection method we employ, we generally charge about 50%-60% of what our nationally recognized competitors charge. And, while they only provide "cross tabs" for that money, we always undertake a detailed data analysis, draw well thought-out conclusions, provide readily implementable recommendations, and make an "in-person" final presentations to enable our clients to probe, question, and challenge our findings, conclusions and recommendations in whatever detail they desire. Personal Interviews. There are many times that we use personal interviews. They work very well when it is necessary to display special effects, such as "story boards," television or radio advertisements," or graphic displays. For example, personal interviews may involve "mall intercepts" or interviews in homes, in offices, or in restaurants. While we often use subcontractors or hire interviewers to conduct interviews, we ensure that they are well-trained and wellsupervised. Telephone Interviews. For very simple surveys or when very quick response time is necessary, telephone interviews work great. We've long worked with a telephone subcontractor who can have 300 interviewers on the phone simultaneously in one location. If we're doing a substantial telephone interviewing effort, we'll ordinarily expect them to complete all the interviews within a 2 or 3 week time frame. However, for an urgent "down-and-dirty" effort, they've invariably come to our aid because they truly enjoy working with Millennium Marketing. They are kind enough to accept a job on a Friday afternoon and provide us with "top line" results on Monday. All at no extra cost! Mail Surveys. When the survey instrument is complex, we find that mail surveys work quite nicely. We've long said that the design of the survey instrument is a learned skill, and over the years we've learned many valuable lessons about survey design. Tom Schori, one of our principals, had developed a procedure which we term the Optimal Brand Positioning Model. It requires the gathering of some pretty complex information. Mike Garee, our other principal, developed a seven-page survey instrument, which was then mailed to those within the client's target market. When the completed questionnaires started arriving, it became very apparent that respondents were having a great deal of trouble completing the questionnaire. Tom was perplexed until it dawned on him that every other time he'd employed the Optimal Brand Positioning Model, he had gathered the necessary information with a one-page questionnaire. Mike then redesigned the questionnaire, which we re-fielded at our own expense. Not only did respondents now fill out the questionnaires completely, the response rate for the re-fielded questionnaire was twice that which we had expected. Many people have the false impression that mail surveys take a long time to field and generate useful data. The fact of the matter is that 90% of the responses will have been received after the instrument has been in the field just a week! Internet Surveys. Anything that can be done with a mail survey of course can also be accomplished with an Internet survey, and much more. And it is really fast. Bringing that point home, Tom Schori, one of our principals, who has taught marketing management and marketing research for many years, recently conducted an Internet survey project with his marketing research students. Individuals within the target market were contacted via personalized e-mail and were invited to participate in a research effort by following a hyperlink to the site where the survey instrument was stored. When a respondent finished the survey, he/she simply clicked on "submit" and his/her responses were stored in a secure site for later data analysis. Within four days after sending out the e-mail invitations to participate, about 1,200 surveys had been fully completed. This represented about a 70% response rate among 13

those with active e-mail accounts. Outstanding! You can bet that more and more survey efforts will be conducted over the Internet over the coming years.

Steps in research design Step 1: Problem definition The point has already been made that the decision-maker should clearly communicate the purpose of the research to the marketing researcher but it is often the case that the objectives are not fully explained to the individual carrying out the study. Decision-makers seldom work out their objectives fully or, if they have, they are not willing to fully disclose them. In theory, responsibility for ensuring that the research proceeds along clearly defined lines rests with the decision-maker. In many instances the researcher has to take the initiative. In situations, in which the researcher senses that the decision-maker is either unwilling or unable to fully articulate the objectives then he/she will have to pursue an indirect line of questioning. One approach is to take the problem statement supplied by the decision-maker and to break this down into key components and/or terms and to explore these with the decision-maker. For example, the decisionmaker could be asked what he has in mind when he uses the term market potential. This is a legitimate question since the researcher is charged with the responsibility to develop a research design which will provide the right kind of information. Another approach is to focus the discussions with the person commissioning the research on the decisions which would be made given alternative findings which the study might come up with. This process frequently proves of great value to the decision-maker in that it helps him think through the objectives and perhaps select the most important of the objectives. Step 2: Hypothesis generation A hypothesis is a conjectural statement regarding the relation between two or more variables. There are two key characteristics which all hypotheses must have: they must be statements of the relationship between variables and they must carry clear implications for testing the stated relations. These characteristics imply that it is relationships, rather than variables, which are tested; the hypotheses specify how the variables are related and that these are measurable or potentially measurable. Statements lacking any or all of these characteristics are not research hypotheses. For example, consider the following hypothesis: Step 3: Decision on type of study Marketing research can be carried out on one of three levels: exploratory, descriptive or causal. Exploratory research: The chief purpose of exploratory research is to reach a better understanding of the research problem. This includes helping to identify the variables which should be measured within the study. When there is little understanding of the topic it is impossible to formulate hypotheses without some exploratory studies. Exploratory research is intended to help researchers formulate a problem in such a way that it can be researched and suggest testable hypotheses. Descriptive research: As the name suggests, descriptive research is concerned with describing market characteristics and/or marketing mix characteristics. Typically, a descriptive study specifies the number and size of market segments, the alternative ways in which products are currently distributed, listing and comparison of the attributes and features of competitive products, etc.

14

This type of study can involve the description of the extent of association between variables. For example, the researcher may observe that there is an association between the geographical location of consumers and their tendency to consume red meat. Note that the researcher is able to describe the relationship rather than explain it. Nonetheless if the relationship between the two is fairly stable this descriptive information may be sufficient for the purposes of prediction. The researcher may, for example, be able to predict how fast the per capita consumption of red meat is likely to rise over a given time period. Causal research: Causal research deals with the "why" questions. That is, there are occasions when the researcher will want to know why a change in one variable brings about a change in another. If he/she can understand the causes of the effects observed then our ability to predict and control such events is increased. Step 4: Decision on data collection method The next set of decisions concerns the method(s) of data gathering to be employed. The main methods of data collection are secondary data searches, observation, the survey, experimentation and consumer panels. Each of these topics is dealt with later on, so they are simply noted here. Step 5: Development of an analysis plan Those new to marketing research often intuitively believe that decisions about the techniques of analysis to be used can be left until after the data has been collected. Such an approach is ill-advised. Before interviews are conducted the following checklist should be applied: Is it known how each and every question is to be analysed? (e.g. which univariate or bivariate descriptive statistics, tests of association, parametric or nonparametric hypotheses tests, or multivariate methods are to be used?) Does the researcher have a sufficiently sound grasp of these techniques to apply them with confidence and to explain them to the decision-maker who commissioned the study? Does the researcher have the means to perform these calculations? (e.g. access to a computer which has an analysis program which he/she is familiar with? Or, if the calculations have to be performed manually, is there sufficient time to complete them and then to check them?) There is little point in spending time and money on collecting data which subsequently is not or cannot be analysed. Therefore consideration has to be given to issues such as these before the fieldwork is undertaken. Step 6: Data collection At this stage the researcher is ready to go into the field and collect data. The various issues relating to data collection constitute the main body of the text and therefore, are not dwelt upon here. Step 7: Analysis of data The word 'analysis' has two component parts, the prefix 'ana' meaning 'above' and the Greek root 'lysis' meaning 'to break up or dissolve'. Thus data analysis can be described as: "...a process of resolving data into its constituent components, to reveal its characteristic elements and structure."

15

Where the data is quantitative there are three determinants of the appropriate statistical tools for the purposes of analysis. These are the number of samples to be compared, whether the samples being compared are independent of one another and the level of data measurement. Data Collection Method

Internal data usually centers on sales or cost information & includes:

o o o o o

Sales invoices - customer profiles, sales trends. Accounts receivable reports - profit margins. Quarterly sales reports - dollar and unit sales. Sales activity reports - sales and competition. Other forms - customer letters/ comment cards, mail-order forms, credit applications, etc.

Steps involve in questionair designing

Methodology-Data collection Sample and sampling method Statistical or qualitative methods used for data analysis Sample description Findings 16

 Results, interpretation and conclusions. The summary of findings is perhaps the most important component of the written report, since many of the management team who are to receive a copy of the report will only read this section. The summary of findings is usually put right after the title page, or is bound separately and presented together with the report. The introduction should describe the background of the study and the details of the research problem. Following that, automatically the broad aim of the research can be specified, which is then translated into a number of specific objectives. Furthermore, the hypotheses that are to be tested in the research are stated in this section. In the methodology chapter the sampling methods and procedures are described, as well as the different statistical methods that are used for data analysis. Finally, the sample is described, giving the overall statistics, usually consisting of frequency counts for the various sample characteristics.. Once the sample has been described, the main findings are to be presented in such a way that all objectives of the study are achieved and the hypotheses are tested. As mentioned before, it is essential that the main findings are well interpreted and conclusions are drawn wherever possible. Data presentation Easy-to-understand tables and graphics will greatly enhance the readability of the written research report. As a general rule, all tables and figures should contain: 1. Identification number corresponding to the list of tables and the list of figures 2. A title that conveys the content of the table or figure, also corresponding to the list of tables and the list of figures, and 3. Appropriate column labels and row labels for tables, and figure legends defining specific elements in the figure. There are a number of ways to produce tables and figures. When typing a report on a typewriter or word-processor, it is sometimes easiest to type a table out by hand. However, when complicated tables have to be produced, it is advisable to use spreadsheet software like Lotus 123 or Excel The nature of secondary sources of information Secondary data is data which has been collected by individuals or agencies for purposes other than those of our particular research study. For example, if a government department has conducted a survey of, say, family food expenditures, then a food manufacturer might use this data in the organisation's evaluations of the total potential market for a new product. Similarly, statistics prepared by a ministry on agricultural production will prove useful to a whole host of people and organisations, including those marketing agricultural supplies. No marketing research study should be undertaken without a prior search of secondary sources (also termed desk research). There are several grounds for making such a bold statement. Secondary data may be available which is entirely appropriate and wholly adequate to draw conclusions and answer the question or solve the problem. Sometimes primary data collection simply is not necessary.

17

 It is far cheaper to collect secondary data than to obtain primary data. For the same level of research budget a thorough examination of secondary sources can yield a great deal more information than can be had through a primary data collection exercise. The time involved in searching secondary sources is much less than that needed to complete primary data collection. Secondary sources of information can yield more accurate data than that obtained through primary research. This is not always true but where a government or international agency has undertaken a large scale survey, or even a census, this is likely to yield far more accurate results than custom designed and executed surveys when these are based on relatively small sample sizes. .

The problems of secondary sources Whilst the benefits of secondary sources are considerable, their shortcomings have to be acknowledged. There is a need to evaluate the quality of both the source of the data and the data itself. The main problems may be categorised as follows: Definitions The researcher has to be careful, when making use of secondary data, of the definitions used by those responsible for its preparation. Suppose, for example, researchers are interested in rural communities and their average family size. If published statistics are consulted then a check must be done on how terms such as "family size" have been defined. They may refer only to the nucleus family or include the extended family. Even apparently simple terms such as 'farm size' need careful handling. Such figures may refer to any one of the following: the land an individual owns, the land an individual owns plus any additional land he/she rents, the land an individual owns minus any land he/she rents out, all of his land or only that part of it which he actually cultivates. It should be noted that definitions may change over time and where this is not recognised erroneous conclusions may be drawn. Geographical areas may have their boundaries redefined, units of measurement and grades may change and imported goods can be reclassified from time to time for purposes of levying customs and excise duties. Measurement error When a researcher conducts fieldwork she/he is possibly able to estimate inaccuracies in measurement through the standard deviation and standard error, but these are sometimes not published in secondary sources. The only solution is to try to speak to the individuals involved in the collection of the data to obtain some guidance on the level of accuracy of the data. The problem is sometimes not so much 'error' but differences in levels of accuracy required by decision makers. When the research has to do with large investments in, say, food manufacturing, management 18

will want to set very tight margins of error in making market demand estimates. In other cases, having a high level of accuracy is not so critical. For instance, if a food manufacturer is merely assessing the prospects for one more flavour for a snack food already produced by the company then there is no need for highly accurate estimates in order to make the investment decision. Source bias Researchers have to be aware of vested interests when they consult secondary sources. Those responsible for their compilation may have reasons for wishing to present a more optimistic or pessimistic set of results for their organisation. It is not unknown, for example, for officials responsible for estimating food shortages to exaggerate figures before sending aid requests to potential donors. Similarly, and with equal frequency, commercial organisations have been known to inflate estimates of their market shares. Reliability The reliability of published statistics may vary over time. It is not uncommon, for example, for the systems of collecting data to have changed over time but without any indication of this to the reader of published statistics. Geographical or administrative boundaries may be changed by government, or the basis for stratifying a sample may have altered. Other aspects of research methodology that affect the reliability of secondary data is the sample size, response rate, questionnaire design and modes of analysis. Time scale Most censuses take place at 10 year intervals, so data from this and other published sources may be out-of-date at the time the researcher wants to make use of the statistics. The time period during which secondary data was first compiled may have a substantial effect upon the nature of the data. For instance, the significant increase in the price obtained for Ugandan coffee in the mid-90's could be interpreted as evidence of the effectiveness of the rehabilitation programme that set out to restore coffee estates which had fallen into a state of disrepair. However, more knowledgeable coffee market experts would interpret the rise in Ugandan coffee prices in the context of large scale destruction of the Brazilian coffee crop, due to heavy frosts, in 1994, Brazil being the largest coffee producer in the world. Sources of information Secondary sources of information may be divided into two categories: internal sources and external sources. Internal sources of secondary information Sales data : All organisations collect information in the course of their everyday operations. Orders are received and delivered, costs are recorded, sales personnel submit visit reports, invoices are sent out, returned goods are recorded and so on. Much of this information is of potential use in marketing research but a surprising amount of it is actually used. Organisations frequently overlook this valuable resource by not beginning their search of secondary sources with an internal audit of sales invoices, orders, inquiries about products not stocked, returns from customers and sales force customer calling sheets. For example, consider how much information can be obtained from sales orders and invoices: Sales by territory Sales by customer type Prices and discounts Average size of order by customer, customer type, geographical area Average sales by sales person and Sales by pack size and pack type, etc. This type of data is useful for identifying an organisation's most profitable product and customers. It can also serve to track trends within the enterprise's existing customer group.

19

Financial data: An organisation has a great deal of data within its files on the cost of producing, storing, transporting and marketing each of its products and product lines. Such data has many uses in marketing research including allowing measurement of the efficiency of marketing operations. It can also be used to estimate the costs attached to new products under consideration, of particular utilisation (in production, storage and transportation) at which an organisation's unit costs begin to fall. Transport data: Companies that keep good records relating to their transport operations are well placed to establish which are the most profitable routes, and loads, as well as the most cost effective routing patterns. Good data on transport operations enables the enterprise to perform trade-off analysis and thereby establish whether it makes economic sense to own or hire vehicles, or the point at which a balance of the two gives the best financial outcome. Storage data: The rate of stockturn, stockhandling costs, assessing the efficiency of certain marketing operations and the efficiency of the marketing system as a whole. More sophisticated accounting systems assign costs to the cubic space occupied by individual products and the time period over which the product occupies the space. These systems can be further refined so that the profitability per unit, and rate of sale, are added. In this way, the direct product profitability can be calculated. External sources of secondary information The marketing researcher who seriously seeks after useful secondary data is more often surprised by its abundance than by its scarcity. Too often, the researcher has secretly (sometimes subconsciously) concluded from the outset that his/her topic of study is so unique or specialised that a research of secondary sources is futile. Consequently, only a specified search is made with no real expectation of sources. Cursory researches become a self-fulfilling prophecy. Dillon et. al3 give the following advice: "You should never begin a half-hearted search with the assumption that what is being sought is so unique that no one else has ever bothered to collect it and publish it. On the contrary, assume there are scrolling secondary data that should help provide definition and scope for the primary research effort." The same authors support their advice by citing the large numbers of organisations that provide marketing information including national and local government agencies, quasi-government agencies, trade associations, universities, research institutes, financial institutions, specialist suppliers of secondary marketing data and professional marketing research enterprises. Dillon et al further advise that searches of printed sources of secondary data begin with referral texts such as directories, indexes, handbooks and guides. These sorts of publications rarely provide the data in which the researcher is interested but serve in helping him/her locate potentially useful data sources. The main sources of external secondary sources are (1) government (federal, state and local) (2) trade associations (3) commercial services (4) national and international institutions. Government statistics These may include all or some of the following: Population censuses Social surveys, family expenditure surveys Import/export statistics Production statistics Agricultural statistics. Trade associations Trade associations differ widely in the extent of their data collection and information dissemination activities. However, it is worth checking with them to determine what they do publish. At the very least one would normally expect that they would produce a trade directory and, perhaps, a yearbook. Commercial services Published market research reports and other publications are available from a wide range of organisations which charge for their information. Typically, marketing people are interested 20

in media statistics and consumer information which has been obtained from large scale consumer or farmer panels. The commercial organisation funds the collection of the data, which is wide ranging in its content, and hopes to make its money from selling this data to interested parties. National and international institutions Bank economic reviews, university research reports, journals and articles are all useful sources to contact. International agencies such as World Bank, IMF, IFAD, UNDP, ITC, FAO and ILO produce a plethora of secondary data which can prove extremely useful to the marketing researcher. Preliminary decisions in questionnaire design There are nine steps involved in the development of a questionnaire: 1. Decide the information required. 2. Define the target respondents. 3. Choose the method(s) of reaching your target respondents. 4. Decide on question content. 5. Develop the question wording. 6. Put questions into a meaningful order and format. 7. Check the length of the questionnaire. 8. Pre-test the questionnaire. 9. Develop the final survey form.

Types of Sampling Methods The types of sampling roughly fall into two main categories - probability sampling and judgement sampling. Most sampling theory has been developed for probability sampling and we will later consider the detailed theory of some of these methods. First an outline/description of sampling methods. Probability Sampling
o o

Simple Random Sampling Stratified Random Sampling 21

o o o

Cluster Sampling Multistage Sampling Systematic Sampling

NonProbability/Judgement Sampling
o o

Quota Sampling Purposive Sampling

Probability Sampling In probability sampling it is possible to state which units belong to which sample, and each sample has a known probability of being selected.

Simple Random Sampling Stratified Random Sampling Cluster Sampling Multistage Sampling Systematic Sampling

Simple Random Sampling Here each sample of size n from the population of size N has an equal chance of selection. In practice ``each unit in the population is numbered 1 to N and n units are randomly drawn from the N''. Advantages (i) simple to apply (ii) analysis of data is reasonably easy and has a sound mathematical basis Disadvantages (i) if population heterogeneous estimates have large variance Stratified Random Sampling In stratified sampling the population of N units is first divided into non-overlapping subpopulations called strata. If sampling from the strata is simple random sampling then whole procedure is called stratified random sampling. Stratification is a common technique : (i) If data of known precision is wanted for certain subdivisions of the population, then each subdivision or strata can be treated as a population. (ii) Administrative convenience may dictate its use, so that each field office can supervise one strata. (iii) Sampling problems may differ markedly within a population (eg people in prisons and people outside). (iv) Stratification will almost certainly produce a gain in precision in the estimates of the whole population, because a heterogeneous population is split into fairly homogeneous strata. Advantages : (i) to (iv) above . Disadvantages 22

(i) problems if strata not clearly defined. (ii) analysis is (or can be) quite complicated. Cluster Sampling Suppose that a survey is to be done in a large town and that the unit of enquiry is the individual household. Suppose further that the town contains 20,000 households, all listed on convenient records, and a sample of 200 is needed. A simple random sample of 200 could well spread over the whole town incurring high costs and much inconvenience. However one might decide to concentrate the sample in a few parts of the town. Suppose for simplicity the town can be divides into 400 areas with 50 households in each then one could select at random 4 areas (1/100) and include all households in these areas. Note that, unlike stratified sampling, the clusters are thought of as being typical of the population, rather than subsections. In fact clusters neet not necessarily be natural aggregates, but can simply be artificial (grids on a map). Note also that a cluster of units in one survey may be a unit in another, eg if household is the unit then cluster is group of households, if family member is unit then the cluster is household. Also in any one design several levels of cluster may be used :

Constituencies Wards Polling Districts Households

Advantages (i) reduced field costs (ii) applicable where no complete list of units is available (special lists only need be formed for clusters). Disadvantages (i) clusters may not be representative of whole population but may be too alike (ii) analysis more complicated than for simple random sampling. Multistage Sampling The example above is of multistage sampling using clusters. With ``large'' populations it is often necessary to carry out the sampling in 2 or more stages. For example, to survey the attitudes of school children ; (i) sample of education authorities (ii) sample of towns in each local authority (iii) sample of schools in each town (iv) sample of classes in each school (v) sample of children in each class The advantages and disadvantages are as for cluster sampling. Do not confuse multi-stage sampling with multiphase where some information is collected from the whole sample and additional information is collected from sub-samples of the full sample. Systematic Sampling At first sight this is very different. Suppose that the N units in the population are numbered 1 to N in some order. To select a systematic sample of n units, if then every k-th unit is selected commencing with a randomly chosen number between 1 and k. Hence the selection of the first unit determines the whole sample, e.g., N = 5,000, n = 250 therefore k = 5000/250 = 20. Therefore, select every 20th item commencing with (say) 6. 23

Question : Is it equivalent to simple random sampling? Strictly speaking the answer is No!, unless the list itself is in random order, which it never is (alphabetical, seniority, street number, etc). Advantages (i) easier to draw, without mistakes (cards in file) (ii) more precise than simple random sampling as more evenly spread over population Disadvantages (i) if list has periodic arrangement then it can fare very badly

24

FORMAT FOR A RESEARCH REPORT Research reports consist of several sections where appropriate, such as: Title Page, Abstract, Introduction, Method, Results, Discussion, References (and, occasionally, Appendix), which are briefly described here: Title Page: Includes the title, author, date, etc. Abstract: A brief summary of the study. Introduction: Explains the purpose of the paper. In most cases this summarizes the theoretical importance and previous research in the area and includes a clear statement of the research hypotheses or aims of the paper. Method: Describes the study in enough detail to permit another investigator to replicate it. The Method section is often divided into three subsections: a. Research participants/subjects b. Apparatus or Research Instruments/Tools (if necessary) c. Procedures. Results: Summarizes the data and the statistical treatment of them. Graphs and tables are included if they make the results more intelligible. Discussion: Evaluation and implications of the research, including how the results support or do not support the argument; comparison of results with previous research; and problems with the research. Appendices: Includes supplementary material not appropriate in the body of the report. References: All research referred to in the report are referenced using a recognized bibliographic format.

25