Some Notes on the Nature and Importance of Research Design within

Educational Research

Colin Lankshear
Visiting Scholar
Faculty of Education, Monash University

Origins

These notes are based on work being done with Michele Knobel toward the preparation
of a Handbook of Teacher Research, under contract to Open University Press.

The work we are doing builds on our philosophy of research. We begin from the belief
that research is a discursive practice that must be methodical, systematic, and accountable
to norms and standards of rigor with respect to validity, reliability and logic. In other
words, for something to count as research in the first place, it must be the kind of activity
that recognizes the need to be concerned with such norms and standards. It must be
systematic and methodical in its approach and procedures. Whether it is good quality
research will depend upon how methodical and systematic it is in its approach, and how
well it meets relevant norms and standards for validity and reliability. If an activity does
not reflect a concern with these sorts of things, from our perspective it simply is not
research.

In relation to this philosophy of research we have formed the opinion that a lot of studies
being undertaken in the name of teacher research and/or educational research are not
really research at all. This is because much of this work is not methodical, systematic, or
accountable to norms and standards in the way we have indicated. It includes much work
being done by students enrolled in university degree programs in Education, and by
academics employed in Education faculties of universities and other tertiary institutions,
as well as by people operating as “teacher researchers”. In many cases we thought that
this was because the people involved did not actually understand the discursive nature of
research, and that research has to display certain kinds of discursive qualities.

In 1999 we wrote a short introductory book commissioned by the Primary English

Teaching Association (PETA) that spelled out what we regarded as the most important
generic features of research, and made these as clear as possible. This book was called
Ways of Knowing (Knobel and Lankshear 1999). It dealt with the following generic
features of research. To be research, an inquiry must

• be based on a definite research problem or issue and good quality research questions
• be informed in appropriate ways by theory and concepts
• have an appropriate approach to collecting and organizing data (a data collection and
organization design)
• have an appropriate approach to analyzing data (a data analysis design)
• offer an informed interpretation of results, and its findings must be consistent with the
research question and implementation of the research design
We have found that many people do not understand the concept of research as a logical
procedure: a process of developing approaches to collecting, analyzing and interpreting
data that cohere (are coherent) with the questions they are asking, the problems they are
trying to address, and the resources for inquiry they have available to them. It seems to us
that, more than any other single thing, this is the aspect of research that people find most
difficult to grasp. Unfortunately, if this concept of research as a logic involving a
coherent design is not understood, then it may not be very useful to keep reading more
and more books and articles concerned with specific research methods and techniques.
This is because we will not be able to understand how and where these methods and
techniques fit into the overall activity of conducting methodical and systematic inquiry—
which is what research is.

The importance of understanding the concept of research design

In our view, comparatively few books about educational research focus clearly enough on
the research process as a logical process. It seems to us that good research is like a good
argument, in the technical sense of ‘argument’ (as understood by logicians). For
logicians, an argument is an attempt to assemble a set of claims or premises, and to
arrange them in such a way that a conclusion follows logically from these premises, or
can be inferred (deductively or inductively) from them in a coherent or non-contradictory
manner. Good arguments—that is, arguments that we are justified in accepting, even
though they may subsequently be displaced by better arguments—have two general
qualities:

• Their premises are strong, in the sense that we have good reasons for accepting them
(e.g., they are supported by good evidence, or are widely accepted by experts or
authorities in the area).

• Their premises are well organized, in the sense that we can accept the conclusion that
is drawn from them without offending against reasonable standards for deductive or
inductive inference.

Similarly, good quality research investigations contain strong evidence in the form of
data that is relevant to a question or a problem. In good quality research studies, such
evidence has been collected and organized and analyzed in ways that allow us to accept
that the findings based upon them are reasonable inferences. That is to say, they can be
inferred reasonably from the data and the analysis of data involved in the research.
Moreover, the data and analysis themselves are appropriate in terms of the kind of
problem involved and the kinds of question(s) being asked.

The key concept involved here is the idea of a design. A design can be thought of as an
appropriate procedure or guideline for doing something under certain conditions. For
example, a particular building design that may be very acceptable for areas where
earthquakes never occur are might not be acceptable for areas where earthquakes are
common.
Alternatively, we might think of different kinds of ‘designs’ for convincing people about
certain things under different kinds of conditions. Suppose that our purpose is to convince
people under discursive conditions where they can be expected to be reasonable, and
where we define ‘reason’ in terms of being ‘logical’. Under such conditions, inductive or
deductive forms of argument—e.g., the syllogism—might be highly appropriate designs
for obtaining consent or agreement. In other words, we can look at inductive and
deductive arguments as designs for the purpose of trying to convince (reasonable) people
to accept something on the grounds that it is reasonable. If, however, we are trying to
convince people who are quite unreasonable, and who never accept something because it
is reasonable to accept it, we may have to adopt other ‘designs’ for convincing them in
order to obtain their agreement or consent. In such contexts appropriate ‘designs’ might
include things like ‘manipulation’, ‘bribery’, ‘coercion’, ‘force’, and so on.

This is relevant to how we think of research. Our concept of research is the idea that we
try to understand aspects of the world more clearly, accurately, and predictably by
assuming

(i) that there is some kind of order to things, and

(ii) that this can be revealed by approaching the world in systematic and reasonable
ways.

Research aims to identify those aspects of the world we want to understand more clearly
and, having identified these, to set about examining them methodically, systematically,
and with the assistance of what we already know about the world. What we already know
about the world has been organized, summarized, and made meaningful by use of
theories, concepts, and bodies of information. In order to conduct further research into
aspects of the world we need to find ways of

(i) locating gaps in knowledge and understanding

(ii) framing these gaps clearly and concisely as problems and questions

(iii) working out systematic, methodical, and reasonable ways of exploring these gaps

(iv) seeing how we can assist our inquiries by using whatever relevant concepts,
theories, and information we already have available.

Another way of saying this is to say that we need to find ways of arranging these four
aspects into a research design.

Our research design will build on clearly and concisely framed problems and questions
and a clear sense of our research purposes—that is, what we hope to achieve through our
research. Our research purposes might be identified as a set of aims and objectives that
relate to our research focus in the form of our research question(s) and/or our research
problem. A clear and concise statement of our research purposes is absolutely essential
for doing good quality research.

The research design will also contain a theoretical and conceptual framework that helps
clarify the questions, problems and purposes we are concerned with, tells us what is
already known about these matters, and helps us understand how particular concepts and
elements of theory might be useful for our own inquiry.

Guided by careful analysis of our research questions and problems, by our conceptual and
theoretical framework, and by our reading of the research literature, our research design
will also contain a strategy for collecting and organizing data that is relevant to our
research problem(s) and/or question(s). This will be a strategy that enables us to collect
the right kind of data for addressing our research focus, for collecting good quality data
of an appropriate kind, and for collecting an appropriate amount of data for investigating
our research problem and question in a rigorous and illuminating way. To be more
precise, our research design will contain as one of its key components a data collection
design. On the basis of this data collection design, and in accord with the various
resources of time, money, energy, etc., available to us, we will eventually develop a data
collection plan.

In addition to a strategy for collecting and organizing our data we also need a strategy for
analyzing our data. This may be thought of as a research analysis design. The most
important thing about our data analysis strategy or design is that it must cohere with—or
be consistent with—our research purposes (our research question/problem and our aims
and objectives), as well as with our theoretical and conceptual framework and the kinds
and amount of data we collect. For example, if we want to collect and analyze a large
amount of data we will need to use forms of data analysis that can handle large amounts
of data and give us the kinds of findings we seek. Otherwise we will not be able to
complete the research. On the other hand, if we want to concentrate on less data but
analyze it in great depth, we need to use forms of data analysis that will give us the kind
of depth and detail we seek.

Finally, our research design must contain a strategy for interpreting the analysis of our
data in order to provide us with findings and conclusions from our research, and which
may allow us to advance recommendations or implications based on our investigation.

The important point to note here about the idea of a research design is that all of its
components must ‘fit’ with each other or ‘go together’ with each other in a coherent
manner. Our theoretical and conceptual framework must ‘fit’ with our research goals and
purposes. Likewise, our data collection strategy must ‘fit’ with our research purposes, our
conceptual and theoretical framework and our approach to data analysis. Our data
analysis strategy must fit with our data collection strategy, our conceptual and theoretical
framework and our research purposes. And our interpretive strategy must allow us to
relate what emerges from our data analysis back to our research purposes, and forward to
whatever recommendations and implications we want to suggest that pertain to our
purposes.
To repeat our message: unless one has a research design in this sense one is not really
doing research at all. If we do have a research design in this sense, the quality of our
research will be a function of how competently we develop and implement these
components of our research design.

Major components of research design and the relationships between them

To make these components and the relationships between them, as clear as possible we
use a series of diagrams.

Literature

• Problem • Data collection • Conclusions

• Questions • Data analysis • Results
• Aims • Interpretation • Recommendations

Conceptual Theoretical
framework Ideas, examples, framework
clues, models, etc. to
use

Diagram 1: Research Design Components Diagram

A second diagram simplifies the original diagram in a way that helps us to focus on data
collection and data analysis.

Diagram 2: Research Design Table

Research Purposes Data to Collect Data Analysis Approach Informants

By placing the data analysis next to the research purposes column we are reminded that
we have to collect data that is pertinent or relevant to our research question. The data has
to fit the question. Similarly, by placing the data analysis column next to the data
collection column we are reminded that we have to select methods and techniques of data
analysis that cohere with our data.
Finally, we have a column called ‘Informants’. In this context we are using ‘informants’
to refer to literature (particularly, published literature) that is relevant to our own
research. In this sense, informants are the people who have published work that is
relevant to our research purposes. (Note that within research contexts the term
‘informants’ has a more common meaning. This is where ‘informants’ refers to people
participating in the study, as research subjects, who inform us about matters we are
interested in by giving us interviews, or allowing us to observe them or collect data from
them in other ways. We are using ‘informants’ here, however, to refer to other
researchers and theorists who inform our study at the more general level of designing the
study, as opposed to those whose data informs the implementation of the study.)

Through their writing, informants provide us with options for the kind of theoretical
approach we take to framing our research, and with key concepts or constructs around
which we can organize our inquiry. The informants are crucial for helping us determine
how to make our data collection and data analysis ‘fit’ (or be coherent with) our research
question. Very often there will be more than one kind of theoretical approach, and more
than one conceptual framework, that we could use to produce a coherent research design.
When that occurs we have a choice. We might choose a theory and a framework that we
find more attractive for personal and professional reasons. Or we might choose a
framework that has been used successfully by other researchers whose work we find
interesting and useful for addressing questions and problems similar to our own. And so
on. But whatever theory and conceptual framework we develop for our study, it must
cohere with our problem and question, and it must be capable of informing us how to
collect appropriate data and analyze it in appropriate ways.

The Informants column has been placed at the opposite end of the diagram from the
Research Purposes column. This is in order to remind us that we need to use our reading
of the literature to help us ensure that all the components cohere with each other. We
need to keep reading the literature until we are confident that we understand what our
question really is and what kinds of theory and concepts ‘fit’ our question. We also need
to use that information to help us make coherent and justifiable decisions about what
kinds of data to collect and how to analyze it. We are aware that we have repeated this
point many times, but we do not think it can be repeated too often. Unless the
components of the research design are all coherent with one another the ‘research’ will be
either poor quality research or, alternatively, not really research at all. So, the Research
Purposes and Informants columns are like ‘book ends’. They are necessary for keeping
the data collection and data analysis components ‘together’ in a proper manner. We
cannot determine what data to collect and how to analyze it without taking fully into
account the kind of research problem and question we have. But at the same time, we
cannot know what the options are and what the most effective choices will be without
referring to guidance from Informants in the literature.

To understand and use this table to guide our research, it is useful to think in terms of a
number of key questions that we can put in each column. These questions act as a guide
to help us structure our research project coherently. In other words, in order to conduct
our research rigorously we need to produce answers to each of these questions. Our
research unfolds as we answer these questions and implement the tasks and follow the
leads provided by our answers.

Questions relating to the Research Purposes column include

• What is my research problem?

• What are the key dimensions of this problem?
• What specific questions does this problem generate?
• Which of these questions can I hope to address well?
• How can I analyze my problem and questions to specify a succinct set of research
aims and objectives, and a finite set of research questions?

Questions relating to the Data column include

• What kinds of research data will I (need to) collect?

• How will this data help/allow me to address my research purposes?
• How will I collect this data?
• How much data will I collect?
• How will I validate this data or establish that it is good quality data?
• How will I organize this data so it will be in good shape for being analyzed?

Questions relating to the Data Analysis column include

• What forms of data analysis will I use?

• What justifies these forms of analysis? How will they help me achieve my research
purposes?
• How are these forms of analysis conducted?
• What do I need to know, be able to do and have access to do in order to use these
forms of analysis in an expert way?
• What will these forms of analysis let me ‘say’ about the data (and not say)?

Questions relating to the Informants column include

• Who writes about the aspects covered in the previous questions, and who can
therefore help guide us in our work? In more specific terms, the following questions
may be asked:
• Who helps us understand how to frame and refine research purposes? What are some
relevant books, articles, and chapters that we can read here?
• Who provides us with theories and concepts that are relevant to our research problem
and questions, and that will guide us in deciding what kinds of data to collect and
how to analyze it?
• Who provides good advice on how to collect, validate and organize data? What are
some relevant books, articles, and chapters that we can read here?
• Who provides good information about how to analyze data? What are some relevant
books, articles, and chapters that we can read here?

To make the point as clearly as we can, our previous diagram will now look like this.

Diagram 3: Research Design Table as Questions to be Addressed when Developing

our Design

Research Purposes Data to Collect Data Analysis Informants

Approach
• What is my • What kinds of • What forms of data • Who helps us understand
research research data must I analysis will I use? how to frame and refine
problem? collect? • What justifies these research purposes?
• What are its key • How will this data forms of analysis? • What are some relevant
dimensions? help me address my • How will they help books, articles, and
• What specific purposes? me achieve my chapters here?
questions does it • How will I collect research purposes? • Who provides theories
generate? this data? • How are these and concepts relevant to
• Which of these • How much data forms of analysis our research problem
questions can I will I collect? conducted? and questions?
hope to address • How will I validate • What do I need to • How do these guide us
well? this data or know, be able to do in deciding what kinds
• How can I establish that it is and have access to of data to collect and
analyze my good quality data? do in order to use how to analyze it?
problem and • How will I organize these forms of • Who provides good
questions to this data so it will analysis in an advice on how to collect,
specify a be in good shape expert way? validate and organize
succinct set of for being analyzed? • What will these data?
research aims forms of analysis • Who provides good
and objectives, let me ‘say’ about information about how
and a finite set of the data (and not to analyze data?
research say)?
questions?

Summary

This table or schema provides a concise summary of some of the key points we have tried
to make in this document about research and research design. In particular, it focuses our
attention on the relationship between research problems, research questions, and research
aims and objectives. These are not self-evident matters. In order to clarify what our
research is really going to be about we need to do a lot of analytic work in which we
analyze our research problem to discover what components or dimensions it might
contain. We need to analyze these components or dimensions in terms of the kinds of
questions we can ask that will allow us to address them in ways we (and others) find
satisfying. And we need to know how to analyze both our problem and our questions in
ways that allow us to identify what our research aims and objectives are. Without aims
and objectives we will not be able to know when our tasks have been met or how well we
have addressed these tasks.

Of course, unless we are clear about our purposes and know what kinds of questions are
involved we will lack a strong and clear base for deciding what kind of data to collect,
how much of it to collect, and so on. Therefore, we need to be able to analyze and specify
our questions and purposes in ways that guide our data collection. While this may seem
obvious, it is surprising how often people fail to recognize the importance of the
relationship between our purposes and decisions about data. This occurs, for example,
every time someone collects their data before they have a clear research question. In
terms of a logical research process it is simply absurd to collect data before one has a
question. This is like buying ingredients for cooking before knowing what it is that one
wants to cook. In addition, the data aspect draws attention to the fact that it is not rational
to invest time in analyzing data unless we are confident that the data is trustworthy and of
good quality. It doesn’t matter how competently we analyze our data if the data itself is
of poor quality. If the data is poor even excellent analysis won’t be able to give us
research outcomes we can have faith in. Once again, although this seems an obvious
point it is surprising how often data is analyzed without researchers having checked to
see if it really does pertain to their purposes and whether they are justified in believing
the data is of good quality. In addition, it is important to recognize that data accumulates
very quickly, and that analysis can become a very difficult task—if not impossible—
unless we start organizing our data in appropriate ways from an early stage in the
research process. Often the kinds of analysis we intend to conduct require us to arrange
our data in one kind of format rather than another.

When we are thinking about collecting our data we need also to be thinking about how it
will be analyzed. Some kinds of analysis require much more data than other kinds. Some
require a lot more time than others. If we collect more data than we have the necessary
time and other resources for analyzing, we are wasting time and effort—our own and
other people’s. Alternatively, if we find at the point of analysis that we have not collected
enough data or enough good quality data it may be too late to go back and supplement the
data we have. This could undermine the entire research activity.