Professional Documents
Culture Documents
Abstract
In the discipline of food science and technology a range of methods has been developed
in order to translate qualitative perceptions into data with numerical value. These sensory
evaluation methods are widely used both by the food industry and academics. The
translation of qualitative information into quantitative data allows statistical analysis and
more accurate monitoring of changes. There are various models that have been developed
which can be selected to fit the goals of research. This paper gives some suggestions,
based on our own experience, of how different sensory evaluation models can be used in
development-based research.
A model for consumer preference testing was applied in rural Tanzania, to assess
acceptability of newly developed products. The paper explains how the tests were
conducted in a participatory way, and outlines important adaptations such as training of
interviewers and a simplified design. Further it discusses how the outcomes related to the
findings obtained by conventional qualitative assessments. A model using a semi-expert
trained panel was applied to monitor changes in estimated market value of products. It
out lines the importance of priming of semi-trained panellists, and the findings are
compared to qualitative appraisals at the markets. Line scales were used as a tool during
stakeholder workshops. This technique assisted in the prioritisation of different
development alternatives. The use of line-scales is a powerful tool because it is visual for
the stakeholders and thus understandable for a large range of people. It also gives
quantitative data allowing various methods of statistical analysis.
Introduction
Sensory evaluation methods that are used in food science and technology are designed to
translate qualitative information into numerical data by applying a statistically valid
experimental design and controlled testing conditions. These sensory evaluation methods
are widely used both by the food industry and academics. Jellinek (1985) gave the
following definition:
Sensory analysis of food relies upon evaluation by our senses (odour, taste,
colour, tactile, temperature, pain etc). Only by applying exact scientific testing
methods can reproducible results be obtained and analysed statistically.
In development sciences there is an increasing awareness that the power and acceptability
of participatory and qualitative research methods can be enhanced through more
thoughtful and systematic combination of qualitative and quantitative methods.
Participatory approaches that focus on qualitative data collection do have many
advantages, such as more reflection, judgement, and comparisons, which are considered
adequate enough for practical purposes (Chambers, 1994). Researchers have gained
excellent results with PRA methods in insights, documentation and descriptions of a
situation. The participatory approaches may use ranks or other techniques to get
quantifiable results and reasonable results have been obtained using ranking. Some
researchers highlighted the limitations of ranking. A major issue here is that ranks do not
include weighted values and thus need to be interpreted with care (Maxwell and Bart,
1995). They suggested that scoring systems would give more useful data.
Numerical data have the advantage that they can be easily analysed statistically, and
demonstrate whether there are significant effects or differences between treatments.
Statistical packages for data analysis have recently become more accessible for a greater
number of people, including development researchers in developing countries. Other
disciplines also make use of quantitative approaches to qualitative data, there may be
scope for development researchers to adopt some of these methods.
There are various ways in which qualitative and quantitative methods may be combined
to improve the trustworthiness of survey and experimental findings. Kleih et al., (2001)
have reported various strategies and ways in which this may be done. While this report
does address the sampling approach, and the increase the trustworthiness of the research,
it does not specifically address how qualitative information, which may relate to the
senses of peoples may be quantified.
This paper considers how the framework used in sensory evaluation may find application
in development research. First an outline is given of the principles of sensory evaluation
and several popular sensory evaluation methods are described. Then three case studies
are presented using sensory evaluation in development research. Finally some ideas are
discussed on how sensory evaluation methods could be applied in development research
for other aspects than food evaluation.
Table 1
Test Method
Experts
Trained
Untrained
assessors
assessors
Paired comparison
20
30
Triangle
15
25
Duo-trio
Consumers
100+
20
Two-out-of-five
10
20
Classification
Rating
30
20
50 (2 samples)
100 (3 or more
samples)
Ranking
Profiling (QDA)
10
100+
The design and use of sensory facilities including a controlled environment is needed,
with a minimum of distractions. This is essential in order to achieve consistent results
with least possible bias. The testing area concerns booths and a preparation area. The
booths should be identical, have easy access, comfortable working temperatures,
odour-free facilities, a neutral colour scheme, and good even lighting. It is important
that the booths and preparation areas are kept clean and hygienic
Valid conclusions can only be drawn for a product as a whole if the samples tested
are representative and therefore general principles for sampling should be applied.
Most test methods compare two or more samples at once. To avoid bias, the order of
sample presentation should be random and a consistent way of presentation.
Containers should be chosen so as not to affect the test and lighting should be
controlled when appearance is assessed. Codes (instead of alphabetic symbols) for
sample identification. Samples should look identical, and palate cleansers can be used
such as water, crackers.
A panel leader experienced in sensory methods and trained laboratory personnel need
to be appointed. Setting up a panel is an important part of the sensory evaluation. One
needs to take into consideration the cost, the amount and the type of work to be
carried out, i.e. is it short or long term.
The number of experts, trained assessors and assessors required depends on the test
method used. Training will improve precision and will enable the panel leader to
select those with the best ability for the particular test method and product.
Acceptability testing
Food acceptability is often referred to as liking, preference, enjoyment, selection and
consumption of a food or drink or food quality. Food acceptability represents different
forms of behaviour to food products. It is therefore vital that the objective of any
consumer study is clearly defined in advance and the experimental design and
questionnaires are carefully designed. For example, "How much do you like" is not the
same as "how much do you eat" because consumption is influenced by price, availability,
whether the consumer is on a diet etc.
The selection and choice of food by an individual are determined by factors resulting
from both the food product and the individual. The figure below illustrates how these
factors relate to each other in influencing consumer acceptability.
- Food habits
- Eating location
- Financial status
- Quantity eaten
- Age, sex
- Frequency of eating
- Nutritional status
- Expression of liking
- Emotional mood
FOOD
HUMAN
BEHAVIOUR
- Sensory properties
- Constituents, ie, protein, fat,
carbohydrate, toxins, microorganisms
- Packaging
- Environment
- Environment
- Price
Figure 1
- Preparation requirements
Research Association)
Methods available include: Self completion diaries (retrospective, present), qualitative
group discussions for product concepts etc., central location tests (located in shopping
centres, can have mobile laboratory) and home tests (often used at final stages)
Blind presentation is used to focus only on product characteristics, especially sensory
ones. Consumers are not shown the packaging and no other information is given.
Identified presentation is used for the assessment of product performance as the product
identity can dominate the sensory properties of the product.
The number of samples presented will depend on the product and the information
required. In central location tests, not more than six samples should be presented to
consumers to avoid fatigue and demotivation.
Questions must be clear and easily understood by the consumer. Open-ended questions
should be avoided but the facility for recording spontaneous comments should be
retained. Long questionnaires carry the risk of fatigue and demotivation. Typical
interviews last 20 min, shorter ones are used when only limited information is required.
Consumers are usually selected on their age, sex, socio-economic group etc. Specific
geographic locations may be chosen. The number of assessors selected is determined by
the need to have statistical confidence in the results. Near-market consumer testing
requires several hundred assessors, but exploratory studies can be carried out with as little
as 50.
Statistical tests
Paired comparison test
Most sensitive test and commonly used is the paired comparison test. This is a test in
which samples are presented in pairs for comparison and detection of differences on the
basis of some defined criteria. The test is to determine whether there is a difference, and
if so, the direction of the difference or to determine if there is a preference. The assessor
is presented with one or more pairs of coded samples presented in a controlled or random
order. The two samples in each pair may be the same or different. The assessor is asked
specific relevant question(s) referring to the difference, the direction of difference or
preference. Within this there are 2 strategies: either a forced choice in which the
assessors must indicate which sample they find more intense or prefer even it they cannot
find a difference, or to allow for no-preference. Additionally, the assessor can be asked
how confident they are in their choice.
The significance of a difference can be tested using standard Tables with the forced
choice using one-sided tests, and a two-sided test is to find out if there is a difference in
intensity or preference).
If no-difference or no-preference replies have been permitted, the results can either be
ignored, that is subtract them from the total number of replies from the panel, or half of
the no-difference or no-preference replies can be allocated to each of the two categories
of replies.
Tests using scales and categories
Quantification of sensory data on the basis of the perceived intensity of attributes requires
the use of some form of scaling procedure. Most detailed analysis and interpretation of
sensory characteristics require the intensities to be given a numerical value on some form
of scale. The following types can be distinguished:
Ranking.
Classification.
Rating.
Scoring.
Grading.
result assigns a preliminary ranking. This ranking can be checked and adjusted by reexamination of the samples. The assessor records the finding on a score sheet.
In order to analyse and interpret the results, samples are decoded and the rank orders to
each sample given by each assessor tabulated. Where there are tied rankings, record the
mean rank. Calculate the rank sum for each sample by summing the ranks for each
assessor. By comparing the rank sums for the samples it is possible to obtain an
evaluation of the differences between the samples.
Scoring is a form of rating using a numerical scale. The numbers used in scoring from
an interval or ratio scale (i.e., the different scores have a defined and mathematical
relationship to each other). Scoring is used for evaluating the intensity of one or more
attributes. The recommended number of assessors is 1 or more experts, 5 or more trained
assessors, 20 or more assessors.
The assessor assigns to each sample a value according to a predetermined scale e.g.,
descriptive, line etc. Figure 2 presents a sample of a currently often used line scale,
measuring 6 inches, with anchors at 0.5 inch at the ends (Stone and Sidel, 1998). The
results obtained for one sample may be summarised as a median or a mean with some
method of variation such as the standard deviation. If only two samples are involved and
the distribution of the results is normally distributed the t-test may be used to compare
means and the F-test to compare variances. If more than two samples are involved, the
analysis of variance (ANOVA) procedure is recommended.
not sweet
Figure 2
very sweet
Example of a line scale, 6 inches in length, anchored at 0.5 inch at the side
forming the ideal length for line scales (Stone and Sidel, 1998).
11
The number of assessors will vary according to the grading method applied. The results
can be summarised as for classification and may be used to make a decision about a food
product, e.g., its price, accept/reject etc.
Sensory Profiling
Sensory profiling methods use scales and are the most sophisticated techniques available
to the sensory analyst. Profiling aims to produce a comprehensive description of the
appearance, flavour and textural characteristics of a product and to quantify the intensity
of each present. Hence, sensory profiling techniques are often referred to as descriptive
analysis techniques.
Several descriptive methods have been described, the main ones being, flavour profile
method (FPM), texture profile method (TPM), quantitative descriptive analysis (QDA),
free choice profiling (FCP). The same basic principles apply to each of these methods
being (1) verbalisation of perceptions and (2) definition of vocabulary used to describe
the perceptions.
The subjects are trained to measure this intensity with a certain level of accuracy and
reliability; their assessment should be independent of like or dislike tendencies. All the
descriptive techniques (except for free choice profiling), assume individual assessors
have the same meaning for a descriptive term and associate with it the same perception.
The Quantitative Descriptive Analysis (QDA) technique was developed to overcome
some of the restrictions of TPM and FPM methods. These are the reliance on the
consensus judgements of a relatively small number of highly trained assessors and the
use of product characteristics and scales that have been previously defined by the
experimenter and not by the panel. This makes these techniques expensive and timeconsuming to carry out. Additionally, FPM and TPM are restricted to a specific group of
sensory characteristics and therefore ignore interactive effects of sensory characteristics.
12
13
Case studies
Three case studies are reported here to illustrate how sensory methodology can be used to
assess parameters in development.
Case Study 1: Consumer Preference Testing: using rating
As part of a DFID project on commercialisation of cassava to increase incomes, the
consumer acceptability of newly developed products of cassava using cassava chipping
equipment was compared with the traditional cassava products: fermented udaga
(fermented cassava product) and makopa (dried cassava product). These products are
prepared in the form of a stiff porridge, ugali. During group discussions it was found out
that most consumers eat ugali as a mixture of maize and cassava. The tests were
conducted in three villages in three districts, and a total number of 150 consumers. These
villages had been selected on the basis that a surplus of cassava was produced, which
would be a requirement of commercialisation.
Seven different mixtures were tested, which included: udaga, makopa, chips of sweet and
bitter cassava varieties which were processed using a manual chipper. The product to test,
ugali, was prepared by local women. During initial meetings with the women it was
discussed how ugali is normally cooked and a standard cooking method was agreed.. The
important sensory characteristics are appearance, colour, smell, texture, flavour/taste and
stickiness.
A team of four villagers were trained to help conducting the tests. They would ask the
questions and mark the forms. This was necessary as some villagers would only speak
their tribal language and had had either none or minimal education.
The consumers were presented with three samples at a time and these were in random
order. These were randomly selected from the seven possible mixtures, using randomised
tables, and labelled with letters. Ideally three figure coded samples are used, but it was
thought that numbers would be more prone to confusion.. The samples were presented in
orange bowls, standard for Tanzanian consumers. First the consumers were asked to taste
the samples plain, then a sauce was added.
14
Consumers were asked to give a score from 1 to 5 representing how much they liked a
particular attribute of the product. In this 1 = very poor, 2 = poor, 3 = normal, 4 = good, 5
= very good. The taste tests took at a central location at the villages on market day.
Adaptations :
-
Assessments were written out by trained villagers who conducted the trials
The samples were tested without a sauce first and then with a sauce. This was done
because Tanzanians would never eat their ugali without a sauce. For sensory
evaluation purposes, however, the sauce may distract the taster from testing the
quality of the product. Therefore adding sauce at a later stage was a useful
compromise.
15
appearance
colour
flavour/taste
smell
stickiness
texture
5
4
3
2
1
0
makopa +
maize
Figure 3
chips bitter
fermented +
maize
maize 100%
udaga + maize
fermented
bitter chips
(100%)
Figure 3 shows mean preferences of the mixtures. This scoring gives a better indication
then ranking, because it is more quantitative. The overall preference can be deduced from
the average score, which is highest for makop/maize. We can see that Udaga/ Maize
scored lowest for appearance and colour (3.5 and 3.4), while the appearance for
makopa/maize scored highest (4.3). For colour the bitter fermented maize scored highest
(4.3). The flavour/taste of maize 100% scored highest (4.1) while the lowest score for
flavour/taste was for the bitter chips fermented/maize (3.4). For smell makopa/maize
scored highest (4.0). The best level of stickiness was obtained from makopa/maize (4.1)
while the poorest stickiness was obtained for 100% bitter fermented cassava (2.8).
Makopa/maize also had the best texture (4.3) followed by chips fermented bitter/maize
(4.2), while the poorest texture was found for maize 100%.
16
17
Instead of tasting food samples, the semi trained panellists were here asked to only
give scored based on appearance in the same manner that consumers assess the
product in the market place. Because the assessment was this is simple, twenty heaps
could be assessed at the time.
The method of display was chosen to resemble the display at the market, and it was
emphasised to the panel leader that it would be important to do this exactly in the
same way every time. Therefore the assessment took always place at the same place,
using the same materials, and labels.
Results:
During storage the estimated market value declined for all sweet potato roots (Figure 4).
The decrease in value was greater for the damaged roots than for those that were
undamaged. After eight weeks storage, the value of the damaged and undamaged roots
18
declined to 20.9 and 31.2 TSh/kg respectively. The results here demonstrated that
selecting for undamaged roots is more important than cultivar.
80
Polista dam aged
70
60
50
40
30
20
10
0
0
10
Figure 4
Table 2: Mean consumer acceptability values for rice type with location
20
Location Imported
Imported
(Upper East
(Kumasi)
(Tamale)
(US Tilda)
(US)
region)
Accra
7.4 (0.015)
7.5 (0.155)
6.0 (0.162)
6.4 (0.177)
4.7 (0.204)
3.9 (0.225)
Kumasi
7.7 (0.152)
7.4 (0.145)
6.1 (0.158)
6.2 (0.166)
5.5 (0.184)
4.1 (0.226)
Tamale
7.3 (0.174)
7.6 (0.195)
6.9 (0.187)
6.2 (0.156)
4.8 (0.1.95)
4.5 (0.236)
Figure 5. Spider chart for most and least preferred rice by a semi-trained panel in Ghana.
Uniform appearance
Sticky texture
100.0
Whitish appearance
80.0
Grainy texture
60.0
Yellowish colour
40.0
Hard texture
Brown colour
20.0
US parboiled
Ghana parboiled
0.0
Sour taste
Black heads
Blandness
Strength of odour
Creamy taste
Rice odour
Sweet taste
Figure 6. Correlation between consumer acceptability and sweet taste scored by a semitrained panel.
21
8.0
Consumer acceptability
7.0
6.0
5.0
4.0
30.0
50.0
70.0
Figure 7 Trends for consumer acceptability for rice with increasing numbers of
consumers interviewed (1 to 100)
22
9
8
7
Raw imported
PB Imported
PB Upper East
PB Accra
PB Kumasi
PB Tamale
6
5
4
3
2
1
0
10
20
30
40
50
60
70
80
90
100
Number of consumers
23
how does development increase the perceptions of people? Happiness? Trust? Utility?
All of the items above are certainly as or more abstract than using our senses to test food.
But just like the parboiled rice in Case Study 3, each of the issues may consist of
characteristics that bring quantitative and qualitative judgements together. Sensory
evaluation techniques that can help to understand the complexity of sensual experiences
associated with food, can also be used to describe, analyse and understand development
issues. The following is an example to illustrate how the concept could work.
24
The profiling concept may be a useful tool to map the aspects of the constraints and
issues that arise. Similar to the piece of food that contains different dimensions of its
appearance, aroma, taste and texture, adoption of a technology may include different
dimensions of constraints, advantages or changes in their situation. We need to determine
and quantify these factors. The approach is similar to that described for quantitative
descriptive analysis.
a. Screening prospective panellists: These would people who have adopted or evaluated
the equipment.
b. Selection of panellists. People can be selected by criteria such as type of equipment
user, gender, age, stakeholder in the marketing chain etc.
c. Development of a descriptive language by panel discussion: The group will form a
consensus regarding the advantages, constraints and influence on livelihoods in
development. This may include, for example, cost, water use, drying space, drudgery,
neighbours opinions, pain in the back, marketing of the product, labour use.
d. replicated judgements on development issues: Unlike sensory testing where the food
product is in front of the panellist, scoring using this approach may be by recall. The
panellist has to refer to experiences that occurred several weeks or months ago. The
panellist may also score on concepts relating to how much an event or experience has
changed. For example, what effect has the technology had on family income, how has
the role of the women changed?
25
The advantage is that this will facilitate the prioritising of factors that should be
addressed and relationships between them.
The method is rapid and easy to understand by people with minimal education.
Scores based on recall and on changes in a factor will be less subject to bias from
sampling and environmental conditions.
The application of profiling techniques in development could mean that some of the
principles and rules may not apply. For example, the data may not be normally
distributed and relationships may be non-linear. However, simple graphical approaches
such as spider diagrams and scatter plots may assist in helping the researcher interpret the
results. Principal component analysis and cluster analysis may help to summarise
relationships in the multivariate data. Care, however, is required on the choice of line
scale. Our experience of using line scales at a recent project workshop in Tanzania
indicated that if the anchors, at each end of the scale, are too extreme, the results are
skewed. In this case, a happy and sad smiley were placed at opposite ends of the scale.
Because the majority of participants were happy with the technology, they scored at the
extreme end of the happy smiley making prioritisation difficult. Replacing the sad smiley
with a neutral face may have been more appropriate.
26
Conclusions
Sensory evaluation techniques offer a range of possibilities to translate qualitative
information into numerical data. Its application is useful where the question can be
27
translated into a scale enabling a numerical value to be assigned to the response and a
hypothesis to be tested. The use of semi-trained and trained panels can reduce costs
significantly and give reliable data and has been shown correlate with consumer data.
The application of sensory methods would not conflict with PRA approaches, because the
methods would take place at the different stages of development process.
.
Acknowledgement
Collaborators are greatly acknowledged for their inputs to the case studies from LZARDI
Ukiriguru, Biharmulo District Office, Tanzania and the Food Research Institute, Accra,
Ghana. This publication is an output from a research project funded by United Kingdom
Department for International Development (DFID) for the benefit of developing
countries. The views expressed are not necessarily those of DFID [R7543, R7520,
R7498, R7497: Crop Post-Harvest Research Programme].
28
References
Chambers, R. 1994. Participatory Rural Appraisal (PRA): Analysis of Experience. World
Development, 22 (9), 1253-1268.
Digby P., Galwey N. and Lane P., 1989. Genstat 5: A Second Course. Clarendon Press
Oxford, UK. ISBN 0-19-852218-5. pp 233.
Jellinek G. (1985) Sensory Evaluation of Food - Theory and Practice, Ellis Horwood,
Chichester, UK.
Krzanowski W.J., 1988. Principles of multivariate analysis: A users perspective. Oxford
University Press, Oxford, UK. ISBN 0 19 8522304 4.
Maxwell, S. and C. Bart. 1995. Beyond Ranking: Exploring relative preferences in
P/PRA. PRA Notes 22, Feb. 28-34.
OMahony M., 1995. Sensory measurement in food science: fitting methods to goals.
Food Technology, April; 72-82.
Richards, P. 1995. Participatory rural appraisal: a quick-and-dirty critique. PLA Notes 24,
October, 13-16.
Stone H., Sidel J., Oliver S., Woolsey A., and Singleton R.C., 1974. Sensory evaluation
by qualitative descriptive analysis. Food Technology, 28 (Nov); 24-34.
Stone, H. and J. L. Sidel. 1998. Quantitative Descriptive Analysis: Developments,
Applications and the Future. Food Technology, 52 (8); 48-52.
29
Tomlins, K.I. 2000. Methods for the Sensory Evaluation of Food and Drink Products.
Natural Resources Institute, Chatham Maritime, Kent UK.
Tomlins, K.I. J.T. Manful, P. Larwer and L. Hammond. (submitted). Urban consumer
acceptability and sensory evaluation of locally produced and imported parboiled and raw
rice in West Africa, Food Quality and Preference.
Van Oirschot, Q.E.A, J.L White, T. Ngendello, S. Kolijn and A Westby. 2002. Cassava
processing technologies for farmers in Tanzania. PhAction. 5 (March) 20-22.
Van Oirschot Q, T Ngendello, R Amour, E Rwiza, D Rees, K Tomlins, D Jeffries, D
Burnett and A Westby. 2000 Preliminary observations on the potential for long-term
storage of fresh sweetpotatoes under tropical conditions. In Nakatami and Komaki, (eds)
Potential of Root Crops for Food and Industrial Resources. Twelfth Symposium of the
International Society for Tropical Root Crops, Sep 10-16, Tsukuba Japan 2000, p 341344.
30