Comprehensibility of illustration - an analytical model

Evelyn Goldsmith Department of Visual Communication, Polytechnic

Brighton

It is suggested that"themeans to analyse iliustration would be of great benefit to students, illustrators, publishers and others, and a model is presented identifying /2 elements which the author believes contribute to the comprehensibility of pictures, particularly those intended to be in support of text.

Dr Evelyn Goldsmith is a Research Fellow in the Department of Visual Dommunication, Brighton Polytechnic Author's address: Department of Visual Communication Brighton Polytechnic Grand Parade Brighton BN2 2JY Evelyn Goldsmith Information design
Journal Vol 1, 204-213

1980

I have read the paper by Clive Ashwin (1979) in the first issue of this journal with a great deal of interest, not least because of its basic premise that illustrations can be analysed and that, on both a practical and theoretical level, it may often be helpful to do so. For some years I have myself been developing a model for the analysis of illustration, and more than once have encountered the suggestion that such a proposition 'is neither possible nor desirable. My own feeling isthat an analytical framework should be seen as a valuable thinking tool for illustrators, students, publishers, teachers and anyone else who produces or uses illustration: the very opposite of the formulas and .superficial guidelines which critics of research seem to assume are the inevitable outcome. There are, of course, different ways of, and different reasons for, taking illustration apart. My study grew from an interest in illustration's supportive educational role, particularly where it is intended to help adult beginning readers identify words; and although considerations of preferences for various styles are certainly not irrelevant in this context, I decided to try and construct a model that would most usefully lend itself to the evaluation of the comprehensibility of illustration. The result used alongside, or perhaps integrated with, Ashwiri's proposals could provide the basis

for a comprehensive analytical structure encompassing considerations of the cognitive and affective aspects of illustration. Here briefly are the main points of the model presented in full in my thesis (m!l4$ij1ii!ii:;1~78);: Analytical proposals 1: Semiotic levels ( suggest that comprehension of a picture involves three different levels of response rL'A response to graphic signals as a discr-iminable image or set of images. 2-) A response to images in terms of the meanings the artist intended to set down. A response to the artist's meanings in terms of all the elcvant previous experience and present judgment of the viewer. The similarity to a passage in the Bullock Report (1975) describing the skills involved in the complex process of reading is deliberate. The adaptation is to underline my argument that certain parallels can be drawn between the understanding of text and of pictures. My first proposal therefore is that the study of supportive illustration should be undertaken at three levels; and since Morris (1938) has provided a suitable basis for such a distinction in his theory of signs, I have adop-ted his terminology. The first level, known as syntactic, has already been well described by Ash win. It could be summarised as

J}

204

the level which does not presuppose any recognition or identification of image. The second and third levels, both referred to by Ashwin as semantic, arc designated by Mortis as semantic and pragmatic. These appear to correspond respectively with Ashwirr's 'semantic properties which are reasonably objective, verifiable and stable', and 'a more sophisticated semantic level, that of interpretation of the image in terms of psychological, emotional, aesthetic or moral significance' (p.53). This higher level implies interpretation by a viewer; ind at this s"tagecomprehension of a pictu~~n:!.~y depend on developmental, culturalii1d-srmilar factors. There is a sad little cartoon by Charles Addams which exemplifies the three levels clearly. It shows two unicorns stranded on a small mound surrounded by water, with an ark sailing away in the distance. The syntactic level requires an ability to distinguish the bounds of the separate images and to perceive pictorial depth -,The semantic level demands recognition of the images as unicorns, water, and an ark. But the point of the drawing is lost at the pragmatic level if the viewer is' unaware of the story of the Flood and the current scarcity of unicorns. It must be understood, of course, that in practice the levels are often interdependent, and that in any ease consideration of the higher levels automatically presupposes the existence of those below. Analytical proposals 2: Visual factors

picture which might be recognised as having a . separate identity, even if the identity is not known. The degree of separateness is obviously going to vary with the level of discussion appropriate to the intention of the picture or the interest of the viewer: for example, in a landscape the single image might be 'cow', while in a portrait : it might be 'eye' or even 'pupil'.

Location
As soon as there is more than one image in an array, a second factor has to-be considered: the spatial relationship between the images. It particularly encompasses the various devices available for depicting pictorial depth, such as overlap, relative distance up the picture plane, and the different forms of gradient identified by Gipson (1950'): gradients of tone, size, clarity, texture and so on.

Emphasis
This factor also refers to the relationship between' images, but is hierarchical rather than spatial. Many of the experiments relating to the learning effects of pictures set out to test simplicity versus complexity of visual material (egHolmes, 1963,; Dwyer, 1972:). But as Fuglesang (1973)' points out, it is often not the simplicity or complexity in itself which is important, but the amount of relevant detail. Just as a verbal exposition sometimes needs to consider a number of issues in order to present an argument in its true perspective, so a drawing can lose much of its communicative value if in an attempt at simplicity it is denied an appropriate context. But in neither situation need the increase in complexity imply a resultant move towards confusion. The successful visual communicator will make clear the successive levels of importance in even the most complex pictorial representation; and such control is surely preferable to what Amheim. (1954) calls poverty of abstinence.

Unity:
Attempts 'grammar' have often been made to establish a of Visual forms, most of which (eg l)on;dis.,'l97J') take docs, lines and so on as their base. However, I agree with M'oriis(1946) who argues that the component lines of a painting are no more signs than are the individual phonemes of a spoken language. I have called this first factor unity: a term I am not completely satisfied with, since it could cause confusion. However, I have been unable to think of any other suitable word which means 'oneness', and I hope that the context in which it is used will resolve potential ambiguity. Unity, then, will refer to any area in a

Text parallels
The three factors so far outlined all refer to the pictorial aspect of the communication. But earlier 205

Figure 1: Analytical model showing interaction between factors and levels

Levels Factors Unity Location Emphasis Text parallels

Syntactic
1

Semantic
2

Pragmatic 3 6 9
12

4 7
10

5 8
11

I mentioned that my concern was with supportive illustration; and the analysis would be incomplete without consideration of the relationship between the-picture-and its text. If. as in the case of books for beginning readers, the picture is thought of as a translation of the text, or vice versa, and responses to the picture do not reflect reasonably accurately the meaning expressed verbally, then for some reason the translation is inadequate. Text parallels are proposed as the factor which promotes verbalisation of the translation process. It is the stage at which the assessor of the picture/text relationship looks at the text and asks himself which of the textual elements can be portrayed directly, which can be suggested indirectly, and which cannot be portrayed at all. Analytical model A model can now be constructed, based on the interaction between the four factors identified and the three semiotic levels at which each factor can be considered. This gives a total of 12 elements which I propose contribute to the comprehensibility of supportive illustration (Figure 1). The numbers shown at each of the co-ordinate positions on the model refer to the sections below which give a very brief outline of the compass of each element. Where appropriate I have included a pair of examples in which the particular element has been isolated as a variable, so that the first of the pair represents its less comprehensible aspect. Syntactic unity Although some psychologists claim that there is no perception without recognition, I would suggest that in some circumstances a mark on a surface could be generally accepted by all obververs as one image rather than a group of images: it would have what RuhinU91S) calls 'thing-character'. In practice, of course, images arc rarely simple. Printed photographs, for example. are reproduced by a halftone process, and in drawings and painting contours are often left open and bounds to images implied rather than expressed.

Figure 2A: Syntactic line print

unity:

Figure 28: Syntactic halftone print

unitY:

206

.... ...
;

-;

Some of these conventions can rely on the observer's expectations, and in this case it could be argued that without recognition the image would assume a different shape in the mind of the observer, more closely related to its retinal projection: that is, it would probably be interpreted literally as a wiggly line or a flat patch.
Figure 3A: Semantic unity : few distinguishing features Figure 38: Semantic unitv : strong distinguishing features

Nevertheless, some perceptual tendencies have been shown to exist which are claimed by the gestalt theorists to be a result of organisational factors present in the visual cortex. The theories are well-known, and it may be sufficient here to recall that two of the major phenomena identified are the tendency to group marks, by proximity, similarity and so on; and to complete figures. For details, reference may be made to almost any writer on perceptual psychology Cqt Vernon,.19.;S2; Zusne, i970}. The implications of this for incomplete or understated images must be apparent. In Figure 2A the tolerance of perceptual processes has been stretched quite considerably; yet of 38 people to whom I have shown this line print.ronly four could not idcntify it correctly. Semantic unity In syntactic unity it is only necessary to be able to distinguish one from among a number of images, or to discern that what ~ay appear to be discrete. marks are in fact intended to represent a single figure. Semantic unity, however, requires recognition of the image, and representation adequate for this purpose relies not on slavish imitation of an object, but on the clarity of distinguishing features which give relevant information. The number or strength of distinguishing features that any object possesses controls the extent to which its image can be distorted. In the case of many round objects such as certain fruits, for example, a single feature such as colour or scale may be crucial; whereas a picrure of a human being can undergo extensive modification (eg as in a cartoon by Gerald Scarfe) without becoming unrecognisable. In Figures 3A and 3B the fruits were photographed to the same scale and in exactly the same conditions, and the prints were made to duplicate the degree of focal distortion. The quality of focus can be seen to be irrelevant in the case of the banana, but important for the positive recognition of the apple. Wrong identifications have included orange,-peach, and even pea.
207

Figure 4A: Pragmatic unity: impoverished information

6
Figure 48: Pragmatic unity: the benefit of context

Pragmatic unity Relying on probabilities in our environment enables us to make rapid decisions. If every percept had to be checked we could scarcely function at all. But this implies familiarity with a situation, reliant on such factors as age, culture, training, interests and so on. F-~iliarity carloT course work for or against the viewer. Most of us at some time have been misled by faulty perception which has been a result of expectations that tum out to be unwarranted. On the whole, however, familiarity provides support in the form of understanding of contexts. An example is given in Figures 4A and 4B. In 4B impoverished information is supplemented by a context which leaves no room for doubt as to identity provided that the viewer is aware of the story of the Garden of Eden.
1

Syntactic location There are many cues [Q depth in a picture, some of which were mentioned earlier, and all of which have been known to painters for centuries. To warrant inclusion at the syntactic level the device for depiction must be independent of the viewer's recognition of the image; and one of the most compelling is the gradient of texture usc.~_~~e variable in Figures SA and 5B. I have used baubles in this example to avoid the inevitable assumption that the objects are at the same horizontal level ; thus removing relative . upward location in the picture plane as a potential cue to depth in SA.

Figure 5A: Syntactic Figure 58: Syntactic

location: location:

gradient cue missing gradient cue present

.:
..

208

Figure 6A: Semantic location: scale cue missing Figure 68: Semantic location: scale cue present

Semantic location This does not refer to the way in which location affects the semantic level of reading a picture, but the way in which the semantic level affects the assessment of location. In a line drawing with few indications of depth, a useful cue can be variations in size of a known object. However, this can be awcak cue, and Hoch f;> erg (1.9'64) has shown that although in a laboratory situation differently-sized playing cards will suggest that some are more distant than others, a contradictory frame of reference locating them on one plane will easily negate the previous assumption. In Figures 6A and 68 all cues to distance are' removed with the exception of known size, and it is expected that in 6B the extra information will facilitate judgements of relati~e distance.

Pragmatic location In some cases, the process of inference characteristic of the pragmatic level can be counted on to correct the sort of perceptual anomalies that are most prevalent in the work of photographic novices, but not unknown in more professional circumstances. In viewing the real world, in order to see a reflection in a shop

window or an object in the background, we have to change the focus of our eyes quite substantially; and unless we make a conscious effort to do this such events will go unnoticed. However, presented in two-dimensional form, no change of focus is needed and the disturbing element becomes apparent. .The extcn t to which the un wanted juxtaposition interferes with the information value of the picture depends on two things: the experience of the viewer in reading pictures, and the acceptability of the portrayed situation. If a literal interpretation of a scene would be absurd, it will usually be discounted, but dislocations unnoticed by the artist or photographer can lead to misinterpretation. In an experiment by Smith and Watkins (1972) to test ways of drawing children's attention to dangerous objects in the home, a complex drawing of a kitchen shows a young child reaching up toward the handle of a steaming saucepan on the stove. However, one subject, noticing the child's hand level with the gas taps, assumed it was turning them on and listed this as the hazard. In Figures 7A and 7B the paintbrushes are in both cases in a jar on a shelf about 15 inches away from the man. In 7A, however, the juxtaposition leads to misreading, while in 7B it does not. 209

F igu re 7 A: Pragmatic location: juxtaposition which could be misread Figure 78: Pragmatic

location: unambiguous juxtaposition

210

Figure 8A: Syntactic emphasis: regular visual array Figure 8B: Syntactic emphasis: irregularity causing attraction

Syntact ic emphasis I am using the term emphasis to describe thc aspect of pictorial organisation that helps the viewer extract relevant information. It is largely concerned with attracting and directing attention; and one way of classifying attention identified by James (1890) is by making a division between objects of sense, which attract sensorial attention, and ideal or representational objects demanding intellectual attention. These two categories correspond usefully with the syntactic and semantic/pragmatic levels adopted here, and it becomes a simple matter to identify the types of visual stimulus appropriate to this particular element. Attention can be attracted by contrast of all kinds: tone, colour, size, direction, etc; position within the picture frame, isolation and so on i and directed by linear devices such as edges of roads, swords, arms, or echoes of colours or shapes that tend to group in the manner described by the gestaltists. In Figure 8B attention is attracted by contrast of organisation. The slightly displaced spot has no semantic value, and in 8A receives no attention at all. People asked to mark a spot which attracts them in Figure 8A concentrate on the centre four, and in particular the spot top left of centre: a phenomenon confirmed byJ3;Jtindt(l~'4:S:) using eye movement cameras.

Semantic emphasis Eye movements have been studied extensively by a number of researchers, notably Buswell (1935) who confirms that of all representational images that having most universal attraction is the human face: in particular, the eyes. The direction of attention at the semantic level is a simple matter in any picture containing human beings: once attracted to the eyes of the main figure the natural tendency is to follow the direction of gaze. More subtly, direction may be achieved at the syntactic level: an example of the two levels working together in the organisation of the picture, Pragmatic emphasis Since the pragmatic level depends upon the experience and interests of the individual viewer:... it might be thought difficult to evaluate this clemdnt of an illustration. However, certain considerations arc common to large groups of people, and it is worth remembering that some perceptual phenomena mentioned in connection with the syntactic level may be influenced by developmental or cultural factors. The tendency of viewers to look top left of centre in Figure 8A is likely to be.partly due to reading habits. Also under this heading can come emphasis which is arranged arbitrarily: colourcoding, for example, as in the London Underground sign system. Syntactic text parallels The use of the word syntactic in connection with text parallels should not suggest an analysis of the text itself. The syntactic level here refers to the relation between the pictorial and the verbal signs. As with syntactic location, the concern is with spatial relationships: the position on the page, or series of pages, -of the picture relative to its text. Smith and Watkins (1972) carried out a wellplanned series of experiments designed to test this element. Some interesting points emerged, but the findings are complicated to summarise and one would do well to read the original report,

_eee eee_

eeee

eeee

eeee eeee eeee eeee

211

Brandt (1945) also did some investigations with an eye movement camera which demonstrated that if a picture and text are placed horizontally adjacent the text receives more attention if the picture is on the left. Brandt proposes that this is due to the picture's having greater attraction value than'the text. This means that attention is initially drawn to the left, and reading habits then make it more natural to move to the right. Semantic text parallels In semantic unity consideration is given to the question of distinguishing features of an object, and problems of identification. However, teachers using flash cards containing a single image and word are familiar with the danger of not wrong but alternative identification. In an experiment to exemplify this element, I drew two pages of objects, one containing items that could be expected to produce a reliable single response, and the other containing objects having several alternative names. A picture of an apple, for example, was consistently named correctly, while a jumper was labelled jumper, jersey, pullover, woolly and sweater. Pragmatic text parallels In a book in which the intention is that the text" should be closely supported by the illustration, it could be said that the text and illustration are intended to be symbolic of the same reference. The fact that the mode of sy~bolisat;on is different and the reference often not concrete makes for difficulties, however. I have made an analysis of the first group of Schoneil's(932) 'essential spelling' list comprising about 400 words which it is proposed form the basic vocabulary of. children. Even in this most concrete of levels I could find only abou t 120 words that could be directly illustrated (of which only 80 could be called reliable). Of the remaining 280, almost half would defeat the most ingenious artist: for example 'hope', 'only', 'yet', 'soon'; while the rest present problems that arc not insuperable, given reasonable context.
212

It is the function of text parallels to consider this context; and at the pragmatic level the common experience of a known readership can be assumed. It is not possible to draw 'a cold' for example, bu t we can observe not only its symptoms but the habits of people suffering from such an ailment. This element implies in the artist the observational powers to recognise the significance of quite small informational cues; and in the viewer the experience to notice and take advantage of such cues when they are offered. Much of the depiction may rely not on denotation, the literal components of the drawing, but on connotation: the invisible elemen t brough t to the drawing by the viewer. A pin man with a hat and stick, for example, will often be described by younger viewers as 'an old man'. It is an awareness of such connotations that makes reasonable correlation between text and picture possible at all; and if a picture fails it may be because pragmatic text parallels have not been sufficiently clearly thought out. Once a decision is made how to suggest a certain state - 'old', 'tired', 'a cold' and so on - the responsiblity lies with the pictorial factors. If it is agreed that a pin man will look old given a hat and a Stick, and nobody recognises the hat as a hat, then it is the fault of one of the levels of unity; but if everyone gives a correct literal description of the drawing without including the word 'old' then the fault is with text parallels. Either it must be agreed that 'old' cannot be suggested, or another way must be found to do it. Conclusion I have suggested that it is important to be able to analyse illustration, and have proposed a model that would facilitate this process. There is only room here for brief examples of each clement, and it is in any case doubtful whether any list could ever be exhaustive. It is likely that most established illustrators already have the experience to know whether or not their drawings will work in the context in

which they are set; but I feel that a method of analysis could be an invaluable aid to students who may not be aware that emphasis, for example, . functions at different levels, and who may be unwittingly dissipating attention by allowing syntactic and semantic forms to cancel each other out. Analytical studies of renowned artists could yield understanding of principles that could be adapted to other styles and applications, and perhaps make their own work less reliant on direct derivation. Control of complexity, and informed

experimentation, could result in more exciting artwork in the field of information. Publishers, teachers, resource? centres and so on could find analysis and the terminology pertaining to it helpful in evaluating and discussing pictures that they arc proposing to use. The model outlined here was designed to question the comprehensibility of illustration, but it could easily be used for other purposes, and possibly integrated with Clive Ashwin's analysis of style to provide a basis for a more comprehensive approach .

. Ashwin Clive The ingredients of style in contemporary illustration: a case study Information Design Journal 1979, vol 1,51-67 Brandt Herman The Psychology of Seeing NY: Philosophical Library 1945 Bullock Repon See: Education and Science, Department of Buswell GT How People Look at Pictures University of Chicago Press, 1935 Dondis Donis A A Primer of Visual Literacy Cambridge Mass: MIT Press 1973 Dwyer Francis M Jr A Guide for Improving Visual ised Instruction Learning Services. State College, Pennsylvania 1972 Education and Science, Depanment of A Language for Life: Report of the Committee of Inquiry appointed by the Secretary of State tor Education and Science (Bullock Report) London: HMSO 1975

Fuglesang A Applied Communication in Developing Countries. Ideas and Observations Uppsala Sweden: The Dag Hammarskjold Foundation 1973 Gibson JJ The Perception of the Visual World Boston: Houghton 1950 Goldsmith Ewlyn An Analysis of the Elements Affecting Comprehensibility of Illustrations Intended as Supportive to Text Unpublished CNAA PhD thesis. Brighton Polytechnic 1978 Hochberg Julian F Perception NJ: Prent ice-Hall 1964 Holmes Alan C A Study of Understanding Visual Symbols in Kenya OVAC London 1963

Morris Charles Foundations of the Theory of Signs vol 1 no 2 Foundations of the Unity of Science International Encyclopaedia of Unified Science Univ. of Chicago Press 1938 Morris Charles Signs, Language and Behavior NY: Braziller 1946 Rubin E Svnoplevede Figurer Copenhagen 1915. Translated in Readings in Perception ed Beardslee DC and Wertheimer M Princeton 1958 Schonell Fred J The Essential Spelling List London: Macmillan Education 1932

of

James William The Principles of Psychology NY: Dover 1950 (First edition 11390)

Smith J and Watkins H An Investigation into Some Aspects of the Illustration of Primary School Books Typography Unit, Univ of Reading 1972 Vernon MD A Further Study Perception CUP 1952

of Visual

Zusne Leonard Visual Perception of Form NY: Academic Press 1970 213