This article was downloaded by: [b-on: Biblioteca do conhecimento online UA] On: 01 May 2013, At: 12:37

Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of New Music Research

Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/nnmr20

The effect of implied harmony, contour and musical expertise on judgments of similarity of familiar melodies
Emery Schubert & Catherine Stevens
a b a b

School of Music and Music Education, University of New South Wales, Australia

School of Psychology & MARCS Auditory Laboratories, University of Western Sydney, Australia Published online: 16 Feb 2007.

To cite this article: Emery Schubert & Catherine Stevens (2006): The effect of implied harmony, contour and musical expertise on judgments of similarity of familiar melodies, Journal of New Music Research, 35:2, 161-174 To link to this article: http://dx.doi.org/10.1080/09298210600835000

PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

Journal of New Music Research 2006, Vol. 35, No. 2, pp. 161 174

The Eect of Implied Harmony, Contour and Musical Expertise on Judgments of Similarity of Familiar Melodies
Emery Schubert1 and Catherine Stevens2 School of Music and Music Education, University of New South Wales, Australia; 2School of Psychology & MARCS Auditory Laboratories, University of Western Sydney, Australia
1

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Abstract
According to Western music theory, familiar melodies containing alterations which shift them a long distance (harmonically) from the original should be considered perceptually dissimilar relative to the original. It is possible to obtain such a large shift with a small change in melodic pitch. However, this creates a paradox with pitch (contour) shifting, which is known to reduce similarity minimally if the distance from the original is small. We investigated the hypotheses that (1) manipulations to contour and implied harmony of an original melody reduce similarity scores and (2) novices are less sensitive to implied harmony changes than experts, but as sensitive as experts to contour manipulations. Twenty-eight novices and 44 expert musicians rated similarity of familiar nursery rhyme tunes compared with close and distant harmonic transformations plus close and distant pitch shifts. Results indicated that both groups use contour (pitch distance) to determine similarity, but that musically experienced listeners also use implied harmony to make further distinctions. It is argued that as listeners become more experienced, they rely on more sophisticated strategies for encoding and organizing melodies in memory, with deeper structural aspects of music being used when other strategies such as contour are uninformative or non-distinctive. These new ndings, that contour and harmony aect judged similarity for simple, familiar melodies, have implications for theories of memory for music, and for the design of automated music retrieval and data mining systems.

1. Introduction
The study of perceived melodic similarity provides psychologists with information about how the brain processes, stores and retrieves musical information. Studies of perceived melodic similarity are also important for search and retrieval systems (Kim et al. 2000; Byrd & Crawford, 2002; Zhu & Kankanhalli, 2002), compression algorithms (e.g. Hofmann-Engl, 2001), and to further our understanding of processes involved in perceiving theme and variations and recognizing music in naturalistic listening settings (Deliege, 2001; Lamont & Dibben, 2001; McAdams & Matzkin, 2001). If a presentation of Melody A is rated as being similar to, or confused with (Dowling & Bartlett, 1981), a dierent Melody B then we can assume that the features that dier across the two melodies have been assimilated into the same melodic schemata activated in long-term memory. One melody contains some kind of perceptual redundancy with respect to the other melody. In other words, the study of melodic similarity asks how physically dierent a melody must be before it no longer bears perceptual equivalence to its original form (e.g. see McAdams & Matzkin, 2001). However, the study of the phenomenon is complicated by factors such as individual dierences, and the selection and denition of the independent variables. Regarding the former, can we assume that all listeners will judge two melodies as being dissimilar by roughly the same amount? Regarding the latter, should surface structures be investigated, such as pitch and rhythm variations, or deeper structure, such as the harmony implied by a melodic progression or

Correspondence: Emery Schubert, School of Music and Music Education, University of New South Wales, Sydney, 2052 NSW, Australia. E-mail: E.Schubert@unsw.edu.au DOI: 10.1080/09298210600835000 2006 Taylor & Francis

162

Emery Schubert and Catherine Stevens patterns. Levitin (1999) proposed that transformations in timbre, pitch, loudness, spatial location, starting pitch (or transposition Van Egmond & Povel, 1996), reverberation and tempo across a melodic transformation, in general, do not aect the identity of a melody. He argues that rhythmic changes can have an eect but are generally not strong. However, the pattern of ups and downs in pitch have a signicant eect on judged similarity. The related issues of melodic contour and implied harmony are the main focus of the present investigation of melodic similarity. 3.1 Melodic contour A number of denitions for melodic contour exist in the literature. Edworthy (1985) summarizes them as
. . . ranging from a global description of the predominant conguration of a musical theme . . . to a specic, note-bynote representation of a melodic sequence taking into account both interval size and direction of melodic movement between adjacent notes. A midpoint between these levels of analysis exists whereby contour is dened as a sequence of ups and downs in a melody or tone sequence, independent of precise interval size (pp. 375 376).

structural landmarks (for a discussion and denition of surface and deep structure, see Lamont & Dibben, 2001)? This paper examines some of these issues, and identies and investigates an apparent paradoxical eect of harmony and contour upon judgment of perceptual similarity.

2. Individual dierences and melody familiarity

Judgement of melodic similarity is neither objective nor universal. Many factors come in to play when we ask How similar is Melody A to Melody B?. One factor is the listeners familiarity with the melodic material (Welker 1982; Halpern, 1984a,b, 1989; Jones et al. 1987; McAdams & Matzkin, 2001). Dowling (1988) found that children younger than about 3 years retrieve basic melodic contour information from long-term memory, but that older children use more specic pitch-relationship information, facilitating more accurate retrieval of the melodies from long-term memory (see also Trehub et al. 1984; Trehub 1990, 2001; Dowling, 1994). However, even in adulthood, contour is still an important component in cognitive organization of melodic pitch streams (Dowling, 1994; Levitin, 1999; Trainor et al. 1999, 2002). Perceived dierences in melodies may also be a function of listeners aural acuity and experience. It is common for experimental studies to exercise some control over this participant factor. The most common grouping is musician versus non-musician. Yet studies generally report little or unsurprising dierences between these dierent groups of people. In general, the dierences might be traced back to familiarity with materials, a likely feature of the musician group. Greater familiarity with a tune or the rules of the tunes genre should allow greater amounts of information to be stored in memory and the use of more sophisticated problem solving strategies (Anderson & Schooler, 2000; Ericsson, 2003), facilitating accurate judgment. Perhaps experienced musicians have a better understanding of deep level musical structure than less experienced musicians. Much of the literature examines responses to surface level manipulations (such as pitch, contour and rhythm). It is relevant, therefore, to investigate and compare dierences between the processing of so-called deeper structures by musicians and non-musicians (Lamont & Dibben, 2001).

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

3. The eect of musical features on perceived similarity

What musical parameter manipulations lead to perceived dissimilarity? Cambouropoulos (2000) notes that musical context can aect judgments of the similarity of auditory

Edworthys midpoint denition is typical of those reported in subsequent literature. However describing contour as patterns of ups and downs can be misleading because a contour is a line representing a body or gure the information about the outline of the pattern is retained (see Schoenberg, 1967 for examples of this detailed kind of contour). In contrast, the commonly used denition of contour described by Edworthy is information reduced. Patterns of ups and downs capture melodic direction, and not the ner detail of the actual contour drawn out by the pattern of pitches over time (e.g. see Welker 1982). In fact, when comparing a melody with respect to another, the keep the contour same method produces no change in the comparison melody, making melodic direction have only two codable values for testing up and down. This is dierent to melodic displacement, where the distance of the notes (chroma) from the source melody is what becomes coded (for example, in units of semitones). An increased movement away from the original source note leads to an increase in displacement, regardless of direction. Therefore, we propose that what has been referred to in some literature as melodic contour is, more accurately, melodic direction, and that denitions which embrace multilevel positioning away from a reference point refer to melodic displacement (or distance)1.

In this paper we therefore use the term pitch displacement as a subset of contour.

Implied harmony, contour and expertise Kim et al. (2002) discuss various coding resolutions comparing simple directional coding (up, down, same) with ve and seven level representations. Seven level displacement allows a higher degree of resolution to be coded in a test melody with respect to the original, as shown in Figure 1. In practice only 6 levels are relevant in a comparison task because the same/no-change condition is the original source melody against which the comparison is made (as shown in the horizontal centre of the gure). In the gure there are three levels of displacement coded in the test melody (3 being a large displacement, 1 being small), and the sign indicates the direction of the change (7 for changes in opposite direction with respect to original note-to-note transitions, for changes in the same direction with respect to original note-to-note transitions). The Kim et al. study was an examination of the physical dierences and algorithmic eciency of the dierent methods and concluded that ve level contour produced an optimal compromise between the eciency of three level coding, and the robustness of seven level coding. In the present study the coding of contour is an important issue. It is possible to manipulate implied harmony (discussed below) such that its theoretical eect upon similarity is in opposition to the eect of contour. For example, consider the situation where a test melody is compared with a melody stored in memory. Contour changes using the displacement denition are known to degrade perceived similarity. This can be represented visually as shown in Figure 1. With no change in physical contour, a high similarity rating is provided. However, as the melodic pitch shape is changed, similarity scores reduce. When the melody is changed such that the size of melodic steps and leaps are increased with respect to the

163

original, the similarity judgements decline, as shown in the right half of Figure 1. When the direction of a step or leap is changed, and the amount of this change is increased, this too is associated with a decrease in perceived similarity, as shown in the left half of Figure 1. This approach allows a fairly simple means of quantifying change because the dierence (in semitones) of each note of a test melody (compared with the corresponding note of the original) can be calculated. These dierences are then converted into a single standard deviation score that quanties how dierent the contour of the test melody is from the original. 3.2 Implied harmony To a Western listener, a melody that conforms to the conventions of Western tonal music is more than a collection of pitches and note durations. For example, Bigand (1993) points out that the notation represents a complex network of tension and relaxation (p. 51) and Povel and Jansen argue that harmonic relations between consecutive tones play an important role in the perception of music (Jansen & Povel, 2004b, p. 47). Dowlings studies of contour (e.g. Dowling, 1978, 1991; Dowling & Bartlett, 1981; Bartlett & Dowling, 1988) suggest that scale structure is an important aspect of melodic organization, and this has been supported in studies by Povel and Jansen (Povel & Jansen, 2002; Jansen & Povel, 2004a, 2004b). Alterations to a melody that maintain the scale structure are considered more similar than those in which scale structure is violated. For example, if a piece is in C major, changing each g to a g# violates the expectation of diatonic (within-key) notes being used. This produces lower liking scores

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Fig. 1. Theoretical representation of the eect of contour (melodic displacement) variation upon similarity judgement. As the average of consecutive pitch distance increases with respect to the original melody, similarity decreases, whether the changes are in the same direction as the original transition (right side) or the opposite direction (left side).

164

Emery Schubert and Catherine Stevens Consider the nal two notes (b, c) of a melody in C major that imply dominant to tonic harmony. Alterations to such a melody within the implied harmonic constraint should be more similar than other changes. Any of the chord tones of the dominant G could be used to substitute for the b (that is, d or g), and any chord tones in C could be used to substitute for the c (i.e. e or g) to produce a similar melodic eect (compared with nonchord tone replacements, such as the within-key and outof-key replacements discussed above). If any of the melodic notes are chosen outside the implied harmony, the result should be a more dissimilar melody. This hypothesis is foreshadowed in results reported by Stoll and Parncutt (1987), Sloboda and Parker (1985), Holleran et al. (1995) and Trainor and Trehub (1994b). These results also suggest that musically experienced listeners use harmonic structure in melody cognition whereas inexperienced listeners may not. It appears that implied harmony plays some role in determining similarity in Western tonal music, and that it is a more important factor for musically skilled listeners. What we do not know is how implied harmony and contour interact in judgments of similarity. Implied harmony is a complex concept because it does not provide unique solutions for harmonic expectations (unlike melodic displacement) several dierent harmonies can sometimes be used to make the same melody sound good or pleasing. As Benward and White (1997) remark, while there is some leeway in the selection of chords, a certain standard of musical communication . . . prevents you from exercising complete freedom (p. 194).

(Cross et al. 1983), lower goodness ratings (Krumhansl & Keil, 1982; Povel & Jansen, 2002), sounds more funny (Dowling, 1988), and is more noticeable (Dowling, 1978). Bartlett & Dowling (1988) proposed several explanations for why this deviation from the diatonic scale produced such results. For the purpose of studying similarity, deviation from the diatonic may present only a general explanation of why foreign notes disrupt a melody. According to this paradigm, any non-diatonic (out-of-key or chromatic) transformation should aect similarity. However, what would happen if a diatonic note was substituted for the original such as an a in place of a g (assuming we are in the key of C major, to which both g and a belong), instead of a g#? The story then becomes more complex. Western music theory allows us to determine whether the a is harmonically related to the original note (e.g. see Krumhansl, 1990). More specically, non-diatonic (chromatic) pitch substitutions in a melody are less likely to fulll the requirements of the harmony implied by the melody in question. If this were the case perceptually, it would provide evidence that melodic information, particularly of veridical or stylistic familiarity, is stored not just as a collection of contours, pitches and rhythms, but also as a web of harmonic implications. That is, harmonic structure is abstracted by the listener. Such a notion is supported in experiments by Tan et al. (1981) and Bharuchas neural network model of music perception (see Bharucha and Krumhansl, 1983; Tillmann et al. 2000) and several more recent studies (Tillmann et al. 2000; Povel & Jansen, 2002; Jansen & Povel, 2004a; Tillmann & Bigand, 2004). Research on perceived melodic similarity has focused mainly on the parameters of contour or rhythm or both (e.g. Davidson, 1985; Edworthy, 1985; Carterette et al. bert & Peretz, 1997; Halpern 1986; Jones et al. 1987; He et al. 1998; Cambouropoulos & Widmer, 2000; Byrd & Crawford, 2002). In contrast, implied harmony has received relatively little explicit attention in studies of melodic similarity (important exceptions are Sloboda & Parker, 1985; Stoll & Parncutt, 1987; Trainor & Trehub, 1994b; Holleran et al. 1995). It is well understood by music theorists that Western melodies can be harmonized so as to support the melody (i.e. make it sound good). Further, there are a limited number of solutions to the harmonies that can form appropriate or pleasing accompaniments to a given melody (Jansen & Povel, 2004b). The limitation on the number of harmonic solutions further supports the view that an unaccompanied melody can be associated with some kind of implied harmony.

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

5. Pitch displacement harmonic distance paradox

Music theoretic assumptions and empirical literature about implied harmony can lead to predictions of similarity judgments that conict with predictions derived from the eects of melodic displacement. According to the melodic displacement perspective, the smallest shift of a semitone in a melody with respect to the original would produce the smallest change in contour distance. Hence there should be little change in similarity. However, this same shift can produce a large (chromatic or out-of-key) departure from the implied harmony of the original melody (see Krumhansl, 1990). From this perspective, the clash against the implied harmony of the original should reduce the similarity rating. A shift by another semitone might produce a pitch which is diatonically (within-key) dierent from the implied harmony of the original. In this case the transformed stimulus is harmonically closer than for the rst semitone change to the harmony implied by the original, even though the melodic distance from the

4. An implied harmony explanation of perceived melodic similarity

Implied harmony, therefore, should provide a further constraint upon judgments of melodic similarity.

Implied harmony, contour and expertise original has increased. The theoretical similarity response would then appear as shown in Figure 2. In the gure, the 3 refers to a distant harmony compared to the original, the 2 a diatonic shift (close key) and 1 refers to a dierent note from the same implied harmony as the original. The prole is in conict with the theoretical contour prole (Figure 1). The eect may be that the two inuences are additive and cancel each other. Therefore, it is appropriate to examine each variable level against all other variable levels (i.e. a full factorial design). The present study, investigates the pitch displacement harmonic distance paradox. It diers in several ways from previous research that has examined the eect of implied harmony upon melodies. First, both contour and harmonic distance are manipulated. Second, we use familiar melodies, in case this provides less experienced musicians with an increased capacity to use implied harmony information. This approach also makes the comparative task simpler because the listener has the original tune in memory a priori. Third, we manipulate groups of notes rather than single target notes. Fourth, we ask listeners to make comparisons with their longterm memory of the melody, rather than making a direct comparison or after a tune-learning phase. We believe that this might help to mitigate possible confounds due to short-term memory strategies, and therefore provide

165

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

access to tonal schemata that are used in more typical listening situations (where short-term memory strategies are not a central part of the cognitive organization of the music). Finally, we investigate three levels of harmony (most studies only examine two within chord and outof chord manipulations), although within-harmony changes are analysed separately. Since same-harmony changes will often require relatively large interval shifts (generally greater than or equal to a minor third), it is possible that a confounding variable of disrupted melodic expectation will also be manipulated (for example, an expectation of close proximity in melody is changed to a lack of proximity for more information see Narmour, 1990; Schellenberg, 1997; Huron, 2001; Margulis, 2003). That is, a shift in pitch that maintains the harmony of the original will usually require a minimum of a chordal skip (an interval of a 3rd or a 4th). So there may be cases when it is not possible to have a small change in contour with a no-change-inharmony condition, apart from the original melody. To minimize this constraint, the same harmony condition is not considered in the main factorial design, but will be reported separately. The hypotheses were that (1) deviations in implied harmony and in pitch displacement contour reduce similarity and (2) inexperienced musicians are relatively less sensitive to changes in implied harmony than

Fig. 2. Theoretical similarity perception due to harmony (assuming implied harmony dominates response). When the consecutive pitch distance increases by only one semitone (small pitch displacement), in some cases the harmonic distance with respect to the original suddenly becomes large, and hence reduces similarity. It is therefore possible to increase the pitch displacement so as to, at the same time, decrease the harmonic distance and hence increase similarity, in contrast to Figure 1.

166

Emery Schubert and Catherine Stevens melody shown in the Appendix. The original melody notes were substituted by chord tones from the original implied harmony in certain sections of the test melodies (bar 3 to 4 and bar 7 on lines 3 to 5 in the case of Lightly Row). This transformation required some relatively large leaps for the same harmony transformation (line 3). Most changes required an alteration of an interval of a third or fourth (3 to 5 semitones), setting the lower limit of possible corresponding pitch displacement variations (that is, they can often be displaced by no less than 3 semitones). In the close-harmony condition the notes could be selected so as to be non-implied harmony notes, but with a diatonic shift (that is, another note from the same scale within-key, but not from the same harmony). The distant-harmony condition could be, and on some occasions was, achieved by a semitone shift, so as to select notes that are not only non-chord tones, but also harmonically weakly related to the original (out-ofkey), more weakly than the diatonic shift of the closeharmony condition. While same-harmony alterations were restricted to chord tones, only relatively distal transformations in pitch were possible on most occasions. However, the other two levels of harmony manipulations could be more varied. Therefore, to maintain a factorial design same-harmony manipulations were analysed separately. With this variation, two levels of pitch displacement from the original were produced for close and distant harmony conditions: proximal (where pitch transitions were kept close to the original) and distal (where pitches in the transformation were shifted further with respect to the original). To ensure a large change in pitch contour for the distal conditions, the direction of the melody was changed where possible. Notice, for example, in the bottom two lines of Lightly Row (shown in the Appendix) that in the treated bars (3 to 4 and 7) the melody descends, in opposition to the original melody direction. The distances of the transformations were checked by calculating the sum of the deviations between each note of the original and the corresponding note of the transformed melody (in units of semitones). This value was standardized for each transformation by dividing by the number of notes in the piece enabling comparison of pitch displacement across stimuli. The values are shown at the end of each line of the Appendix. Tempi and key (starting pitch) were not randomized in the screening stage. In Stage 2, two random variables were introduced to reduce the chance of participants relating the test stimuli in Stage 2 to the tempo and key of the screening (original) stimuli used in Stage 1 (see Section 6.4). Small variations in tempo (+10 beats per minute) about the original, and a random key (starting note) between 5 and 76 semitones of the original were chosen for each stimulus to minimize artefactual comparison of similarity with tempo and absolute key with Stage 1 presentations. For each example, a

experienced musicians. Pitch displacement and harmonic distance were both within-subject variables and musical expertise a between-subjects variable. The dependent variable was perceived similarity.

6. Method
6.1 Participants Seventy-two students were recruited from the School of Psychology at the University of Western Sydney and the School of Music and Music Education at the University of New South Wales. Based on a survey, two groups were formed: Those who reported having lessons on an instrument greater than 9 years were placed in the expert group. Those with less than 1 year formal training in music were placed in the novice group. There were 44 (mean age 21.1 years, SD 4.7, youngest 17, oldest 42, 12 males, 32 females) and 28 (mean age 22.4 years, SD 8.3, youngest 18, oldest 55, 8 males, 20 females) participants in the expert and novice groups, respectively. 6.2 Stimuli Nursery rhyme tunes were selected which were thought to be highly familiar to most participants: London Bridge is Falling Down, Mary had a Little Lamb, This Old Man and Lightly Row. In Stage 1 (see Section 6.4), the four untransformed melodies were used. For Stage 2 of the experiment, the melodies were transformed by harmony and pitch displacement. In each case, the opening bar (measure) remained the same as the original to provide a cue for activating the memory of the tune and to provide a harmonic frame of reference for the experimentally manipulated notes. Other sections were altered unless the opening bar was repeated (as might occur in a second musical phrase). The Appendix shows each of the original melodies and their transformations. The chord labels above each system provide examples of implied harmony solutions for the same-harmony conditions and against which the close and distant harmony conditions would be pitted where possible. As discussed in Sections 3.2 and 4, several harmonizations are possible for the original melodies. For the present study, the template harmonization was selected from the three primary triads (tonic, dominant and subdominant), providing a simple approach for harmonizing simple melodies. This solution was checked and agreed to by an experienced Lecturer in harmony. Stimuli were organized such that variations in melodic displacement from the original stimulus were more-or-less independent of variations in harmony. Initially, implied harmony had three values: same, close and distant. For example, consider Lightly Row, the rst

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Implied harmony, contour and expertise marimba timbre was used and the nominal tempo was set to 100 quarter beats per minute for Lightly Row, London Bridge and This Old Man, and 120 quarter beats for Mary Had a Little Lamb. The nominal key was set to C major. Therefore, for each melody, there were six presentations investigated in Stage 2 (the original melody compared with itself, the same-harmony change, and the 262 harmony by contour manipulations). Up to 4 melodies were selected from Stage 1, and therefore up to 24 stimuli were rated in Stage 2. Additional stimuli were rated but are not reported here.

167

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

original?. The latter sentence pointed to a Play button which the participant clicked using the mouse to commence hearing the test stimulus melody. The piece to think about stimulus to be played combination was selected in random order without replacement. When the participant was ready, they pressed a play button. The participant would then rate the similarity of the test stimulus with the one they were asked to imagine. The response scale indicated similarity as a percentage (visually represented scale) from 0 to 100% (with 100% labelled identical). They were encouraged to give their rst impressions, and were allowed to relisten up to four times. The experiment lasted 25 to 30 min.

6.3 Equipment The stimuli were presented to individual participants using software written by the rst author and presented on Power Macintosh 8500 computers. The melodies were played using a sampled marimba sound. All auditory stimuli were played over stereo headphones at a comfortable listening level. 6.4 Procedure Participants sat at a computer that recorded background information before commencing the experiment. The experiment was conducted in two stages: screening and testing. The rst (screening) stage was used to ensure that participants were familiar with each of the test melodies. If they were unfamiliar with a melody, it was deleted from the test stage. If they indicated familiarity with the excerpt (Very Familiar or Familiar) they were asked to select the name of the piece. This label would be used in the second stage of the study. If the participant selected a not sure option, the tune was eliminated from the test stage. In Stage 2 (testing) the participant was asked to think about one of the melodies selected in Stage 1. The tune to think about was prompted by the computer: Imagine the original tune [name of tune as indicated by participant in Stage 1]. How similar is the tune to the

7. Results
Similarity scores were averaged across each piece for each factor level. A repeated measures ANOVA was conducted using the 262 within-subject factors, harmony and pitchdisplacement, and one between-subject factor, skill (novice versus expert). There was a signicant main eect for pitchdisplacement (F[1,70] 148.03 p 5 0.001), but not for harmony (F[1,70] 2.121 p 0.15), as shown in Table 1. There were signicant interactions for skill with harmony (F[1,70] 12.172 p 0.001), and skill with pitch-displacement (F[1,70] 15.568 p 5 0.001). These two interactions can be interpreted by examining the four plots to the right of each pane of Figure 3. Novices rated proximal pitch shifts as more similar to the original than distal transformations (percentage similarity rating M 60.2% SD 2.37% and M 41.39%, SD 2.33% respectively), regardless of the implied harmony relationship. Expert musicians also rated proximal transformations as being more similar than distal transformations, but to a lesser extent (M 53.14% SD 1.89% and M 43.51%, SD 1.86% respectively). However, expert musicians made further distinctions in similarity which corresponded to the implied harmony manipulations. Each level of the 262 factors (harmony, pitch displacement) produced a signicant dierence which, from most similar to least,

Table 1. Pairwise comparisons of similarity response for harmony and contour by skill. 95% Condence interval for dierenceb Lower bound 74.87 2.324 Lower bound 15.208 6.711 Upper bound 1.026 7.027 Upper bound 22.508 12.534

Skill level HARMONY Close Close CONTOUR Proximal Proximal

Mean dierence

Std.

Sig.(a)

Novice Expert Novice Expert

a

Distant Distant Distal Distal

71.922 4.676a 18.858a 9.622a

1.478 1.179 1.83 1.46

0.198 0 0 0

Signicant at p 0.05. Adjusted for multiple comparisons: least signicant dierence.

168

Emery Schubert and Catherine Stevens

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Fig. 3. Perceived similarity by implied harmony and contour for experts and novices. The y axis indicates average perceived similarity percentage. Novice n 28; Expert n 44. Error bar is +1 SE.

were close-proximal, distant-proximal, close-distal, distant-distal, as shown in Figure 3. Figure 3 also shows the mean response to ratings of the original melody by level of expertise. The results are collapsed across the random tempo and starting note (key) levels. The results demonstrate higher similarity of ratings of original melodies by the musical expert group (M 95.2%, SE 0.665%) compared with the novice group (M 85.85%, SE 1.956%). The experts seemed to reach a ceiling, hence the small standard error.

Finally, responses to within harmony changes were examined. As shown in Figure 3, changing only the harmony note (that is, making pitch more distal with respect to original without changing harmony) produced a drop in similarity scores for both experts (M 51.61%, SE 1.23%) and novices (M 54.23%, SE 1.97%). Although similarity scores obtained from novices are slightly higher than experts for same-harmony manipulation, the pitch shift factor alone demonstrates a trend for the novices which can be wholly determined by contour, which is, in decreasing order of similarity:

Implied harmony, contour and expertise proximal contour shift, within harmony change (which constitutes a contour shift in between proximal and distal) and distal contour shift. For the expert group, a rough equivalence or trade-o can be seen between the same harmony condition and proximal contour-distant harmony condition. These ndings reinforce the notion that novices almost ignore the inuence of implied harmony upon similarity judgements, whereas for music experts the two factors have some interplay, with contour, nevertheless, being the dominating factor.

169

8. Discussion and conclusion

Consistent with previous literature, the results suggest that melodic contour plays a major role in the organization of melodies in long-term memory, and more so than deeper level implied harmony, supporting Lamont and Dibbens (2001) nding that contour (surface structure) seems to be more important in determining similarity than deeper structure (in this case, implied harmonic relationships). In addition, the experiment demonstrates utility in investigating contour as a displacement with respect to its dierence (in semitones) from a memorized melody. The greater the displacement of a melody from the source melody, the greater the dierence in perceived similarity. The experiment indicates that implied harmony plays a secondary role, if any, in determining similarity, and that the information is used by more experienced musicians in agreement with Holleran et al. (1995). The results of the Holleran et al. study can be further rened based on the data of the present study. First, for musically experienced individuals, diatonic (close harmony) and chromatic (distant harmony) changes produced some perceptual alterations of the melody. Consistent with music theory, distant harmony shifts are considered more dierent than are close harmony shifts with respect to the original melody. This nding is also consistent with that of Massaro et al. (1980) and Trainor and Trehub (1994b). Second, Holleran et al. reported that less experienced musicians are still inuenced by implied harmony but to a lesser extent than highly skilled musicians. In their study, the low-skilled listeners had at least 5 years of formal musical training. In our experiment, the novices had one year or less formal musical training. This suggests that there may be some kind of sliding scale in the use of implied harmony information, where novices use little of this information in determining the similarity of two melodies. The present study also extends previous ndings to melodies stored in long-term memory, and not just specially composed melodies directly compared with their modications (Holleran et al. 1995; Trainor & Trehub, 1994b), or less familiar melodies (Sloboda & Parker, 1985). Further, it has examined the factors of contour and harmony manipulations and their interaction.

The overlapping similarity ratings for the distant and close harmony conditions could be taken to imply that novice listeners are unable to make ne distinctions between within-key (diatonic) and out-of-key (chromatic) changes. The data suggest that for novice listeners, all dissonances are equal. However, the within-chord (sameharmony) change is rated slightly lower in similarity than the proximal pitch shift condition, meaning that even when the alteration produces greater consonance, the response still has a lower similarity rating. Two explanations of this result are proposed. (1) Harmonic distance for novices is a red herring, and imposes no systematic alteration of similarity judgments (dissonance is dissonance), or (2) there is a third variable, which could not be controlled in the within-chord change of the current study namely voice leading which might cause an articial lowering of similarity scores. Indeed, the attimes unusual leaps introduced in the same-harmony condition could have been responsible for the drop in similarity, masking the eect of harmonic distance. Further research is required to determine which explanation holds. It may also be worth investigating whether these eects are peculiar to the simple pieces chosen, or whether they generalize to more complex kinds of music (such as those using chromaticism and modulation) where the question of harmonization itself becomes still more complex. The question of whether implied harmony would assume a more or less important role in judgements of similarity in other contexts becomes quite interesting. The lower similarity rating of the original melody compared with the memory of the original melody for the novices is an anomalous nding which is consistent with Holleran et al. (1995). Since tempo and key were varied in the present experiment, it would make sense to predict that listeners with good pitch and tempo memory (represented by the expert group) might provide lower similarity ratings for the original stimuli with altered tempi and keys. The reverse was the case. It might be that novices are less condent in identifying precise matching because they are less able to maintain as precise a representation of the melody under investigation. It might also be that expert listeners are more willing to assimilate features of a melody which are well conserved under transformation, such as starting note (key) and tempo (Levitin, 1999). In fact, it is also conceivable that novice listeners may have quite good pitch memory (Schellenberg & Trehub, 2003), and may be more sensitized to these eects. This issue is a point for further investigation. The results present a dilemma in that the prediction derived from music theory seems to apply to experienced musicians, rather than the general, enculturated population. For example, since adults have more experience, they should be more likely to use implied harmony to help organize melodic material than would children. Trainor & Trehub (1994a) reported that by the age of 7 years children

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

170

Emery Schubert and Catherine Stevens tually be found which may have identical direction contours and, as the number increases even further, displacement information may also be insucient. Thus alternative strategies may emerge, such as attending to harmonic implication. Experts are likely to have exibility in global or analytic strategies depending on task, instructions and context. Novices are more limited in available strategies and cognitive resources. Future work is required to determine whether novices are insensitive to implied harmony, or whether they simply do not use their implicit knowledge of it. Infant sensitivity to out-ofharmony changes (Trainor & Trehub, 1992) suggests that infants and adults do encode harmonic information. By building on this nding, the present study suggests that novice listeners may encode this information but cannot or do not use it to the same extent as highly skilled musicians.

were responding at about the same level as adult (mostly novice) listeners in their use of implied harmony information (see also Costa-Giomi, 2003). The question then remains: Do musicians develop a strategy for detecting dissimilarity? If so, the present study may be detecting nothing more than a pre-conditioned response, or a more complete enculturation of the rules of harmony. In post-experiment surveys there was no evidence to support the notion that musically experienced participants were using any conscious strategy that related to harmonic relationships. It seems that the dierence between musician and non-musician is a consequence of training and a more thoroughly absorbed internalization of harmonic relationships and expectancies. It suggests that deeper level structures in music may be dened in terms of enculturation. Ultimately, the implication of this nding for melodic system retrieval and for cognitive architecture is that there are important individual dierences, particularly among novice and expert musicians, in the way musical information is retained and retrieved. The present ndings also have application to the development of automated retrieval systems in that they suggest the need for an additional level of complexity in designing such systems, one which accounts for the underlying harmony of the melodic information. While a semitone shift in some parts of a familiar melody may not violate the contour of the melody, it will clash with the expected harmonic ow (Jansen & Povel, 2004b; Povel & Jansen, 2002), and may be considered less similar than its seed if an experienced listener is being modelled or simulated. Modications to models such as those discussed by Kim et al. (2000) and Holleran et al. (1995, p. 739) may require a musical experience input level. If future research supports the relative importance of the role of harmonic structure with respect to contour in the organization of melodic material, algorithms such as those proposed by Krumhansl (1990) for measuring key-distance, Lerdahl (1996, 2001) for measuring tonal tension, and Jansen and Povels (2004b) on-line harmonic processing model may prove useful in providing information retrieval systems with resources for identifying harmonic relationships implied in melodies. The reason for the greater reliance on harmony information by experienced listeners may be due to the development of more analytic listening and feature extraction and feature weighting processes. However, it may also be an eect of ecient memory strategies. If experienced listeners have much musical material to store, process and retrieve, they require more rened strategies, such as chunking, for organizing this large amount of information (e.g. Ericsson, 2003; Ericsson & Delaney, 1999; McAuley et al. 2004). The simplest, directional contour coding strategy would be of little use if the number of melodies requiring storage and processing becomes very large (see Kim et al. 2000). Some psychologically dierent melodies will even-

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Acknowledgment
This research was supported by a University of Western Sydney Research Encouragement Grant awarded to the second author. We are grateful to Colin Watts from the School of Music and Music Education at the University of New South Wales for his assistance in the preparation of the musical examples and to Melinda Gallagher for assistance with data collection.

References
Anderson, J.R. & Schooler, L.J. (2000). The adaptive nature of memory. In E. Tulving and F.I.M. Craik (Eds.) Handbook of Memory, pp. 557 570. New York: Oxford University Press. Bartlett, J.C. & Dowling, W.J. (1988). Scale structure and similarity of melodies. Music Perception, 5, 285 314. Benward, B. and White, G. (1997). Music in Theory and Practice, Vol. 1, 6th Edn. Madison, WI: Brown and Benchmark. Bharucha, J.J. & Krumhansl, C.L. (1983). The representation of harmonic structure in music: hierarchies of stability as a function of context. Cognition, 13, 63 102. Bigand, E. (1993). Abstraction of two forms of underlying structure in a tonal melody. Psychology of Music, 18, 45 59. Byrd, D. & Crawford, C. (2002). Problems of music information retrieval in the real world. Information Processing & Management, 38, 249 272. Cambouropoulos, E. (2000). Melodic cue abstraction, similarity and category formation: a computational approach. Retrieved 1 August 2003 from http://citeseer. nj.nec.com/cambouropoulos00melodic.html Cambouropoulos, E. & Widmer, G. (2000). Melodic clustering: motivic analysis of Schumanns Trumerei. Paper presented at Proceedings of the III Journes d Informatique Musicale, Bordeaux, France. Retreived 15 July 2003 from http://citeseer.nj.nec.com/423201.html

Implied harmony, contour and expertise

Carterette, E.C., Kohl, D.V., & Pitt, M.A. (1986). Similarities among transformed melodies: the abstraction of invariants. Music Perception, 3, 393 410. Costa-Giomi, E. (2003). Young childrens harmonic perception. Neurosciences & Music, 999, 477 484. Cross, I., Howell, P., & West, R. (1983). Preferences for scale structure in melodic sequences. Journal of Experimental Psychology: Human Perception & Performance, 9(3), 444 460. Davidson, L. (1985). Tonal structures of childrens early songs. Music Perception, 2, 361 374. Deliege, I. (2001). Introduction: similarity perception 54 categorization 54 cue abstraction. Music Perception, 18, 233 243. Dowling, W. (1978). Scale and contour: two components of a theory of memory for melodies. Psychological Review, 85(4), 341 354. Dowling, W.J. (1988). Tonal structure and childrens early learning of music, In J.A. Sloboda (Ed.), Generative Processes in Music, pp. 113 128. Oxford: Clarendon Press. Dowling, W.J. (1991). Tonal strength and melody recognition after long and short delays. Perception and Psychophysics, 50, 305 313. Dowling, W. (1994). Melodic contour in hearing and remembering melodies. In R. Aiello & J. A. Sloboda (Eds.), Musical Perceptions. pp. 173 190. London: Oxford University Press. Dowling W.J. & Bartlett, J.C. (1981). The importance of interval information in long-term memory for melodies. Psychomusicology, 1, 30 49. Edworthy, J. (1985). Interval and contour in melody processing. Music Perception, 2, 375 388. Ericsson, K.A. (2003). Exceptional memorizers: made, not born. Trends in Cognitive Sciences, 7, 233 235. Ericsson, K.A. & Delaney, P.F. (1999). Long-term working memory as an alternative to capacity models of working memory in everyday skilled performance. In A. Miyake and P. Shah (Eds.), Models of Working Memory: Mechanisms of Active Maintenance and Executive Control, pp. 257 297. Cambridge: Cambridge University Press. Halpern, A.R. (1984a). Organization in memory for familiar songs. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 496 512. Halpern, A.R. (1984b). Perception of structure in novel music. Memory and Cognition, 12, 163 170. Halpern, A.R. (1989). Memory for the absolute pitch of familiar songs. Memory and Cognition, 17, 572 581. Halpern A.R., Bartlett J.C. & Dowling W.J. (1998). Perception of mode, rhythm, and contour in unfamiliar melodies: eects of age and experience. Music Perception, 15, 335 355. bert, S. & Peretz, I. (1997). Recognition of music in He long-term memory: are melodic and temporal patterns equal partners? Memory & Cognition, 25, 518 533.

171

Hofmann-Engl, L. (2001). Towards a cognitive model of melodic similarity. In J.S. Downie and D. Bainbridge (Eds.), Proceedings of the Second Annual International Symposium on Music Information Retrieval, pp. 143 151. Retrieved 15 June 2003 from http://ismir2001. ismir.net/pdf/hofmann-engl.pdf Holleran, S., Jones, M.R. & Butler, D. (1995). Perceiving implied harmony: the inuence of melodic and harmonic context. Journal of Experimental Psychology-Learning Memory & Cognition, 21(3), 737 753. Huron, D. (2001). Tone and voice: a derivation of the rules of voice-leading from perceptual principles. Music Perception, 19, 1 64. Jansen, E. & Povel, D.J. (2004a). Perception of arpeggiated chord progressions. Musicae Scientiae, 8(1), 7 52. Jansen, E. & Povel, D.J. (2004b). The processing of chords in tonal melodic sequences. Journal of New Music Research, 33(1), 31 48. Jones, M.R., Summerell, L., & Marshburn, E. (1987). Recognizing melodies: a dynamic interpretation. Quarterly Journal of Experimental Psychology, 39A, 89 121. Kim, Y.E., Chai, W., Garcia, R. & Vercoe, B. (2000). Analysis of a contour-based representation for melody. Retrieved 15 June 2003 from www.media.mit.edu/ chaiwei/papers/Kim.pdf Krumhansl, C. (1990). Cognitive Foundations of Musical Pitch. Oxford: Oxford University Press. Krumhansl, C.L. & Keil, F.C. (1982). Acquisition of the hierarchy of tonal functions in music. Memory & Cognition, 10(3), 243251. Lamont, A. & Dibben, N. (2001). Motivic structure and the perception of similarity. Music Perception, 18, 245 274. Lerdahl, F. (1996). Calculating tonal tension. Music Perception, 13, 319 363. Lerdahl, F. (2001). Tonal Pitch Space. New York: Oxford University Press. Levitin, D.J. (1999). Memory for musical attributes. In P.R. Cook (Ed.), Music, Cognition and Computerized Sound: An Introduction to Psychoacoustics, pp. 209 227. Cambridge, MA: MIT Press. Margulis, E.H. (2003). Melodic expectation: a discussion and model. Dissertation Abstracts International, 64(4-A), 1127. Massaro, D.W., Kallman, H.J. & Kelly, J.L. (1980). The role of tone height, melodic contour, and tone chroma in melody recognition. Journal of Experimental Psychology: Human Learning and Memory, 6, 77 90. McAdams, S. & Matzkin, D. (2001). Similarity, invariance, and musical variation. Biological Foundations of Music Annals of the New York Academy of Sciences, 930, 62 76. McAuley, J.D., Stevens, C., & Humphreys, M.S. (2004). Play it again: did this melody occur frequently or was it heard more recently? The role of stimulus familiarity in the formation of episodic memories for music. Acta Psychologica, 116, 93 108. Narmour, E. (1990). The Analysis and Cognition of Basic Melodic Structures: The Implication-Realization Model. Chicago: University of Chicago Press.

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

172

Emery Schubert and Catherine Stevens

Trainor, L.J., McDonald, K.L., & Alain, C. (2002). Automatic and controlled processing of melodic contour and interval information measured by electrical brain activity. Journal of Cognitive Neuroscience, 14(3), 430 442. Trainor, L.J. & Trehub, S.E. (1994a). A comparison of infants and adults sensitivity to Western musical structure. Journal of Experimental Psychology: Human Perception and Performance, 18, 394 402. Trainor, L.J. & Trehub, S.E. (1994b). Key membership and implied harmony in Western tonal music: developmental perspectives. Perception and Psychophysics, 56, 125 132. Trehub, S.E. (1990). Human infants perception of auditory patterns. International Journal of Comparative Psychology, 4, 91 110. Trehub, S.E. (2001). Musical predispositions in infancy. In R.J. Zatorre & I. Peretz (Eds.), The Biological Foundations of Music, Annals of the New York Academy of Sciences, 930, 439 442. Trehub, S.E., Bull, D., & Thorpe, L.A. (1984). Infants perception of melodies: the role of melodic contour. Child Development, 55, 821 830. Van Egmond, R., & Povel, D.J.L. (1996). Perceived similarity of exact and inexact transpositions. Acta Psychologica, 92, 283 296. Welker, R.L. (1982). Abstraction of themes from melodic variations. Journal of Experimental Psychology: Human Perception and Performance, 8, 435 447. Zhu, Y. & Kankanhalli, M. (2002). Similarity matching of continuous contours for humming querying of melody databases. LIT Technical Report, February. Retrieved 15 June 2005 from www.comp.nus.edu.sg/cs4241/ TR2002.pdf

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Povel, D.J. & Jansen, E. (2002). Harmonic factors in the perception of tonal melodies. Music Perception, 20(1), 51 85. Schellenberg, E.G. (1997). Simplifying the implicationrealization model of melodic expectancy. Music Perception, 14, 295 318. Schellenberg, E.G., & Trehub, S.E. (2003). Good pitch memory is widespread. Psychological Science, 14(3), 262 266. Schoenberg, A. (1967). In G. Strang (Ed.), with the collaboration of L. Stein (Ed.). Fundamentals of Musical Composition. London: Faber. Sloboda, J.A. & Parker, D.H.H. (1985). Immediate recall of melodies. In P. Howell, I. Cross & R. West (Eds.), Musical Structure and Cognition. London: Academic Press. Stoll, G. & Parncutt, R. (1987). Harmonic relationship in similarity judgments of nonsimultaneous complex tones. Acustica, 63(2), 111 119. Tan, N., Aiello, R., & Bever, T. (1981). Harmonic structure as a determinant of melodic organization. Memory and Cognition, 9, 535 539. Tillmann, B., Bharucha, J.J., & Bigand, E. (2000). Implicit learning of tonality: a self-organizing approach. Psychological Review, 107(4), 885 913. Tillmann, B. & Bigand, E. (2004). Further investigation of harmonic priming in long contexts using musical timbre as surface marker to control for temporal eects. Perceptual & Motor Skills, 98(2), 450 458. Trainor, L.J., Desjardins, R. N., & Rockel, C. (1999). A comparison of contour and interval processing in musicians and nonmusicians using event-related potentials. Australian Journal of Psychology, 51(3), 147 153.

Implied harmony, contour and expertise

173

Appendix: Stimuli used in the experiment

Numbers to the right of each line indicate the average deviation of the pitches with respect to the original (in units of semitones, or chroma). In the screening stage, only the original stimulus, shown as the top line, was

used. In the test stage all stimuli were compared against a participants memory of the tune represented by the original. An implied harmony solution is shown as chord labels above the original melody (see text). The braces below each system indicate the sections of bars in which test stimuli notes were transformed.

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

Downloaded by [b-on: Biblioteca do conhecimento online UA] at 12:38 01 May 2013

174 Emery Schubert and Catherine Stevens