APPLICATIONS| Tha aha ne er VT). Baan ea aadLibrary of Congress Cataloging-in-Publication Data ‘Computer assisted language learning (CALL) : media, design, and applications / introduced and edited by Keith Cameron Poem Includes biographical references and index. ISBN 90265 1543X (hard) 1. Language and languages—Computer-assisted instruction. 1. Cameron, Keith. P53.28.06634 1999 418.00285—de21 99-12697 ap Cover design: Paula van der Heydt ‘Typesetting: Red Barn Publishing, Skeagh, Skibbereen, Co. Cork, Ireland Printed in The Netherlands by Krips b.v., Meppel © 1999 Swets & Zeitlinger b.v., Lisse All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, ‘mechanical, photocopying, recording, or otherwise, without the prior written permission of the publishers. ISBN 90 265 1543 x (hardback)Contents Introduction Keith Cameron (Exeter, SECTION I MEDIA The Shape of Computer-Mediated Communication D.M. Paramskas (Guelph) User-driven Development and Content-driven Research Wilfried Decoo and Jozef Colpaert (Antwerp) Authoring and the Academic Linguist: the Challenge of Multimedia CALL David Bickerton (Plymouth) SECTION II DESIGN Design Processes in CALL: Integrating Theory, Research and Evaluation Mike Levy (Queensland) Evaluating Performance, Approach and Outcome Robin Goodfellow (Open University, Computers as Tools for Sociocollaborative Language Learning Carla Meskill (Albany) SECTION III CALL APPLICATIONS Speech Recognition in Computer-Assisted Language Learning Gregory Aist (Carnegie Mellon) More Intelligent CALL Henry Hamburger, Michael Schoelles and Florence Reeder (George Mason) 13 35 59 83 109 141 165 183Grammar Checking for CALL: Strategies for Improving Foreign Language Grammar Checkers 203 Cornelia Tschichold (Basel) Visual Grammar: Multimedia for Grammar and Spelling Instruction in Primary Education 223 Gerard Kempen (Leiden) Corpus CALL: Corpora in Language and Literature 239 Glyn Holmes (Western Ontario) ‘The Missing Link in Computer-Assisted Writing 27 Martha C. Pennington (Luton) EPILOGUE Is your CALL Connected? Dedicated Software vs. Integrated CALL 295 Tan M. Richmond (St Anne, Nova Scotia))INTRODUCTION Keith Cameron University of Exeter Computer Assisted Language Learning (CALL) is one of the success stories of the end of the twentieth century. Over the last fifteen years it has grown from fumbling infancy to youthful confidence. The perseverance and the enthusiasm of researchers into ways and means of improving the teaching and learning of languages using a technological medium which was designed for other purposes have been encouraging and constitute a phenomenon which, if one were in expansive mood, could be construed as a tribute to the ingenuity, intelligence and dedication of language teachers. The momentum which at first came from practitioners who were anxious to design programs which would allow learners to practise some of the more mechanical aspects of language learning has been stimulated since those early years by the progress in hard- ware and technological enhancement, by developments in the field of Artificial Intelligence, by the installation of international electronic networks, by the World Wide Web and latterly by governments whose interest has been caught by the belief that CALL could be advantageous not only pedagogically but financially also. The progress of CALL has not been without its critics. There is a wide- spread fallacy that increased use of computer assisted learning will lead to a reduction in the need for teachers; its advantages over traditional systems of learning has been questioned: its implementation is rendered difficult by the lack of resources, etc. Such a debate is healthy and it is only normal and right2 KEITH CAMERON that a new teaching and learning method should be challenged. I believe, how- ever, and I say it with conviction, that the computer is here to stay and its use both inside and outside the classroom will increase. It is not a replacement for the teacher but a useful adjunct to teaching, it facilitates language and distance learning and it gives, through the Intemet, users access to a whole wealth of cultural material which can stimulate learning.! CALL programs which are in existence are not a perfect panacea. Much research still has to be effected, hence the need for this volume. The increase in involvement in CALL is attested by the number of confer- ences, the various publications and the quantity of language workstations which are proliferating over the globe. In spite, however, of the waxing num- bers of practitioners and of researchers, there is still a certain lack of cohesion between the various research groups and perhaps also a certain lack of com- mon vision. This publication is intended * to provide a focus for future research; + to be a stimulus to graduates embarking on Masters or Doctoral pro- grammes; * to encourage reflection amongst established CALLers; . to increase awareness of what has been done and what still has to be done in the CALL domain, * to establish a reference work which is unavoidably incomplete but will constitute a source of essential information for all those involved in CALL. Such is our ambition and explains why those who have been asked to con- tribute to this volume are recognised international experts in their respective fields. Each one received the same brief: ‘to review what has been done, what exists, to outline current developments and give useful pointers for the way ahead. Each chapter will give a broad bibliography which will cite books, jour- nals, journal articles, software, URLs, etc., relevant to the research issue being discussed’. We should not lose sight of the fact that we are primarily involved in CALL and not CALT, i., in language learning and not teaching. One could object that the distinction is merely splitting hairs, but I feel our main objective in designing CALL programs has to be to improve the learning capacity of those who are being taught a language through computerised means. With the wide- spread use of the Internet, people have realised its tremendous potential as a source or information and as a means of obtaining access to, and distributing, data both quickly and effectively. Many human-hours have been spent con- verting texts into HTML and to evolving click-on links with data banks throughout the system. Libraries, museums, schools, universities, governmentaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.INTRODUCTION 5 interesting to muse upon the effect that the commercial market has had upon the direction of their research and is a good example to use in arguing the case for greater co-operation between academia and industry. Many would-be CALLers are held back by their lack of computing knowl- edge and so therefore look to authoring tools to facilitate their task. David Bickerton, using as his starting point the recent survey of authoring tools undertaken by the Rapido project, considers their use for Multimedia CALL. The tools that exist are not all user-friendly and often do not fit in with the author’s ambitions or aims. The advent of multimedia and the Internet opens up the possibilities for even further progress in their development and avail- ability — a process which is not without its inherent problems and obstacles and which constitutes yet another area for more research and attention. The importance of well planned design, whether it be of authoring systems or of learning programs seems self-evident. Too many people have too often not given enough attention to this obviously crucial domain. The reasons for this are diverse — in many cases it is because researchers are particularly inter- ested in examining the effectiveness of a given project and CALL programs have been evolved which are content-driven and/or theory-driven which have been modified in the light of use and the limitations of the hardware. As Michael Levy points out ‘though many projects and leaming environments have been completed, relatively little hard information has been accumulated for the potential CALL designer’. We have to make sure that the programs are designed in a way which maximises the computer's capabilities and allows them to be integrated with other non-computer activities, that they cater for user-learner differences, that they incorporate proven cognitive theories, etc. The effectiveness of CALL methodology and design are bywords which run through this collection. It is not easy to evaluate any system of language learn- ing, let alone CALL where there are additional significant factors to be con- sidered, because of the variables involved. This is not a reason, however, why we should not do so. Robin Goodfellow, in a personal case study, addresses the problem and indicates how ‘a better understanding of the way the learner con- ceptualises the interaction [with a given program] can lead to a more appro- priate assessment of its outcome’. The fruits of such evaluation can be fed back into the design of the program and constitute a basis of received knowledge for future development. One of the interesting offshoots of this approach is that it emphasises the necessity to research more the effectiveness of the learning environment. If we can stress the importance of this area, research information obtained in the pursuit of CALL could effect the whole spectrum of pedagog- ical and educational techniques. The learning environment and the interaction between user and program are weighty considerations in the design of any CALL material and the computeraa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.INTRODUCTION 9 The co-authors of this volume have, in their study of current practices, raised a number of problems and suggested a number of ways of researching their solution. What is useful, too, is the serendipity-factor. As one reads the views and the experience of others, cross-fertilisation can take place and new ideas and projects emerge. Another factor is that by discovering what is going on elsewhere or what has already been achieved can save an individual a great deal of wasted, or if not wasted, unnecessary effort. We have reached, it seems to me, the stage in CALL research, when there is areal need to take stock of what has been put in place and for researchers to consider being more methodical in their choice of project. This publication goes a part of the way towards reaching that objective by indicating the weak- nesses in our knowledge which have to be turned into strengths. There is within the pages that follow a message which is often expressed obliquely but which, nevertheless, is ever present. In spite, or perhaps because, of their experience, the contributors point to the need for reflection, to ponder care- fully the direction in which CALL is to go. The design of programs is depen- dent upon the media, the user, the learning environment and the curriculum. Before we adopt a faulty methodology, this is the moment to examine what we are doing and where we want to go more carefully. Because of the nature of CALL and the diversity of the educational and cognitive factors and of its users, there is a great need for more collaboration — collaboration between researchers, both those who work within academic circles and those in com- merce and industry. Commercial interests must not lead to products which dictate academic needs and are not fully consonant with their requirements. This is a time for close co-operation between the two worlds, for them to work together towards a common goal. The seventeen contributors, who come from seven different countries, reveal a faith in the future of CALL, a faith that is sometimes coloured by a tinge of scepticism, but healthy doubt has always been one of the leading forces in research. It can but be hoped that this book will encourage others to join in the pursuit of the perfect learning methodology and of the continuing enhancement of computer assisted language learning. NOTES 1 Fora discussion of the role of culture in the CALL curriculum, see my paper, ‘CALL, Culture and the Language Curriculum: An important issue?”, in Licia Calvi and Walter Geerts (eds) (1998) CALL, Culture and the Language Curriculum. Berlin, Heidelberg, New York: Springer. pp.l-I1. See Borchardt, F. ‘On the history and aesthetics of screen design (or, Why do most screens put leamers to sleep?)’, in Keith Cameron (ed.) (1998) Multimedia CALL: Theory and Practice. Exeter: Elm Bank Publications, pp.3-10.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.14 D.M. PARAMSKAS distinctions have now become blurred; CMC harbours its predecessors within its ample folds. Conceptually, CMC originates in the linguistic domain of discourse analy- sis, which grew in importance in the wake of the European ‘Threshold’ lan- guage-learning (LL) approaches (Stubbs, 1983). ‘Computer-mediated communication’ emerged as a specific term because it was needed to distin- guish between the domain of discourse analysis in the contexts of natural/oral language interaction and those which occur in a computerized context. Discourse in the latter case, although written, was seen as so marked by oral influences that the principles of oral analysis yielded the most informative results. (Berge & Collins, 1995a). Early on, the Internet attracted the attention not only of linguists but also of language teachers, who were already deeply involved in ‘traditional’ or stand- alone applications of CALL. As the Internet has grown, and the World Wide Web has mushroomed, CMC has gradually integrated the pedagogical goals and techniques developed by CALL practitioners and researchers. As a result, CMC has become increasingly difficult to define (Santoro, 1995; Salaberry, 1996). Although some of its formats are distinct from each other and from tra- ditional CALL, many overlap in both theme and function. Those who have tried to construct taxonomies of CMC have been hard put to produce a tidy theoretical structure, and one can almost hear the effort to avoid the category of ‘odds and ends’. 2. TAXONOMIES A few taxonomies are based on user groups (one-to-one; one to many; many to many); some on types of activities related to language competence levels (Lafford & Lafford 1997). Most others, especially those oriented to LL, focus on the CMC format involved: Internet Relay Channel (IRC), UseNet, MOOs (Multiple-user Object Oriented domains), chatrooms; e-mail and listservs; WWW sites (Ellis, 1995). A gradual consensus is, however, emerging as to a taxonomy of pedagogi- cal CMC, based on function (Harris 1995; Séguin, 1997). Harris and Séguin generally concur on the nature of interpersonal exchange (‘Ia communication interpersonnelle’). They agree closely on four subdivisions: (1) e-mail pen- pals, (2) global classrooms, (3) role-playing in pre-constructed virtual contexts as well as online encounters in one’s real persona, and (4) online mentoring, tutoring and teaching support. Séguin, who stresses a little more than Harris does the community-building aspects of CMC, proposes a fifth subdivision: ‘l'apprentissage collaboratif’ , collaborative learning.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.22 D.M. PARAMSKAS But, all too often, technical, administrative and human difficulties abound: lack of response or delayed response from participants, loss of support or fund- ing on one side or the other, inability to find materials of equal interest to both groups, incompatible communication systems, Internet traffic jams, etc. (Sanaoui & Lapkin, 1992). The choice of a common language poses its own unique problems within this format: second language utterances below advanced level tend to bore native speakers, And joining two second-language learner groups whose inter- languages differ markedly presents other pitfalls (Ellis, 1995), Teachers, anx- ious to ensure the maximum context and relevance, tend to over-control the exchanges by requiring the use of certain structures or vocabulary and partic- ularly by imposing evaluative criteria for correct usage. Under these condi- tions, the e-mail format is often reduced to an electronic version of the traditional paper composition, requiring much thought and revision, and call- ing on learning strategies irrelevant to the medium, which serves only to pub- lish the end result. Conferencing, on the other hand, avoids these pitfalls: technical difficulties related to connectivity are non-existent. Responses are immediate and invite immediate reply. The multiplicity of discussion topics raised by the students themselves reduces the instructor’s voice to one among many. And while the instructor does monitor the conference, systematic language correction is nec- essarily relegated to follow-up activities in the classroom (Beauvois, 1992). 3.2.2 Listservs Listservs are typically controlled by a moderator and restrict participation to students from one or more classes, to students at a particular level of compe- tency (e.g., JASETTE sponsored by Laval University, in Québec) or to those interested in cultural contacts (e.g., CAUSERIE, at the Université du Québec). Listservs avoid some of the pitfalls that beset e-mail exchanges. The par- ticipants are either self-selected or carefully monitored by teachers, and the postings, unlike personal e-mail, are automatically sent to everyone as well as archived for possible retrieval. But listservs do not avoid all the pitfalls of e- mail. The main problems they present are pedagogical. Can participating teachers resist the temptation to use the forum for immediate language cor- tection? How can the activity be integrated into a formal course? How can stu- dents be motivated to post regularly messages that are truly interactive when the interaction is asynchronous, that is, irregular or delayed? And the listserv forum has its own drawbacks. The built-in delay in response allows students to skip over messages received. Consequently, assigned discussion topics can be disappointing: they often produce not inter- action but serial monologues composed according to a model provided by theaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.26 D.M. PARAMSKAS supported by visuals, animation and audio, thus continuing some of the best pedagogical aspects of multimedia (e.g., L’Ambassade de France a Ottawa/L’ Alliance francaise: L’Explorateur culturel / Say it in French). It is easy to find weaknesses and retreats in the CMC field. And yet CMC opens doors that simply did not exist before. Motivated second-language stu- dents now have the possibility for authentic use of the language they are study- ing and for exploring culture in ways that the traditional classroom or the text-based library could not offer. While there are no studies on the LL uses of the virtual library, anecdotal evidence from listserv forums (FLTeach, Infobourg, Edufrangais) and the descriptions of numerous web-based research projects (Lafford & Lafford, 1997; Warschauer, 1995) testify to the fact that students gravitate more easily and more eagerly to WWW sources than to those found in the traditional library or language lab. The cultural enrichment which results can only be a positive factor in LL motivation (Ellis, 1995). 4, CONCLUSION Just as in every previous instance of advances in educational technology, the- orists are divided on the issue of benefits and drawbacks. Oppenheimer (1997) summarizes the case for the pessimists: students come away from the com- puter less literate and above all Jess humanistically oriented, less critical in their thinking. CMC is merely the latest manifestation of an unjustified, expen- sive and perhaps dehumanizing use of technology misapplied to learning in general. On the other hand, Young (1997) defends the optimistic side: while any technology will be misused to some extent, all technology has brought some- thing to the advancement of learning. In particular, CMC joins and reinforces current theories related to leamer-centeredness: project-based learning, peer learning, collaborative learning, explorative learning. The traditional lecture is no longer sacrosanct; the teacher deeply involved in CMC must trade his or her role of ‘sage’ for that of ‘guide’ and use classroom time for ‘Its unique advan- tages: person-to-person discussion of important issues, exploring open-end- edness and confronting ambiguities. The controversy will not be settled soon, because the opposing attitudes toward technology are grounded in fundamentally differing assumptions about the role of the teacher and the nature of learning. At one extreme, many teach- ers remain dedicated to the instructor-centred format, in which teachers, as magisters, carefully guide the learner through the various steps they deem nec- essary for ‘real’ learning; the student is seen as a form of tabula rasa upon which the teacher will inscribe the appropriate data. (Ventimiglia, 1994). Ataa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.30 D.M. PARAMSKAS Mandell, L. (1997) ‘Virtual Encounters: Using an Electronic Mailing List in a Literature Classroom’, Profession 1997 (Modern Language Association of America): 126-132. Marquet, P. (1998) ‘Quelques études comparatives entre des enseignements classiques et des enseignements médiatisés’, Cari-info, le mensuel des technologies de l'information et de la communication pour Venseignement 94, Avril. Online: http://cari-info.ac-stras- bourg.ft/Articles/Numero094/EnsMediatise/Article.htm ‘McCreary, E. (1989) ‘CMC and Organizational Culture’ in Mason, R & Kaye, A. (eds) Mindweave: Communication, Computers, and Distance Education. Oxford and New York: Pergamon Press. Oppenheimer, T. (1997) ‘The Computer Delusion’, The Atlantic Monthly, July. Online: http://www theatlantic.com/issues/97jul/computer.htm Paramskas, D.M. (1982) ‘Courseware-Software Interfaces: Some Designs and Some Problems’, CALICO Journal 1 (3): 4-6. Paramskas, D.M. (1993) ‘ELAO: Genaseet Avenir’, in Liddell, P, (ed.) Theory and Application. Proceedings of CCALL2/CCELAO2. Victoria, B.C.: University of Victoria Press: 65-77. Paramskas, D.M. (1995) ‘CMC and Second Language Learning/La télématique ct T’Enseignement des langues secondes', in Knoerr, H. (ed.), The Use of New Technologies in Teaching and Learning Languages/L'Utilisation des nowvelles technologies en enseignement et apprentissage des langues, Ottawa: University of Ottawa (Centre de Recherche en Enseignement et Apprentissage des Langues) and Canadian Foreign Service Institute: University of Ottawa Press: 64-85. Paramskas, DM. (forthcoming) ‘Developing L2 Written Fluency Using Computer-Mediated Communication (CMC), in Bisaillon, J. & Broady, E. (eds), LL Writing in a Computer Environment: Transatlantic Perspectives. Patrikis, P. (1996) ‘Where is Computer Technology Taking Us?" conference given at Washington and Lee University, Leesburg, VA; based on a shorter version in ADFL Bulletin 26 (2). Paulsen, M. (1993) ‘From Bulletin Boards to Electronic Universities: Distance Education, Computer-Mediated Communication, and Online Education’, Research in Distance Education 5 (2): 11-12. Plass, J.L. (1998) ‘Design and Evaluation of the User Interface of Foreign Language Multimedia Software: a Cognitive Approach’, Language Learning & Technology 2 (1). Online: http://polyglot.cal.msu.edu/litivol2num | /article2/index.html Pouts-Lajus, $. & Riché-Magnier, M. (1989) ‘Les technologies éducatives, une occasion de repenser la relation pédagogique’, Cari-info, le mensuel des technologies de informa- tion et de la communication pour Venseignement. Online: http://cari-info.ac-stras- hourg. fr/Dossiers/Dossiers-frame htm Redeker, R. (1997) ‘L’école doit-elle fabriquer des internautes ou instituer des citoyens?", Le Monde, 12 septembre. Remon, J. (1997) ‘L’Enseignement du frangais langue seconde par le WY", Colloque LIL97. L'informatique dans les études francaises, Online: http://qsilver. queensu.ca/frenclis Confs/L1197/JosephineRemon.html#haut Rennie, J. (1997) “Internet and Second Language Instruction’, ERIC/CLL minibib. Online: http://www.cal.org/eriecl/Minibibs/Internet.htm Salaberry, M.R. (1996) ‘A Theoretical Foundation for the Development of Pedagogical Tasks in Computer-Mediated Communication’ , CALICO Journal 14 (1): 5-34. Sanaoui, R. & Lapkin, S. (1992) ‘A Case Study of an FSI-Senior Secondary Course Integrating Computer Networking’, Canadian Modern Language Review/RCLM 48 (3): 525-545. Santoro, G. (1995) ‘What is Computer-Mediated Communication?’, in Berge, Z.L. & Collins, M. (eds) Computer Mediated Communication and the On-Line Classroom v. | ‘Overview and Perspectives’. Cresskill, NJ: Hampton Press. ‘Séguin, P. (1997) ‘Internet: une technologie pour l'apprentissage’, report to the Ministry of Education, Québec, Canada. Online: http://www. virtuel.collegebdeb.qc.ca/pedagogicaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.34 D.M. PARAMSKAS Robert Peckham: French Lessons from Everywhere @ Globe-Gate: hitp://ftnc.utm.edu/rpeckham/FRLESSON.HTM Steve Thome: WWW Foreign Language Resources http://www itp.berkeley.edu/~thome/HumanResources.html AltaVista Search Services Web Translation Service http:/faltavista.digital.com. or via hyperlink from http://www.networx.on.ca/-jwalker/waitl.htm Ambassade de France a Ottawa: L’explorateur culturel: http://ottawa.ambafrance.org Alliance frangaise/Say it in French: http://;www.ambafrance.org/ALF Commission spéciale de I’ AATF pour la promotion du francais aux Etat-Unis/ AATF Task Force on the Promotion of French in the United States: Media and Technology in the Teaching of French. http: firesearch. umbc. edu/~frtv/futur1.html L’Hexagone French Club: http://www.nyp.ac.sg/fs/fs_hexa.htmaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.38 W. DECOO AND J. COLPAERT A third group consists of users who need to be tested for certain language skills through specific CALL materials. We should appropriately talk of CALT - computer-assisted language testing. Since 1991 Didascalia has been devel- oping and implementing computerised language tests for the Belgian Civil Service Commission, which is responsible for the evaluation of employees in the public sector. One project concerns first language mastery (Ariane for Dutch or French), another, second language mastery (Atlas for Dutch or French). About 15,000 candidates are tested every year through these CALT materials. The projects are on-going as new modules are being added every year. Here also the target groups are well defined and the objectives of the test- ing very precise. A description of Atlas is given by Troubleyn, Heireman and Van Walle (1996), a description of Ariane by Van Walle, Heireman and Troubleyn (1996). In this part of the chapter we will only concentrate on the school-based lan- guage learners, for here the ‘user-driven’ factor is the most pertinent because of its dynamics. It is also the most interesting subject in terms of extensive impact on thousands of learners and in terms of commercial significance. In the two other groups just mentioned, the ‘user-driven’ impact is actually a pre- condition, since those who commission the material define in advance, albeit with our input, what the objectives, content, strategies and outcome should be. Of course, the end-users in the business and the testing realms do give some feedback which influences subsequent versions and new projects, but their impact is not comparable to that of the school-based learners. 2.2, The impetus coming from school-based learners ‘When we started developing software in the mid-eighties for this ‘general pub- lic’ of school-based learners, a certain concept of ‘the user’ guided our research focus, We envisioned this user in a way as open ended as possible. Any person, dealing with a certain domain of language learning, should be able to determine in a software program what would match her/his situation and needs. The user should be absolutely free to explore, discover and decide for her/himself. We called this principle ‘external versatility’, meaning that the program must be adaptable intemally to a number of external factors, espe- cially the wide variety of needs of potential users. In 1986 an IBM research contract at the University of Antwerp allowed us to develop and implement the principle of external versatility for French con- jugations, which we had first worked out as the VICAT-program in the TICC- IT environment at Brigham Young University (Decoo, 1985). The new program, Verbapuces, can generate more than a million verbal forms in a vari- ety of presentation, training and evaluation strategies. A menu system allows immediate adaptation to the wishes and needs of any user, from the veryaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.42 ‘W. DECOO AND J. COLPAERT customers. Of these sales, 99% pertains to curricular CALL. These figures speak for themselves. In the past few years, our curricular software has followed the trend of tech- nological innovations, less because of the trend in itself, but because of the expectations and requests of users on the advanced technological front. In 1993 Vocapuces for Windows combined in one program both the versatility of the original concept (enabling any combination from the database) and the chronological build-up of lessons from Eventail — by then covering six vol- umes and 9,000 lexical items in context (Uyttersprot et al., 1993). A multi- media CD-ROM, Eventail Interactif 1-2, was launched in 1996, containing, among other things, 1,250 interactive exercises (27,000 items), with the usual additions and refinements that characterise evolving software (see also 3.4). These developments bring us to the ultimate ‘interactive textbook’, as the per- fect match between all classroom material and CALL (Schonenberg & Heughebaert, 1996). The public response to this new concept encouraged us to launch Junior Interactif and Eventail Interactif 34 in November 1997. Two months later, 2,696 copies of these multimedia products had already been sold in Flanders, and this was clearly only the start of a major movement. But again, we had to learn the hard way that non-curricular CALL, even in flashy multimedia format, and however original and well-promoted, does not meet the real needs of our public. In 1996 we developed Flexicon, a multi- media CD-ROM for the learning and practice of ‘exciting new’ vocabulary in the mother tongue (Dutch), since new words are constantly emerging in fash- ionable language use. A needs analysis had ‘proven’ there would be great interest for such a tool. Flexicon is a highly colourful, visual and oral program, including dazzling strategies and games on various levels, usable both as a tool and as a tutor. It is supported by a website with an ongoing contest which includes prizes. Five thousand pre-release free CD-Rom’s were distributed through various channels. Press reviews were excellent. We were so convinced of its coming success that we envisioned a yearly update. But by the end of 1997, we had sold only 123 copies in Flanders and the market was static. We should add that in all of these user-driven developments, the public never asked for cheap gadgets such as bells and whistles. Customer reaction and press reviews have always appreciated the sober office look of these materials and, these past few years, the punctilious harmonisation with the principles of the MS-Windows interface. Efficacy and efficiency are more valued than creative confusion or unfamiliar intricacy. Here also, the users have set the trend. 2.3. The place of the teacher The natural consequence of this major shift towards the home is that we have become less and less concerned about the ways teachers integrate CALL intoaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.46 W. DECOO AND J. COLPAERT adequate interactive procedures, eventually implementing classroom use, and evaluating the results. The limited content that is needed for a research project may be lifted from elsewhere and is not really research-based to take into account some unique properties of CALL as to content structuring. Over the years it became clear that the selection and organisation of content should become a key issue in CALL. The title theme of the 1997 Calico Conference at West Point reflected this concern: ‘Content! Content! Content!” In our own R&D, selection and organisation of content has always been at the core of our concems, simply because our software was never conceived as a demonstration, but has always been aimed directly at a demanding public expecting material for years of study within one CALL program. An import- ant added value of this approach is the possibility to also test the achievements of the learner. Well defined content and objectives allow to test with more validity and objectivity. The relative importance of this content work is naturally reflected in the composition of our research team: less than 10% of the time goes into actual programming. Everything else is devoted to other design and development activities, with a major emphasis on the selection and organisation of content. In the next sections we will present our experience with content-based research for CALL, following again the chain of events that has brought us to the pre- sent point — and beyond. 3.2. Content-based research in generative models ‘We can be brief about this first phase, though it laid the foundation for the important principle of external versatility (see above, Section 1.2). At the time of our first IBM-sponsored research contract (1985-1987), our work concentrated on the capability of the computer to generate the required lin- gual forms from a rule base. Verbapuces for French conjugations, and its variants for Spanish (Verbinario) and Italian (Verbitalia), could each gener- ate more than a million verbal forms, while the whole program remained under 256 K and could fit on a 5.25 floppy disk — the norms of the period. Though limited in its didactic significance (at least in the sense that com- municative trends frawn upon grammatical training), the program allowed us to study, first the criteria and the complexities of rule-based generation and second the linkage between the generated forms and the didactic strate- gies to exploit them. But more than anything, in the eyes of our public, lan- guage software became identified with extensive content usable for years of study and for objective testing. It obliged us to make of extensive content a trade mark for our CALL and oriented us further towards this field of research,aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.54 W. DECOO AND J. COLPAERT more emphasis on communicative training in five skills (listening, reading, speaking, conversing, writing). The transposition to Arcades Interactif was ready by September 1998. This interactive textbook includes a wide variety of training forms based on an extensive communicative typology (Decoo, 1998, Chapter IIT) Two dynamics meet in the interactive textbook: on the one hand the CD- ROM as a data carrier with multimedia capacities, on the other hand the text- book in its entirety, divided into different volumes corresponding to different classes. Previous data carriers could not store the complete contents of a text- book, audio and visual materials included, in less than a couple of hundred flop- py disks. Our research focussed on the organisation of huge amounts of different content in appropriate structures and its complex encoding, in order to be able to implement interactive strategies on the basis of existing printed materials. The link with the schoo! context had already been established beforehand; the soft- ware only added an additional dimension to an existing methodology. Eventail Interactif and Arcades Interactif are more than an ‘electronic’ text- book whose contents are simply made available and searchable in electronic format. It aims to improve the efficiency of learning by presenting the con- tents of an existing textbook in a flexible and interactive way, with multimedia and hypertext support, and with an efficient evaluation and monitoring system (for a more detailed description of the program, see Schonenberg and Heughebaert 1996). 3.5. New research: towards a generic didactic-linguistic model Content thus has become a core issue for our research centre. But these past few years, we have also started to realise more and more how much time and energy is being wasted in content development. Indeed, CALL applications have a limited lifetime due to rapid changes in operating systems, authoring tools, programming languages, hardware, telematics, language itself, language learning didactics, scientific approaches and expectation patterns of teachers and learners. If, in addition, content structure is too specific for each distinct application, there is a lack of transferability and much valuable content, devel- oped for past software, is lost. In the potential interrelation between various CALL environments and applications, content is also not easily transferable. How could one, for exam- ple, directly implement interesting material found on the Internet into a tutor- ial-type CALL program? Or how can material developed for one type of strategy or skill easily be reused for other strategies and skills in another CALL application? Therefore our research will now focus on the elaboration of a generic model for content structuring aimed at increasing reusability of contents. This in-aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.58 W. DECOO AND J. COLPAERT Decoo, W., Dugardeyn, G., Huysmans, M.-L., Preckler, E., Van Tulder, A. (1984) Eventail 1. Lier: Van In. (Subsequent volumes were published a rate of one a year.) Decoo, W., Heughebaert, E., Schonenberg, N. & Van Elsen, E. (1996) “The Standard Vocabulary Programs of Didascalia: in Search of External Versatility and Didactic Optimisation’, Computer Assisted Language Leaming 9 (4): 319-338. Glaser, R. (1960) ‘Comments on the Use of Teaching Machines Techniques in the Language Laboratory’, in F.J. Oinas (ed.) Language Teaching Today. Bloomington, Indiana; Indiana University Research Center in Anthropology, Folklore and Linguistics. Heughebaert, E. & Troubleyn, K. (1996) ‘Company-Specific Language Courseware: Bellingua and Linguatex as Case Studies’, Computer Assisted Language Learning 9 (4): 349-358. Hoven, D. (1997) ‘Instructional design for multimedia: Towards a learner centered CELL (Computer Enhanced Language Learning) Model’, in Kathryn A. Murphy-Judy (ed.), Nexus: The Convergence of Language Teaching and Research Using Technologies. Calico Monograph Series, Vol. 4, Duke University, NC: Calico, pp.98-111. Hubbard, PL. (1996) ‘Elements of CALL Methodology: Development, Evaluation and Implementation’, in M.C. Pennington (ed.), The Power of CALL. Houston, TX: Athelstan Publications, pp. 15-32. Kemmis, S., Atkin, R. & Wright, E. (1977) How do students learn?, Working Papers on com- puter-assisted language learning, UNCAL Evaluation Studies, Norwich: University of East Anglia. Levy, M. (1997) Computer-Assisted Language Learning: Context and Conceptualization. Oxford: Clarendon Press. Pederson, K.M. (1988) “Research on Call’, in W.F. Smith, Modern Media in Foreign Language Education: Theory and Implementation, Lincolnwood, IL: National Textbook Company. Pennington, M.C. (1996) ‘The Power of the Computer in Language Education’, in M.C. Pennington, (ed.), The Power of CALL. Houston, TX: Athelstan Publications, pp.1-14. Schonenberg, N. & Heughebaert E. (1996) ‘Formatting Relational Databases of Practice Forms: ‘Towards a Concept of an Electronic Textbook’, Computer Assisted Language Learning 9 (4): 375-384. Skinner, B.F, (1960) ‘Special Problems in Programming Language Instruction for Teaching Machines’, JAL, XXVI: 167-174. Taylor, R.P. (ed.) (1980), The Computer in the School: Tutor, Tool, Tutee. New York: Teacher's College Press. Troubleyn, K., Heireman, K. & Van Walle, A. (1996) ‘ATLAS: Computerised Second Language Proficiency Testing’, Computer Assisted Language Learning 9 (4): 359-366. Uyttersprot, W., Colpaert, J., Decoo, W. (1990) ‘The GUIDE Principle of User Interfacing in Courseware’, in N. Estes, J. Heene & D. Leclercq (eds.), The Seventh International Conference on Technology and Education. New Pathways to Learning Through Educational Technology. Edinburgh: CEP Consultants, Vol. 1, pp.4-6. Uytersprot, W., Colpaert, J., Decoo, W., Heughebaert, E., Schonenberg, N. (1993) Vocapuces 1.0 voor Windows — software en handleiding. Antwerpen: UIA. Van Walle, A., Heireman, K. & Troubleyn, K. (1996) ‘ARIANE: Computerised Mother Tongue Tests for Belgian Civil Servants’, Computer Assisted Language Learning 9 (4): 367-374.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.62 DAVID BICKERTON From the methodological standpoint, therefore, multimedia in languages can play contrasting pedagogic roles, bringing a variety of cognitive process- es into play. The variable appropriacy of sound and visual media to the acqui- sition of a second language at different learner ages, and the contrasting cognitive activities and strategies to which these media give rise, are areas that warrant significant research effort.© There is, however, some danger in sys- tematising to excess the way in which different pedagogical elements in the learning mix actually interlock. This may be an inheritance from a time when authoring processes did indeed require highly detailed design and implemen- tation. As we shall see, methods inspired by constructivist cognitive princi- ples, and the move from learning to acquisition, have produced a looser, more open view of the application of multimedia materials to language learning. The production processes behind multimedia courses or ‘supportware’ are begin- ning to benefit from the change in language learning methods. Pedagogic objectives, learning mechanisms, no less than the ability and experience of authors, must all be borne in mind when seeking to identify the ‘right tools for the job’. 3. AUTHORING TOOL EVALUATION Whereas techniques for evaluating multimedia courseware rightly focus upon content and the learner (Marton & Harvey 1994), techniques for evaluating multimedia authoring tools (ATs)’ have identified the importance of other cri- teria: * user and leamer ergonomy (i.e., the ease with which author and learner variously interact with either the AT or its product) * _ appropriacy of AT features to didactic objectives * affordability (in terms of human effort, institutional infrastructure and cost/benefit). In 1996, the RAPIDO Project identified 259 products as potential authoring tools and concluded that some 20 to 30 were of significant interest to academ- ic linguists in Europe (Bickerton et al., 1997). Products suitable for MMCALL but indigenous to the North American market were not considered in this European Project.8 However, one of the consequences of Internet-delivered solutions that have appeared recently is that geographical origins are beginning to lose their validity, and reference will be made below to some further prod- ucts with world-wide distribution. The RAPIDO evaluation scheme proposed 211 fields of data for author-related features and 106 for learner-related fea- tures, Evaluators for six leading products were asked to score each feature onaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.66 DAVID BICKERTON Indeed, if one takes a fully professional approach towards the creation of multimedia applications, using languages (or ready-made routines in lan- guages) like Viswal Basic, Java, Delphi (which are not ATs), an immense range of application areas can be addressed; Director and Icon Author, which are generic multimedia development tools, also offer very wide potential for edu- cational application development. It is as though their design as ‘tools for every trade’ confers upon them a complexity which is disproportionate to the narrow purposes of most authors. All these products impose lengthy and steep learning curves, even for authors with programming experience, and are cer- tainly beyond the reach of the great majority of language specialists. One response to the consequential conflict between ambition and compe- tence is to heed the warnings of software developers and media specialists who would have academic authors keep well away and ‘leave it to the pro- fessionals’. Software developers rightly make much of the importance of fol- lowing a proper development cycle for all software applications. As for the professional media people, they wave a quality banner: J Keenan, speaking for production managers, recently went into newsprint to decry ‘multimedi- ocrity’ and ‘awfulware’.!2 Educationalists also tend to produce conceptuali- sations of the development and implementation process for courseware which are dauntingly comprehensive (c.g., Hubbard 1996). One understands the frustration of the commercially-driven professional media man faced with a fragmented educational market, constantly changing curricula and under- financed educational programmes; inevitably, there are few opportunities to mount grandiose schemes with large development teams or win a new BAFTA award for interactive software, or even to make a modest buck or two. But from the perspective of creating multimedia support materials (as opposed to commercial courses), over-specification can prove to be a real red herring for academic linguists. A better response for the academic linguist than throwing in the towel in this way is to accept the consequences of the fact that generic and powerful ATs are harder to usc than those with very selective features. Non-programming courseware authors must consider the alternative, with all its pros and cons, of becoming more targeted and limited in their repertoire, perhaps starting by gi ing a multimedia spin to classical CALL tasks such as MCQ (multiple choice questions) and varieties of gap filling and mixing/jumbling activity. Three major products offer complementary suites of applications: the Wida Authoring Suite (version 1.9 bundles together Storyboard, ‘Testmaster, Gapmaster, Matchmaster, Choicemaster, Vocab and Pinpoint, with associated activities and multimedia attachments); Speaker (version 4.0 permits the cre- ation of dialogues, hypertexts and links, MCQs, text analysis and gap-filling); and the Partner Tools series (the 1998 release combines VideoPartner,aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.CHALLENGE OF MMCALL 69 recently broadcast programmes can be streamed and received at will, and there is a selection of items for ‘listeners’ intent on improving their English (www.bbc.co.uk/worldservice). Thus the technology begins to look as though it will, on this occasion, bring home the bacon. Activity on the Internet seems to have reached the critical mass associated with an unstoppable technology. One factor which is current- ly holding back the full tidal wave of Internet data also looks like being resolved within the lifespan of today’s computers. The capacity of cables to transmit the data required for good functionality is often cited as an insur- mountable obstacle to its success. Fortunately, the telematic industries are responding with data accelerators and also a gradual switch of telephony from cable to air and a transfer of television from air to cable (the so-called ‘Negroponte switch’), The bandwidth bottleneck, which is responsible for user frustration with what can appear slow and unreliable access toa growing num- ber of excellent service sites such as are being developed by many Language Centres, Language Departments and many large organisations in Europe today, could soon be a thing of the past.!7 Whether the financial control imposed by data owners upon data access — somebody, after all has to provide the courseware — allows us to maximise this technological opportunity, is another matter. In a world of free bandwidth — if that is where we are heading — the cost will lie in the data. The main limita- tion imposed upon the way learners receive and interact with learning materi- als, the look of these and the speed and ease of their production, may lie more in the cost of ‘paying as you learn’ than in any design issues surrounding the authoring of MMCALL materials. 6. THE CHALLENGE OF AUTHORING FOR INTERNET DELIVERY It will now be clear that the debate on MMCALL is subtly changing some of its main postulates as authors and users look to the web to provide their sources, outlets and delivery mechanisms. The ‘webolution’, as this paradigm shift has been called, presents a serious challenge to received educational prac- tices, opening the door to more democratic and flexible access to information and learning. The challenge to authoring is equally profound. The idea that, in order to develop MMCALL courseware, one should have years of specialised computer training, somehow seems anachronistic if the means of delivery is to be non-hierarchical, ubiquitous and immediate. One consequence of today’s exponential growth of the Internet is that linguists are not the only budding WWW authors in the world of education. It is as though everyone wants to be able to create multimedia presentations, digitise video sequences, and output10 DAVID BICKERTON it all in HTML. This has given rise to a strong market for excellent CD-ROM- based training packages in some of the applications (like Adobe Premiere) aimed at self-reliant authors.'8 To quote once more the lessons of the RAPI- DO training workshops, it has been shown that training for academic linguists without a serious computing background can be effective provided the author- ing system is appropriate for their skills and their post-training environment is supportive and allows them to practice what they have leamed. A constraint upon authoring for the web lics, naturally, in the novelty of the tools that are available. There are two main aspects to what might be called ‘Internet ATs’. First, there are the ‘languages’, which do not yet have the easy functionality of traditional authoring tools described in Table 1. HTML 4.0 is announced as a successor to HTML 3.0 (the language for ‘marking up’ pages and graphics as hypertexts), and this will enable authors to protect their data from indiscriminate downloading. More importantly, another authoring prod- uct which is being developed holds out the even greater promise of making support materials so easy to generate and deliver as to become truly ephemer- al: SMIL (Synchronised Multimedia Integration Language), the latest develop- ment of the W3C Synchronised Multimedia Working Group, promises to be an easy-to-learn tool for producing ‘cheap, effective and easily updated multi- media courseware, accessible worldwide. Users could view video, follow links, search topics and contact tutors’.!9 Another area being quietly revolutionised by Web delivery is the manage- ment of the learning process itself, and this constitutes a new dimension to Internet authoring. When the RAPIDO survey of products commenced in 1996, computer-mediated interaction between learners and teachers was lim- ited to email, chat (e.g., JRC) and a handful of language laboratory emulation systems which used a significant hardware component (some of those listed in Table |, line 5). Recently, however, not only have high-quality telephony ser- vices made an appearance (e.g., PowWow and NTC Voice®®), and virtual envi- ronments started to develop (Schwienhorst 1997), but Course Management Systems (CMSs: Graziadei 1996) have also appeared with tools for organis- ing text and multimedia files, if not for creating them, and keeping track of the whole learning process. The CMS comes in many guises, In one sense, the language laboratory of yesteryear provided some of the key facilities needed for managing student leaming. This functionality can now be emulated in soft- ware, or more usually with a combination of software and hardware. Put together with the potentialities of the Internet this has the makings of a com- pletely new tool for language course management. There are many products in the CMS category (see the Plattsburgh State University of New York www.plattsburgh.edu/friesep/cms.htm for a CMS review site). Amongst ‘software only’ products one may cite LLI-Net,CHALLENGE OF MMCALL 7 LearningSpace (Lotus), WebCT, VirtualU, and Webcourse_in_a_box. Other products have a software/hardware component; they provide an infrastructure for manipulating leaning materials, screens, and communication, and can link in with course management within a local environment: Robotel (in Europe), CSS (an add-on to Tandberg laboratories), Divace (also from Tandberg, being their new and fully integrated ‘digital recorder’), EDULAC (for the Edu4 lab- oratory, France), Classnet (USA), COMWEB (USA), and AMLS (the Macintosh product also listed in Table 1, line 5). These products typically allow teacher control of student screens, distribution of one screen to others, one-to-one or one-to-many conversations. There are also software products quoted as doing the same sort of thing, only slowly (LANSCHOOL, running under DOS and Windows). Not all of them support full course management, but the market is moving so fast that by the time these words appear in print they well might. Merlin is a recent example of a CMS developed specifically for language learning and which, although not emulating the full interactivity of a labora- tory, does use audio streaming for voice messages and has other features of a web-based CMS.7! Without going to a proprietary system such as this (now commercially available), or to a video conferencing system for language learn- ers such as has been developed by the ReLATe Project — and which is unavail- able commercially and not yet functional on the PC platform (Bucket & Stringer [forthcoming]) — one can see how, in principle, using widely-available conferencing tools such as Microsoft's Net Meeting, or a simple CMS like First Class together with Real Audio, one could create an operational Computing Environment for Language Learning (CELL) over an intranet (for secure bandwidth) or the Internet. Putting it all together and integrating visual communication (using LearnLinc I-Net or simple utilities like CUSeeMe, Timbuktu, Tech Commander) produces the synchronous communication environment familiar to telephone users and to language laboratory teachers extended to vision and underpinned by course management features to track the learning process. Interestingly, the growing simplification of web authoring (as we have seen, most well-established word processors can save files in HTML format), and the existence of management tools (to monitor student access, allow a super- visor to moderate their chat, store their grades and even schedule their sy: chronous communication) means that the whole issue of ‘course authoring’ must now be seen in a new light. Although fully-integrated solutions like IBM’s Personal Learning System have appeared on the market, most academ- ic linguists can expect to use — to varying degrees — some of the simpler and cheaper commercial products quoted above, and Graziadei has highlighted the double challenge this produces to their whole manner of teaching as they moveaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.4 DAVID BICKERTON and in the present age of rapid transition pit-falls for academic linguists do remain. First and foremost is the possible mis-match between the potential authors’ skills and their ambitions. In the absence of careful management, the time taken to bring a project from the idea stage to fruition can be years rather than days. This can be because of the time needed by an ambitious author to climb the learning curve required by some ATs: such is the speed of techno- logical change that some learners never catch their tails. If a user chooses an inappropriate product, the time to become a proficient developer will exceed the technological life cycle of the product, and the investment in time and resources will have been wasted. Another hurdle is copyright freedom. Fortunately, much foreign television material is not subject to UK licensing regulations for re-use in education, and since most users of ATs are not looking for world-wide distribution their use of audio-visual realia is less sensitive than for overtly commercial publishing. One of the revelations of the RAPIDO workshops was the speed and ingenu- ity with which participants downloaded files from the web and from CD- ROMs for integration into their supportware. The ubiquity and ease of access to multimedia objects is such that copyright law stands like King Canute before the electronic waves. Practical and simple mechanisms allowing courseware authors to use existing data are fundamental to the success of the ‘Information Society’, and consequently lie at the core of yet another Framework V key objective: Building, populating and maintaining electronic libraries of multi- media educational material [...] may be reached by providing non technically-oriented authors with intelligent authoring tools to pro- duce high quality education material. Or it may be achieved by improving its accessibility and reusability. At the same time, upgrading current laws regarding intellectual property is another objective. (Weets 1979, 14) It is tempting, in the absence of all necessary conditions for success being present, to retreat into the safe world of permanence and completion held up by the book and traditional media. But as many now acknowledge, this is not the leading paradigm for 21st-century man. Continuity and security must come from programs and data sets which can be used anywhere and anyhow. Transience and change must lie in the data selection, their reformulation and their delivery to the learner. Current developments in MMCALL challenge all academic linguists to question the assumptions behind the way they commu- nicate and teach, to re-assess the way in which the materials they author or select really contribute towards learning, and to place simplicity, speed of pro- duction and renewability at the top of their agenda.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aE) DAVID BICKERTON Ginet, A. (ed.) (1997) Du Laboratoire de langues @ la salle de cours multi-médias, Paris, Nathan. Graziadei, W.D. et al. (1996) ‘Building Asynchronous and Synchronous Teaching-Leaming Environments: Exploring a Course/Classroom Management System Solution’. Available: hutp://washington.uwe.edu/docs/00005/00598.htm, Jan, 199, Hartley, T.A. (1995) The Psychology of Language. From Data to Theory. Hove: Erlbaum. Hubbard, P. (1996) ‘Elements of CALL methodology: development, evaluation and implemen- tation’, in M.C. Pennington (ed.) The Power of CALL, Houston Texas: Athelstan, pp.15~32. Ingraham, B. (1995) ‘Multimedia and Higher Education: some lessons from France InterActive’, in C. O'Hagan (ed.), Empowering Teachers and Learners through Technology, SEDA Paper 90, Staff and Educational Development Association, 9p. Knussen, C., Tanner, GR. & Kibby, MR. (1991) ‘An Approach to the Evaluation of Hypermedia’, Computers and Education \7 (1): 13-24. Landon, B. (1998) Douglas College, Canada, www.ctt.bc.ca/landonline/, available Jan. 1999. Levy, M. (1994) ‘CALL materials development survey report’, Computer Assisted Language Learning 7 (2): 175-189. Locatis, C., Ullmer, E. & Carr, V, (1991) ‘Authoring Systems: An Introduction and Assessment’, Journal of Computing in Higher Education 3: 23-35. Locatis, C., Ullmer, E. & Carr, V. (1991) ‘Authoring Systems Re-assessed’, in Educational Technology, Research and Development 40: 77-82. MacKnight, C.B. & Balagopalan, S, (1989) ‘An Evaluation Tool for Measuring Authoring System Performance’, Communications of the ACM 32: 1231-1236. MacKaight, C., Dillon, A., & Richardson, J. (1993) Hypertext, a Psychological Perspective. Chichester: Ellis Horwood. Marton, P. & Harvey, D. (1994) ‘L’Evaluation des systémes d’apprentissage multimédia inter- actif”. http://www. fse.ulaval.ca/fac/ten/reveduc/html/volll/no3/evalsam.html, available Jan. 1999. ORAVEP (1995) Ressources multimédia pour V'apprentissage des langues. Paris. Park, O.-C. & Seidel, R.J. (1989) ‘Evaluation Criteria for Selecting a CBI Authoring System’, The Journal: Technological Horizons in Education 17: 61-68. Riley, F (1998) Understanding IT: Developing Multimedia Courseware. http:/Awww-hull.ac.ak/itti/nullprod.htmlAfr, available Jan. 1999, Schwienhorst, K. (1997) ‘MUDS and MOOS’, in D. Little & B. Voss (eds) Language Centres: Planning for the New Millennium, Papers from the 4th CERCLES conference, Plymouth: CercleS, pp.126~44. Stenton, T. (1998) “Hypermedia: A New Consensus for the 1990s", in Keith Cameron (ed.) Multimedia CALL: Theory and Practice. Exeter: Elm Bank Publications, pp.1 1-16. Sussex, R.D. (1991) ‘Author languages, authoring systems and their relationship to the chang- ing focus of Computer-Aided Language Leaning’, System 19 (1): 15-27. Weets, G. (ed.) (1997) V Framework Programme. Technologies for Knowledge and Skills Acquisition. Proposal for a Research Agenda, October 1997. http://www2.echo.lu/telem- aticsfeducation/en/interact/bul_Sth2.himl, available Jan. 1999.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.84 MIKE LEVY purely as a research testbed for a particular theory of language acquisition — but not say so. Or a project may have been designed for individual student use in self-access mode without a teacher present, but these intentions are not made clear. The design space along with associated assumptions and limita- tions need to be explicitly stated. My goal in this chapter, then, is to identify recurring contexts for CALL work, and to make more explicit some of the assumptions that have remained largely in the background in the design of CALL programs and leaming envi- ronments, In attempting to accomplish this goal, particular emphasis is given to ways in which the design space might be defined or circumscribed initial- ly. Then levels of conceptualisation are discussed because of their value in managing the many diverse elements that need to be taken into account in the process of design. Looking at design as a hierarchy of levels is especially help- ful in clarifying the role of theory in design. The goal throughout is to help authors define their field of view, and to further assist by pointing out the ram- ifications and assumptions that are implicit in the approach that is adopted. At the same time — as with sets of principles and guidelines — the aim is to pro- vide techniques and approaches that have practical value. At the outset, I would like to point out three features that impact heavily on design in CALL. Firstly, the designer needs to make every attempt to get to know the potential users and the learning context. To quote Shneiderman, “before beginning a design, the characteristics of users and the situation must be precise and complete.’ (1987, p.52). Further, Pederson notes that research clearly indicates that CALL is ‘highly context-bound’ (1988, p.126). Without a clear understanding of the learner and context-related attributes, CALL. designs will founder. Secondly, design is fundamentally a creative process, one which often involves the discovery of new goals (see Shneiderman 1987, p.391). Unless the development process is intentionally constrained as in a controlled experimental study for example, what is initially conceived may undergo significant, even radical change as a result of the development process and feedback from users. Thirdly, the hardware and the software development tools that are employed exert a wide-ranging influence on design. All such tools have their strengths and limitations, especially in the interaction options that they offer, and these attributes variously shape and direct the design. A recurring theme in the chapter will be the importance of attempting to recognise ina very specific way design elements that work in a CALL program so that they may be used again. As Bailin (1990, p.173) and Hubbard (1992, p.40) suggest, ‘strategies’ and ‘tactics’ that work in one program need to be recognised and used as ‘building blocks’ in others. Though many CALL pro- grams and learning environments have now been created, the extent to whichaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.88 MIKE LEVY when the computer is required to evaluate user input in some way, especially in less structured conditions in more open domains. Thus, while the comput- er can present digitised sound effectively, it cannot evaluate sound input very well. So the capabilities of the computer contrive to allow certain kinds of CALL work to be contemplated. The holistic—discrete element distinction is not a hard and fast dichotomy. Recent work by Poyatas-Matas (1997) demonstrates a learning environment that is in some ways holistic, and in other ways rather specific. She has designed a Website, ‘Gramalandia para Gramadictos’ (Grammarland for Grammaddicts), for more advanced students who want to lea and practise Spanish grammar in a communicative setting. The elements of the Website are as follows: 1. a concordancer and dictionaries 2. grammar explanations 3. grammar exercises 4. anoticeboard on grammar problems 5. a section on exams 6. anoticeboard containing grammatical ti 7. anoticeboard on grammar research projects 8. achat section on grammatical issues 9. ind advice from teachers . a noticeboard for teacher's contributions 10. an information section 11. a searching tool 12. a visitor’s counter Recent Web technologies provide dynamic interactive learning environ- ments that can be shaped by users. Nevertheless, in this instance a decision was made by the designer to focus on grammar, and then to make full use of the capabilities of the environment to further its exploration and study. 2.2 Tutor and tool roles The basis for the distinction between the tutor and the tool is that the tutor evaluates learner input at the computer, whereas the tool does not (see Taylor 1980). This distinction is a significant one, and comes close to a genuine dichotomy because the roots of the distinction lie in the difference between human and machine monitoring and evaluation. The tutor and the tool roles for the computer reflect fundamental differences among the goals and aspirations of CALL researchers and practitioners. Thus, in ICALL, for example, the goals concern the creation of tutors that can address and respond to individual learner needs and learning characteristics (see Holland et al., 1995). In con- trast, in Barson and Debski’s GLEn the intention is to maintain the primary function of computers as ‘enabling’ devices that are available to enhance andaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.92 MIKE LEVY (Shneiderman 1987, p.55). In circumstances where specific user details are vague or unknown this option undoubtedly becomes more attractive. The audi- ence remains remote to the author, at least in any detailed or specific sense, and generalised predictions have to be made; arguably, this is easier at beginner level than any other. There is no doubt that the better one knows the needs and characteristics of the user, the better one can make the design. It is here that ‘Web-based publishing has such potential given the options for learner input and feedback on design. 3. LEVELS OF CONCEPTUALISATION AND THE IMPORTANCE OF THEORY By working through the broad distinctions described thus far, and by reflect- ing carefully on the goals and needs of learners and the learning context, the design space will gradually become clearer and more focussed. The question remains, however, of where exactly to begin when attempting to transform these broad design parameters into an actual CALL program or learning envi- ronment. Numerous elements will impinge on the design. They are likely to range from those that are more abstract, such as beliefs about language and language learning, through to those that are more tangible such as the exist- ing curriculum, learner characteristics and the technological resources avail- able (see Hubbard 1992; Ingraham & Emery 1991). All must somehow be managed and taken into account in the design process. One way of managing these rather disparate elements is to look at the process of design in terms of a hierarchy of interacting levels. In this section, ways of conceptualising CALL along these lines are explored. These frameworks will also be valuable in helping us to think more clearly about the relationship between theory and design. In conceptualising Computer-Assisted Leaming (CAL) and CALL systems, hierarchical three-level approaches are quite common. Shneiderman (1987) and Hedberg (1994) describe such hierarchies for CAL; Richards and Rodgers provide a three-level system for describing language teaching methods, which Hubbard (1992) uses as the basis for a methodological framework for CALL (see Table 1). Richards and Rodgers argue that such constructs have consid- erable value for highlighting ‘different degrees of abstraction and specificity” in language teaching (1986, p.15), and one might extrapolate for CAL and CALL. They can help capture important stages in the transition that must occur between theory and actual design practice, and they provide a mecha- nism for purposeful reflection when multiple, interdependent factors that might first appear distant and unconnected are involved.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.96 MIKE LEVY It is important to appreciate the need for theoretical pluralism in prescrip- tive artefact design. In this regard, Boyle notes that presentation design has its own problem space, that it is quite different from conceptual design, and requires its own theories and rationale (Boyle, 1997, pp.2, 121). Different facets of a fully realised CALL program may well draw on a number of com- plementary theoretical bases. For example, in describing a multimedia design framework for the learning of culture and language discourse, the theoretical base was drawn from a number of sources (Levy, 1997b). Specific theoretical bases used include Schmidt's work on ‘noticing’ (1990, 1993), Cumming’s work on levels of learning and transfer (1993) and McCarthy's work on lan- guage as discourse (1991). The second component of the Theory-To-Design framework states that the- ory elements are linked directly to specific design features. This direct link is crucial and enables incremental testing, theoretically motivated, of particular design elements. Proceeding in this essentially modular way allows specific design features to be properly tested and validated. An overt and direct link between theoretical elements and specific design features provides us with a mechanism for evaluating effectiveness that moves beyond broad-based eval- uative studies; this applies for both theory construction and testing and pre- scriptive artefact design. A good example of a study testing a specific design feature is Borras and Lafayette’s work on subtitling (1994). It considers the effects of multimedia courseware subtitling on the speaking performance of students learning French in conditions where students have the opportunity to see and control the sub- titles. Opinions on subtitling have been contradictory. Opponents argue that subtitles are distracting and slow down the development of students’ listening ability, while supporters believe subtitles improve language proficiency by making students ‘conscious of language they might not otherwise understand” (p.61). In a carefully controlled experimental study, Borrds and Lafayette investigate this issue. With a narrow focus on a specific design feature, it reviews the related research, debates research methodologies and data collec- tion techniques, and discusses implications and recommendations. The results provide convincing support for subtitling under certain conditions and these are explained. In this study, it is the detailed assessment of a specific design element that is potentially so helpful. Those contemplating the addition of sub- titles in a multimedia CALL program, would find in such work the informa- tion they would require to make an informed design decision. Jacobson’s ‘design features’ are reminiscent of Salomon’s ‘coding ele- ments’. Salomon provides a strong foundation for understanding the nature and function of these elements in design (Salomon, 1977, 1979a, 1979b). There is not sufficient space to go into detail here, but briefly Salomon definesaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.100 MIKE LEVY Finally, I would also agree with Bailey who argues for the importance of designing research studies that will contribute to the theory-building process and planning research that tests particular theories (Bailey p.89). With these thoughts in mind and in recognising the current state of CALL, I believe research would be desirable in the following areas: . testing effectiveness of specific theory-linked design features or coding elements properly related to task type and learner difference * investigating learner differences in specific CALL contexts using a vari- ety of data collection devices * planning research to test particular theories of learning or cognition, within limited and clearly-defined contexts * using research to inform and develop a taxonomy of tasks for tool-based uses of the computer * investigating teacher attitudes to CALL, especially for teachers choos- ing not to use CALL at present. 7. CONCLUSION Thus far in CALL, though many projects and learning environments have been completed, relatively litle hard information has been accumulated for the potential CALL designer. In addition, existing programs often illustrate sig- nificant gaps between the theoretical orientation — when it has been given at all - and the actual design of the program. In order to address this shortcom- ing, finer-level detail is needed so that designers might be able to construct specific design features with confidence, and know exactly when they might be most appropriately used. Also it is essential that the results of experiments that aim to develop our understanding of theory and design reach those fash- ioning programs for use in real educational settings. According to Ros i Solé, at least, widely held theoretical beliefs and results from basic CALL research are not finding their way into commercial programs. To sum up, in this chapter I have suggested a number of heuristics for design. These may be seen to complement sets of principles and guidelines that are already available. The heuristics may be summarised as follows: Know your audience Clarify the project goals and the design space Know the strengths & limitations of existing CALL materials Review possible ways of approaching complex, multi-faceted design problems, e.g., levels of conceptualisation 5. Choose your theory base(s) Peesaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.104 MIKE LEVY Ingraham, B.D. & Emery, CR. (1991) * “France Interactive”: A Hypermedia Approach to Language Training’ Educational and Training Technology International 25 (4) 321-333, Jacobson, M. (1994) ‘Issues in Hypertext and Hypermedia Research: Toward a Framework for Linking Theory-to-Design’, Journal of Educational Multimedia and Hypermedia 3 (2): 141-154, Jonassen, D.H. (1988) (ed.), Instructional Designs for Microcomputer Courseware. Hillsdale, Lawrence Erlbaum, Jones, CM. & McNally. M.M. (1996) ‘A Digital Audio Database for Foreign Language Study’, Computer Assisted Language Learning 9 (2): 235-250, Keith, C.J. & Lafford, P.A. (1989) ‘Designing Software for Vocational Language Programs: An Overview of the Development Process’, in M.C. Pennington (ed.), Teaching Languages with Computers: The State of the Art. La Jolla, CA: Athelstan, pp.123—144. Kelm, O. (1996) “The Application of Computer Networking in Foreign Language Education Focusing on Principles of Second Language Acquisition’, in M. Warschauer (ed.), Telecollaboration in Foreign Language Learning. Hawaii: Second Language Teaching & Curriculum Centre. Kemmis, S.. Atkin, R., & Wright, B. (1977) How Do Students Learn? Working papers on com- puter-assisted language leaming, UNCAL Evaluation Studies, Norwich: University of East Anglia, Kemp, J. (1985) The Instructional Design Process. New York: Harper and Row. Knowles, S. (1992) “Evalustion of CALL Software: A Checklist of Criteria for Evaluation’, On- CALL 6 (2): 9-20. Kozma, R.B. (1991) ‘Learning with Media’, Review of Educational Research 61: 179-211. Kozma, R.B. (1994) “Will Media Influence Learning? Retraming the Debate’, Educational Technology Research and Development 42: 7-19. Krathwohl, D.R. (1993) Methods of Educational and Social Science Research: An Integrated Approach. New York: Longman. Kuhn, T. (1970) The Structure of Scientific Revolutions. Chicago: The University of Chicago Press. Land, S. & Hannafin, M. (1996) “A Conceptual Framework for the Development of Theories- in-Action with Open-Ended Learning Environments’, Educational Technology Research and Development 44 (3): 37-53. Landauer, TK. (1991) ‘Let's Get Real: A Position Paper on the Role of Cognitive Psychology in the Design of Humanly Useful and Usable Systems’, in J.M. Carroll (ed.), Designing Interaction: Psychology at the Human-Computer Interface. Cambridge: Cambridge University Press, pp.60-73. Laurillard, D. (1987) (ed.), Interactive Media: Working Methods and Practical Applications. Chichester: Horwood. Lawrason, R. (1988-89) ‘Language Lab Technology and the Instructional Design Process: Fashionable Fad or Pedagogical Tool’, Journal of Educational Techniques and Technologies 21 (4): 31-38 Levy, M. (1997a) Computer-Assisted Language Learning: Context and Contextualization, Oxford: Oxford University Press. Levy, M. (1997b) ‘A Design Framework for Multimedia Learning of Culture and Language Discourse for Overseas Non-Native Speakers of English’, paper presented at the Exeter CALL "97 Conference, 21-23 September, 1997, Exeter, UK. Levy, M. & Farrugia, D. (1988) Computers and Language Teaching: Analysis, Research and Reviews. Melbourne: TAFE Victoria. Liou, H-C. (1994) ‘Practical Considerations for Multimedia Courseware Development: an EFL IVD Experience’, CALICO Journal 11 (3): 47-74. Lynch, P. & Horton, S. (1997) Yale C/AIM Web Style Guide. http://info.med.yale.edu/cainy ‘manual/contents htmlaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.Evaluating Performance, Approach and Outcome Robin Goodfellow Institute of Educational Technology, Open University 1. SUMMARY This chapter deals with an approach to the evaluation of CALL for open learn- ing. It looks firstly at the principles of a phenomenographic approach to the design of computer-based learning, and then at the empirical methods that can be used to evaluate its outcome. Using a vocabulary-leaming program as a case study I discuss the interpretation of indications of learner performance and learning approach and show how a better understanding of the way the leamer conceptualises the interaction can lead to a more appropriate assess- ment of its outcome. The conclusion addresses the importance of the learning context and the possible impact on CALL design of the increasingly gener- alised use of communications and information technology in education. 2. INTRODUCTION - WHAT’S GOING ON IN THERE? When language learners use technology they usually have some idea in mind of what they are doing it for. Once, whilst working at a British university, I was intrigued to see a student in the language lab listening on headphones to a videotape of the news in French whilst typing energetically on a computer key- board. It turned out he was having an email conversation in English with aaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME 113 likely to reside in the richness and complexity (the depth) of the conscious reflective processes which they develop in the course of study, as it is in the competencies they acquire. Candlin and Bymes (1994 quoting from Hodgson et al., 1987) argue that, under the rubric of open learning as development, learning materials presuppose learners who are confident and skilled ‘in mon- itoring their own learning’. CALL evaluation methodologies deriving from this view, focus on investigating the learner’s experience of the interaction, drawing to the surface their assumptions about, and dispositions towards, lan- guage learning in general, and their perceptions of the ‘actions’ and ‘events’ that happen via the interface. These ‘subjective’ dimensions of the interaction can be considered in the context of the computer's ‘objective’ record of the learner's performance, to arrive at an assessment of the interaction’s outcome. 4, A PHENOMENOGRAPHIC APPROACH Phenomenographic means describing the way a learner experiences their learning. It is an approach to both design and evaluation, because it assumes that the outcome of learning and the processes by which learning happens, are the same thing. It is also essentially humanistic because it takes the learn- er themself (and not some independently-assessed knowledge or skill that they may exhibit) as the reference point for the assessment of outcome. It is an approach which has not been widely used in language learning up to now, perhaps because, as I have suggested, we have been dominated by mecha- nistic information-processing models of learning. As an approach to evalu- ation it implies a separation of the concepts of performance, and learning approach. ‘We can summarise the kind of data available which a CALL interaction might make available for phenomenographic study as follows: (i) Performance data (how the learner does with regard to the system’s tasks), understood quantitatively. This data is provided by the system’s ‘scoring’ functions. (ii) Performance data understood qualitatively. For example, evidence of specific strategies being applied. Data is provided by log files and by observation. (ii) Learning approach data, understood qualitatively. That is, evidence about the assumptions the learner makes regarding what they are doing and why. Data is provided by the learner talking aloud before, during, and after the interaction. (iv) Introspective data, understood quantitatively. Data is provided by the learner, assessing what they have learned, after the interaction.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME 117 Table 1. The Effects of Spreading Activation on Performance. Subject Output Target Dell’s Error Types Subjects Comment si ‘descrimetu _—descubrimiento. phoneme errors “I'm confused between ‘descrimuto (discovery) ful shift discovery and *descrimutos: Jen/ shift described. +descrimentos def & inf blend etc, “descritos is blanking, out the other...” s2 *Namuras Manuras (plains) phoneme errors. “I’m guessing now...” “llamudas ‘nv /nf substitution Hamaras Jd] frf substitution “*Hamuros: 83 *arregran arriesgan (they phoneme errors “its similar to gar or arreglan risk) fi/ deletion something. *arresgicron Ail shift arr. s4 *incideros colillas (cigarette word errors “cigarette ends . ait butts) non-contextual something like th rato substitution “that means com- plete...” 85 *pertubador *perturbado morpheme errors “there's no 2nd r” /dor/ lado! “I remember writing it wrong before..." S86 *apistonamente apisonamiento —_phonemeerrors. (Comment at notes *apistonamiento Al addition writing time) “steam roller is not right but it is so unusu- al it will remind me.” s7 elaborar desprende word error “T just don’t know...” non-contextual substitution $8 *alludidor aludido phoneme errors *aludor Ail shift *alludor morpheme errors *aluidor dor! fido *alud substitution could also be used to apply principles of recency and expanded rehearsal to the testing. There are a number of reservations about this kind of theorising. Modelling learning as seen through this psychology perspective is an attempt to repro- duce natural learning, But the open language learning context is not one in which a natural approach is appropriate. The learner is engaged inaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME 121 ea FORM ‘CONTEXT. x arate fre Sececetes lees = cre espn.’ WM Bittbod elisha lg a phrovee a 14 aie eee Pewee Mark the Cunent Mask the Cument et see cee. fo Ma sce saeae eee ies coreg \CONCORDANCE | DICTIONARY a ee TREE] Fig. 2. Module 2. Looking up, making notes, forming groups. eens ee SCE get ten 7 tem No. 7 (Text Name: afeat!) [modemas a un grave problema» —-~——. «por otra parte -sigue-. la] GROUP: MEANING phrases [ de hoy en dia. ] NOTES: ...the study adds. WoRneHePeSe TATE 1 ELLA GRAMMAR: FEEDBACK [SOME TARGET WORD ELEMENTS CORRECT age. Fig. 3. Module 3. Self-testing.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME 125 mode precisely until they had scored 100% retrieval of their current selections, whereas others, including $12, often went on to select and investigate new items before they had correctly retrieved all their current ones. Table 3 shows that 4 of these same 100 percenters top the chart for the lowest amount of gen- eral processing per item correctly retrieved (that is, the average number of “click events’ they initiated in going from selection to retrieval of an item). The whole table, in fact, shows a significant correlation between low general processing and high correct retrievals, which suggest that there may be a delib- erate strategy of ‘don’t go on till you’ve got it right’ in operation here. Such a strategy is not explicitly emphasised by the program design and is thus not explicit in the data. However, it is clear that time spent retrieving items, even if the retrieval is very effective in terms of overall processing, has to be balanced against the rate of selection of new items if this effectiveness is to be reflected in the overall learning rate. If learners adopt strategies aimed at maximising their retrieval score (such as not leaving a test session until they have achieved 100%), and take too long doing this, they are not contributing to the improvement of their learning rate. Nor on the other hand, are they improving the rate if they prioritise the selection of new items too strongly over the successful retrieval of current ones, Maximising learning rate and maximising correct retrieval appear to be 2 distinct strategies which the design supports, and which need to be operated complementarily to maximise per- formance. On this analysis of performance, $14’s outcome could be consid- ered as successful as $12's. Table 3. General Processing Compared to Correct Retrieval. Subject Events per item retrieved _% selected items retrieved (rank order) (rank order) $10 i 100 SB 12.7 100 S14 14 100 S17 16.6 100 s9 208 85 S12 2B 81 Sil 23.6 1 S18 25.5 29 S15 29 81 S20 319 100 S16 33.6 15 si9 56.5 65aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME 129 structure in the learning task and learning content (Ramsden op. cit., Laurillard op. cit.). Thus, in the vocabulary-learning context, a deep approach would be one in which the learner consciously addressed the role of textual context in giving meaning to a word, and the system of lexical relations which makes it appropriate in that context. In both cases the focus is on aspects of the struc- ture of the target language, not simply on the denotation of the target word. A surface approach is one which focuses on features of the task rather than its structure, in this case on the meaning of the L2 target-item in isolation, as an L] translation, for example, or on means of memorising rather than under- standing it. A deep approach looks for underlying principles connecting the task features into a system, the aim of the program design is to help the learn- er to develop a deeper level of awareness of what is involved in vocabulary learning, by focusing on processes which involve the use of structural lexical features to aid retrieval. In evaluating the quality of the learners’ approach, a subjective assessment has to be made on the basis of what they do and what they say before, during, and after the interaction. As in the case of the assessments of the groups, these judgements can be given an element of objectivity by asking independent observers to compare assessments based on clearly agreed criteria. This was the methodology adopted for this study. The data which was investigated, and the criteria which were applied, included: (i) General approach Data: Subjects’ comments on how they view the general problems of vocabu- lary-learning. Surface criteria: The Subject views vocabulary learning as mainly to do with memorisation of individual word meanings. They refer to lists, to repetition, and the need to associate L2 items with their L1 equivalents. Deep criteria: The Subject views vocabulary learning as mainly to do with understanding the context that L2 items are encountered in, and/or the rela- tions of meaning that exist between these items. They refer to context, to gist, to the relation of L2 items to other L2 items, and to issues of lexical structure and register. (ii) Selecting items Data: this is in the form of comments made, or answers to questions given, during processing in the selection module related to inferring meaning and cat- egorisation of items in the source text. Surface criteria: The subject is mainly concerned with finding an L1 transla- tion for the target item itself, from memory or from literal translation without taking sense and/or grammatical structure into account, or else tries to relateaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME. 133 in the approach of most subjects, and the design appears to support successful performance aspects from both perspectives, which suggests that it is to some extent adaptable to the individual learning styles of users. A link between depth and strategy in the leaming approach has been revealed through the phenomeno- graphic evaluation methodology, and it may be expected that the same method- ology, involving learners in refiecting on their perceptions and strategies, might be instrumental in helping to develop these higher quality aspects of approach and performance. In this sense the evaluation itself may contribute to the eventu- al outcome. 8. EVALUATION — OUTCOME The evaluation question concerning the overall outcome from this interaction has already been partially answered through the consideration of performance and approach. From a phenomenographic perspective, quality in the approach is synonymous with quality in the outcome. If the design appears able to sup- port deep learning approaches and the interaction generates observable per- formance that the learner can reflect on and apply criteria of success to, then the outcome is more or less guaranteed. Table 6 relates the different aspects of performance and approach that we have considered so far, for each Subject. Subjects are ranked according to an aggregate of the 4 measures, which demonstrates that, in general and allowing for the small size of the sample, there is a consistent relation between learn- ing rates which are average or higher, correct retrieval percentages which are average or higher, and deep or mainly deep learning approaches. Table 6. Learning Outcome. Subject Learning Percent Vocab general Approach to rate (iph) correct approach selection and notes S14 (SFL) 73 100 Deep Deep $12(SFL) 8.7 81 Mainly Deep Mainly Deep $10 (SFL) 58 100 Deep Mainly Deep $20 (ESL) 46 100 Deep Deep S17 (ESL) 48 100 Mainly Deep Deep $13 (SFL) 6 100 Deep Mainly Surface $11 (SFL) 58 n Surface Mainly Deep S15 (ESL) 38 81 Surface Deep S19 (ESL) 35 65 Mainly Surface Mainly Deep S16 (ESL) 25 15 ‘Surface ‘Surface $18 (ESL) 15 29 Mainly Surface Surfaceaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.EVALUATING PERFORMANCE, APPROACH, OUTCOME 137 ment that J then tried to develop, illustrated with examples from my own expe- rience as a designer and researcher, was that these ideas are the ultimate repos- itory of any educational value that the activity has for them. Thus I conclude that I, as a designer and teacher, have a responsibility to facilitate the develop- ment of these ideas into fully considered reflections on language and learning. Evaluation of a CALL design, I therefore claim, should focus on the learner's experience of the interaction and their interpretation of its purpose. In the final analysis the design is judged by the insightfulness it is able to generate in the learner's approach to their own learning. Feeding back our understanding of the phenomenon of the learners experi- encing into the design cycle inevitably requires us to take more of the context of the interaction into account, starting with the learners’ approach to language learning in general, and the strategies that they adopt in making sense of the program for themselves. Incorporating these contextual factors into the design has the effect of increasing the scope of the interaction that the learner expe- riences. Thus we have a design cycle which is ever-widening, constantly mov- ing towards the complete integration of the computer into the leaming environment, as a means through with learning is mediated. This leads us towards a merging of the natural and mediated. For me, this approach is not only central to any non-mechanistic conception of CALL as we currently know it, but it is also highly appropriate to CALL as T expect it to become, in the age of telematics. The same computers for which we are currently designing stand-alone language-learning interactions are already being linked to each other via networks such as the world wide web. As this happens, the facilities of computer-mediated-communication (CMC) are added to the interactive multi-media functionality that we already deploy. In one study that I carried out at the same university in which I met the stu- dent in the introduction, I found that over a period of a term a group of lan- guage students made far more use of the general-purpose computer applications in their lab than they did of the CALL programs, including some that were being specifically designed for them. They even made more use of software games such as solitaire! (Goodfellow, 1997). In these general-pur- pose software environments, the introspective protocols which we have been using in the evaluation described here will have the opportunity to become reflective conversations with remote teachers and other learners. In doing so they will cease to be merely techniques of evaluation and become learning approaches in their own right. The CALL design process will become inte- grated into the development of communications and information technology environments for social and cross-cultural collaborative learning projects. Language learning will thus become computer-integrated not merely computer-assisted.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.Computers as Tools for Sociocollaborative Language Learning Carla Meskill University at Albany, State University of New York 1, INTRODUCTION Our brief history of using computers in second and foreign language instruc- tion has taught us that machines, in and of themselves, do little to enact learn- ing. Yet, the perceptions of efficiency and expediency that accompany things technological persist and in part account for our captivation with associating things technological with more productive learning. However, just as putting a child in front of a book does not guarantee mastery of its content, mere proximity to vast, colorful information on our new machines in no way insures cognitive engagement, let alone the needed practice with and conse- quent development of the language and literacy skills that render such engagement productive. What this period of experimentation with computers in teaching and learning does indicate is that the power of the medium lies in how well it gets used and integrated into the daily classroom scheme so that active engagement in acquisition-oriented work takes place; that is, through integrated, sociocollaborative processes that value active thinking and action with language being both tool and target of the activity. This view of teach- ing and learning is in keeping with current beliefs concerning the value of leamer-centered pedagogies as opposed to teacher-, or in this case, machine- centered approaches. Casting the machine in the subservient role of tool in the service of the larger goals and contexts of instructional communities reflectsaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.COMPUTERS AS TOOLS 145 for supporting and complementing sociocollaborative language learn- 4. A TASK ANATOMY FOR SOCIOCOLLABORATIVE LANGUAGE LEARNING Sociocollaborative language learning tasks can be characterized as what Cohen (1994) terms ‘multiple ability tasks’ (p.68); that is, tasks that drive con- ceptual work rather than simple routines. She outlines the qualities of the mul- tiple ability task as follows: + Has more than one answer or more than one way to solve the problem. + Is intrinsically interesting and rewarding. + Allows different students to make different contributions. + Uses multimedia. + Involves sight, sound, and touch. + Requires a variety of skills and behaviors. + Also requires reading and writing. + Ischallenging. (Cohen, 1994, p.68) In addition to driving conceptual work, Cohen’s multiple ability task fea- tures align with active, participatory, meaning-centered tasks that language instructors strive to design and orchestrate. The valuing of various perspec- tives - more than one way of seeing and solving a problem — and differing sorts of contributions on the part of learners are particularly relevant for het- erogeneous language classrooms representing a range of cultures and social/educational strata. The intrinsic interest, challenge, and reward of Cohen's tasks are characteristic of both Krashen’s criteria for comprehensible input (Krashen, 1985) and the long acknowledged importance of motivation as a critical component in the language learning process (Crookes & Schmidt, 1991). Finally, the engagement of multiple modalities (site, sound, tacitile, aural) is also viewed as a highly positive contributing factors for the language learning process (see, for example Meskill, 1997). To collaborate means that learners, more capable peers, and instructors work together toward shared goals. For sociocollaborative language learning tasks, the process of doing so is considered the primary locus of learning the language under study given a well integrated focus on the forms and functions of language throughout. The following anatomy for such tasks that make use of computers has evolved primarily from the close study of exemplary uses of technologies with learners of English as a second language in U.S. publicaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.150 CARLA MESKILL Functions: Suggesting Locating Directing Offering A built-in awareness of an inherent syntactic form such as singular/plural forms for nouns, will round out the acti Syntax: singular/plural nouns demonstrative pronouns prepositions of place imperatives Let’s 5.2 Workbook: challenging oneself and others Traditionally heralded as the best possible application of technology to lan- guage learning, the workbook (drill and practice) form of computer assisted language learning remains prevalent among publisher offerings. The work- book continues to be a common format, although drills are now frequently dis- guised in impressive graphics and entertainment-oriented functionality. Learners have been quick to see beyond the frills and recognize that what they are doing differs little if at all from off-line workbook drills where a single answer to a multiple choice, cloze, labeling, or combining activity will be assessed, not any thinking, activity or creativity on the part of the learner, but simple algorithms. Indeed, a recent survey indicates that language teachers need to resort to coercive measures to get learners to work at even the most colorful, ‘interactive’ workbook exercises for more than a single, short period of time (Meskill & Mossop, 1997), For sociocollaborative purposes, however, learners can capitalize on the workbook format to jointly develop questions and cloze exercises with which they can challenge one another. Workbook-style software is widely available in all disciplines. As a tool for sociocollaborative, acquisition-oriented tasking, workbook format software that deals with lesser known or unknown disci- pline-specific language (¢.g., physiology, an historical period, math word prob- lems) can be utilized to great benefit. Because the language (though quite probably not all the concepts) may be new, learners can be assigned tasks which require them to negotiate content for several purposes: to exercise deductive meaning making with new vocabulary items; to locate, extract and share unfamiliar items; to help those unfamiliar with concepts attached to new vocabulary to learn these; to help those familiar with concepts, but not neces- sarily with the corresponding terminology in the target language to articulate:aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.154 CARLA MESKILL elegantly simple example of this kind of task is for learners who are studying the daily customs of a non-local culture and need to make electronic queries to the non-local group. Eliciting the needed information electronically requires collaborative planning (What are the appropriate questions? What is the appro- priate form with which to ask them?) and synthesis (How is information best presented to the rest of the learning community? Comments and suggestions elicited from the learning community?) 6. THE ROLE OF THE MACHINE, As is the case in any physcial context where computers are used, in language learning contexts, machines are complex artifacts in the sense of Goodwin and Goodwin (1995); that is, like all contextual artifacts, especially tools used by members of a particular community, the presence of the machine inherently brings about major change in the structure and dynamics of discourse and activity. For language teaching and leaming, computers represent unique cli- mates and contexts to promote student-centered activity when the machine is cast in its proper role as fool in learner-centered pedagogy. The sociophysical space in which the computer resides in an instructional setting — be it in the back, front, side, center, or closet of the classroom or in a laboratory — speaks a great deal about how the machine and its role get perceived and ultimately used. The computer can consequently get cast as integral or marginal to the classroom society. If, for example, the machine is relegated to a ‘private space’ ~ a comer far from central activity — the subsequent activity with and around it will, for the most part, be perceived within the learning community as ‘pri- vate’.! If the computer is centralized both physically and socially, then activ- ity related to it will be perceived as shared and central to the social/academic workings of the class as a whole. There may be times when private work by individual students may be desirable: the risk is in overusing the machine as a babysitter, tutor, reward, or punishment. Machines, especially when there are several in a laboratory, can be per- ceived and used as a way to occupy students so the teacher can attend to other matters. Indeed, this notion of ‘freeing the teacher’ has been a powerful force in educational software marketing. With students productively engaged with the machine, the thinking goes, the teacher can devote time and attention to individual learners. This is certainly a potential plus for language learning con- texts where one-on-one communication with a native speaker is highly desir- able. However, the teacher can be just as freed to be a co-participant if she has carefully constructed authentic tasks that require active participation and col- laboration, rather than individuation, with machines.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.156 CARLA MESKILL the shots. However, learners need to be taught how to work collaboratively. They need to be taught the language they will need to do so. It has been wide- ly documented that, left to their own devices, the agency of the computer in conjunction with learners’ lack to strategic language for pair collaboration results in spotty, impoverished collaboration (Abraham & Liou, 1991; Kumpulainen, 1996; Meskill, 1993) A number of studies on the effectiveness of pretraining sociocollaborative skills prior to having learners work together has demonstrated, however, the success of such an approach (Cohen, 1994: Fisher, 1994; Wergerif & Mercer, 1996). A key feature of well orchestrated task-based language learning activity is that learners can exercise control over the topic and direction of their thoughts as manifest in the language they are learning. Unlike teacher-fronted instruc- tion, it is the individual learner, in consort with peers, who selects the directs the discourse as they deem appropriate and effective to immediate issues and concerns. It is the learner who has control of the mouse, the keyboard, the information, and, to some extent, the process of learning collaboratively. The language professional can capitalize on this learner control by establishing the precedent that each leamer is a knower. She can coach this knowing; espe- cially when it comes to providing a linguistic leg up and prompting task-based conversations to continue. Establishing routines around computers whereby the teacher is not knower but co-participant in the process of discovery is key. Establishing the role of a ‘linguistic court of appeals’ (Higgins, 1988) is also critical to the nurturing of the instructional dynamic. Where emphasis is on the motivated, effortful act of understanding and making meaning with an awareness of the linguistic tools used in that process external mediation of the dynamic can come from an instructor who establishes herself as a facilitator and resource for language and cultural assistance. Additionally, by modeling the behaviors associated with these roles, the instructor can ‘teach’ others to act accordingly as facilitators and resources as needs arise. Instructional tasks, by nature, imply roles to be played on the part of leamers and teachers. The importance of the role of the instructor during language learn- ing task implementation cannot be overstated, especially in terms of supporting awareness of forms and a focus on self correction, and appropriate use of pre- taught forms, functions, and lexis. The instructor draws attention to, makes leamers aware of these forms and how they operate in use, and assists leamers in monitoring their own use of them while they are engaged in task-based work with their peers. This coaching appropriate use of targeted language can serve as a powerful model for learners who can in turn undertake the same kinds of coaching with one another. In designing and implementing instructional tasks around computers, then, teachers build in roles for themselves as orchestrators, facilitators, and participants in these tasks. Contexts where computers serve asaa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.aa You have either reached a page that is unavailable for viewing or reached your viewing limit for this book.number of international experts Ae eve mmU crea mee eon who are acknowledged as lea- ee Language Leaming, were each asked to RMON ot eliteel OM aL aim of providing a critical overview of CARS ROM Moe MMe the design and the applications currently Pram es ecco aaen er. a Om thus at the ‘cutting edge’ of current acti- vity in the domain. Each chapter reviews SACRO LLL Pe a eerie ire Pere el CM oa Mc crt came ela) oT sae Me lcolete MILs RSL books, journals, joumal articles, soft- ware, URLs, etc, relevant to the research erm colt een errata MR ol Se MLS EL CoM om ue ceks elt MeL MS Ue el embarking on a taught masters or on a ra eel Meee mes CGC MAR IS book essential reading to find state-of- DRL ROLL aer eco hTToo SMe Me en MCMC mel reer Re sao RAT ea Aorist user Lt Me ae Rela eeu Cea TULA OMe MR Cet ela MeL MR LAT MR CelL OTs bcd scholarly activity which CALL research involves, |: has been written not only for those TSBN 90-265~1543-x (Nn SWETS & ZEITLINGER PUBLISHERS