You are on page 1of 19

International Forum of Educational Technology & Society

Trends in Educational Technology through the Lens of the Highly Cited Articles Published
in the Journal of Educational Technology and Society
Author(s): Kinshuk, Hui-Wen Huang, Demetrios Sampson and Nian-Shing Chen
Source: Journal of Educational Technology & Society, Vol. 16, No. 2, Grand Challenges and
Research Directions in e-Learning of the 21th Century (April 2013), pp. 3-20
Published by: International Forum of Educational Technology & Society
Stable URL: http://www.jstor.org/stable/jeductechsoci.16.2.3
Accessed: 10-12-2016 05:48 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted
digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about
JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
http://about.jstor.org/terms

International Forum of Educational Technology & Society is collaborating with JSTOR to digitize,
preserve and extend access to Journal of Educational Technology & Society

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Kinshuk, Huang, H.-W., Sampson, D., & Chen, N.-S. (2013). Trends in Educational Technology through the Lens of the Highly
Cited Articles Published in the Journal of Educational Technology and Society. Educational Technology & Society, 16 (2), 3–20.

Trends in Educational Technology through the Lens of the Highly Cited


Articles Published in the Journal of Educational Technology and Society
Kinshuk1, Hui-Wen Huang2*, Demetrios Sampson3 and Nian-Shing Chen4
1
Athabasca University, Canada // 2Wenzao Ursuline College of Languages, Taiwan // 3Centre for Research and
Technology – Hellas (CE.R.T.H.), Greece // 4National Sun Yat-sen University, Taiwan // Kinshuk@athabascau.ca //
huiwen422@gmail.com // sampson@iti.gr // nschen@mis.nsysu.edu.tw
*
Corresponding author

ABSTRACT
The advent of the Internet, World-Wide Web and more recently, advanced technologies such as mobile, sensor
and location technologies have changed the way people interact with each other, their lifestyle and almost every
other aspect of life. Educational sector is not immune from such effects even if the rate of change is far slower
than many other sectors. Researchers have been continuously exploring new ways of using technologies in
education and the field is continuously progressing. This paper looks at this progress by analyzing the highly
cited articles published in the Journal of Educational Technology and Society, in order to identify various trends
and to ponder on the future ahead.

Keywords
Educational technology, Journal of Educational Technology and Society, Highly cited papers, Web of Science,
Social Sciences Citation Index, Research trends

Introduction
The term “educational technology” has been used for quite some time; as early as 1960s when Lawrence Lipsitz first
started Educational Technology magazine. It is difficult to define what educational technology actually means but
researchers and practitioners have typically attributed this term to indicate use of various sorts of technologies to
facilitate educational processes.

With the explosive growth of computers in academia in later half of last century and for individual use in early
eighties, and emergence of the Internet in mainstream education in nineties, educational technology has become
somewhat synonymous to computer based learning and online education.

Journal of Educational Technology and Society came into existence in 1998. This paper aims to provide a vision for
future of educational technology through a systematic analysis of the highly cited papers in the journal, identifying
the themes that survived, those that short-lived, and those that have seen growing popularity over the years.

This bird’s eye view of educational technology through the lens of the Journal of Educational Technology and
Society provides some interesting afterthoughts for both future of education and predictions about the direction
educational technology is taking in coming years.

The next section will describe the rationale behind initiating the journal, its main purpose and a brief introduction of
the key founders. This will be followed by an extensive analysis of various trends that have emerged during past
several years.

Historical background

Since late seventies and early eighties, education sector had started to harness the power of continuously improving
communication technologies, with the computer as its front end. The inter-activity and the inter-connectivity offered
by these technologies promised to have an unprecedented impact on Education - to the extent that Educational
Technology could be talked of as a discipline in its own right, combining the lessons learnt in the diverse fields of
“Artificial Intelligence,” “Educational Psychology,” “Educational Sociology,” and not the least, “Educational
Management.”

ISSN 1436-4522 (online) and 1176-3647 (print). © International Forum of Educational Technology & Society (IFETS). The authors and the forum jointly retain the 3
copyright of the articles. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and that copies bear the full citation on the first page. Copyrights for components of this work owned by
others than IFETS must be honoured. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from the editors at kinshuk@ieee.org.

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
The new found benefits of technology in education caught interest of not only researchers but also of governments
and funding agencies. Millions of dollars were poured in, mostly in America and Europe, with hope that computer
systems would be able to help students in the learning process, hence reducing teachers’ workload. The result was
that research in educational technology touched such advanced issues as intelligent tutoring, simulations, advanced
learning management systems, automatic assessment systems and adaptive systems. However, practitioners, dealing
with real-life academic environment, could not take advantages of all that research with equally fast pace and the
implementation legged seriously behind.

A serious issue emerged with the widening gap of research and implementation that contributed significantly to
further dividing the research and practitioners communities. As evident from the work presented in conferences and
journals in last two to three decades, there has been very little input from actual practitioners in the research process.
Most research in educational technology area has been undertaken by computer scientists and alike. The academics
from other disciplines have been brought from time-to-time in the process of researching advanced systems and
technologies, but mostly to elicit their knowledge so that the systems and technologies could replace them.

Once a learning system is developed, it becomes like a black box to any outsider (including the academics of the
disciplines for which that particular system is developed). There is very little possibility of customization in the
system on the part of the implementing teacher (the one who is expected to use it in his/her curriculum) except
perhaps few pedagogical rules and the chunks of knowledge (learning objects). System designers (primarily
computer science academics) somehow perceive that because they teach their students, they know how to teach, and
therefore the systems developed by them would and should be acceptable by any other teacher, regardless of the
discipline.

This gap between researchers and practitioners was identified in late nineties and as a result, the International Forum
of Educational Technology & Society (IFETS) (http://ifets.ieee.org/) evolved in May 1998 during an informal
discussion at De Montfort University, United Kingdom. The focus of the forum was on the communication gap
which existed between educational system developers and the educators who adopt such systems.

The main purpose of the IFETS forum has been to encourage discussions on the issues affecting the educational
system developer (including artificial intelligence researchers) and educator communities. While recognizing that
this brief might be seen as too broad, it was proposed to conduct multiple discussion threads on more specific topics.
This approach helped in developing specific aspects concerning the design and implementation of integrated learning
environments while sharpening the overall vision about the purpose and processes of education.

To provide a synthesis of the discussions held in the forum and to articulate the thinking of both communities,
Journal of Educational Technology & Society was conceived as an archival entity, which could bring exposure to
everyone’s perspectives, and hopefully provide at least a platform for justifying the differences if not to diffuse those
differences.

The first issue of the journal was published in October 1998. Since then, the journal has been published quarterly and
is a focal point to record on-going discussions in the discipline, on implementation projects, present invited
viewpoints and perspectives from experts in diverse fields and also to encourage peer-reviewed articles providing a
more detailed treatment of the various aspects of educational technology, its objectives and its contexts. Journal of
Educational Technology and Society is meant to be the mouth piece of the diverse membership of both researchers
and practitioners community and each group is strongly encouraged to make their voice heard. Through dialogue and
specialization, journal aims to maintain ‘unity within diversity.’

Analysis of highly cited papers

Generally, the advancement of knowledge is driven by a variety of contributions. The highly cited articles are
considered to play important roles in knowledge contribution because researchers tend to cite high quality articles
that are useful for their own research (Aylward, Roberts, Colombo, & Steele, 2008; Lee, Wu, & Tsai, 2009).
Identifying the highly cited articles seems to be a reliable and objective means because it shows valuable research
topics in the profession (Flores et al., 1999). When one article is cited by many subsequent papers, it means that this
article has its influence and contribution in a particular field (White & White, 1977). The research trends can be

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
highlighted through examining research topics among the highly cited articles. In addition, a list of the highly cited
articles gives novice researchers a direction to focus on the highly influential articles during a specific period of time
and develop their own research interests.

Several scholars stated the importance of reviewing journal publications. For example, White and White (1977)
pointed out that “the importance of a journal is determined by the overall quality of the articles it carries” (p. 301).
Analyzing the publication history of a specific journal can reveal a more accurate view of the publication pattern.
The action of reviewing journal publications can “provide the editors of the journal an opportunity to reflect on the
consistency of their publication decisions in relationship to the journal’s mission statement and policies” (Taylor,
2001, p. 324). To understand the publication pattern, Garfield (1983) proposed citation analysis to explore the
frequencies, patterns, and graphs of citations in articles and books. It measures the importance of particular journals
or authors in a scholarly community (Flores et al., 1999; Rourke & Szabo, 2002; Taylo, 2001).

According to Chiu and Ho (2007), the impact or visibility of an article can be identified by the number of citations.
Previous studies have reviewed the highly cited articles in different fields (Aksnes, 2003; Allen, Jacobs, & Levey,
2006, Blessinger & Hycaj, 2010). However, little research has been conducted regarding the review of the
characteristics of the highly cited articles in the field of educational technology. Through a systematic analysis, the
present study provides insight to the research trends and basic citation trends of the highly cited articles published in
the Journal of Educational Technology and Society (ET&S).

The purpose of the current study has been twofold. First, the authors explored the distribution of major research
topics among the highly cited articles published in the ET&S during 2003-2010. Second, the authors examined the
emerging trends after reviewing the highly cited empirical studies in the ET&S. The reason to focus on empirical
studies with high citation counts was that such information would provide important insight for junior researchers to
plan their research topics from theory to practice. In addition, to avoid a disadvantage of recently published articles
with less citation counts in a long time frame, the authors compared the highly cited empirical studies published
within a four-year time interval, i.e., 2003-2006 and 2007-2010.

In order to provide researchers who are interested in submitting journal papers to the ET&S, the authors
systematically analyzed the highly cited articles in the journal to provide valuable information about using these
articles as guides for their own studies. Thus, the research questions addressed by this study are as follows:
1. What research types were identified from the highly cited articles published in the ET&S during 2003-2010?
What were their variations between the first four years (2003-2006) and the second four years (2007-2010)?
2. What were the characteristics of the highly cited empirical studies published in the ET&S during 2003-2010?
What were their variations between the first four years (2003-2006) and the second four years (2007-2010)?
3. What were the emerging research trends of the highly cited empirical studies published in the ET&S during
2003-2010? What were their variations between the first four years (2003-2006) and the second four years
(2007-2010)?

Related work

Citation analysis is a useful tool to provide a direct and objective means of analyzing influences in a certain research
field (Garfield, 1955; Smith, 1981). Shih, Feng and Tsai (2008) claimed that “articles with more citation frequencies
are usually those that are better recognized by others in related fields. They probably present more fundamental ideas
about the issues for future research” (p. 960). Many studies on citation analysis reviewed highly cited articles in
different disciplinary fields, such as science (Aksnes, 2003), ecology and ecological economics (Leimu & Koricheva,
2005), geomorphology (Doyle & Jlian, 2005), nursing (Allen, Jacobs, & Levey, 2006), software engineering (Wohlin,
2007), e-learning (Shih et al., 2008), instructional design (Ozcinar, 2009), computer-assisted language learning
(Uzunboylu & Ozcinar, 2009), library and information science (Blessinger & Hycaj, 2010) and knowledge
management in education (Uzunboylu, Eris, & Ozcinar, 2011). These studies showed raw citation count to identify
the influence of scholarly work.

Previous studies have attempted to describe the development of educational technology research in different time
periods using content and citation analysis. For example, Klein (1997) analyzed 100 articles published in the
Educational Technology Research and Development between 1989 and 1997. Taylor (2001) conducted a content

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
analysis of all articles submitted to Adult Education Quarterly from 1989 to 1999. Rourke and Szabo (2002)
analyzed articles published in Journal of Distance Education during 1986-2001. Lee, Driscoll and Nelson (2004)
examined 383 articles published in four professional journals in the field of distance education from 1997 to 2002
using content analysis. Tsai and Wen (2005) reviewed the research papers in science education during 1998 to 2002
using manual coding. Aylward, Roberts, Colombo, and Steele (2008) used citation analysis to examined documents
with a large number of citations in a specific journal from 1976 to 2006. Shih et al. (2008) reviewed 444 articles
related to the topic of cognition in e-learning from 2001 to 2005. Ozcinar (2009) examined 758 documents regarding
the topic of instructional design during 1998-2008, retrieved from the Web of Science database, to conduct content
analysis and citation analysis. These researchers illustrated insightful information to examine trends and patterns in
scholarly documents.

According to Noyons and van Raan (1998), splitting the publication data into two periods can help in better
understanding the relationship between the two different periods in terms of monitoring research trends of the
identified topics. Lee et al. (2009) analyzed highly cited science education articles published during 1998-2002 and
2003-2007, and found a dynamic shift in the research topic. Tsai, Wu, Lin, and Liang (2011) selected 228 empirical
papers to examine the research trends regarding science learning in Asia during 2000-2004 and 2005-2009. These
studies helped readers to visualize dynamic trends in different periods of time.

Recently, studies on using author keywords to analyze the research trends have shown that this method can
effectively predict the research tendency (Chiu & Ho, 2007; Mao, Wang, & Ho, 2010; Li et al., 2009; Ozcinar, 2009).
The purpose of author keywords analysis is to identify their frequency and discover directions of scientific research.
To find the most frequently appeared words, McNaught and Lam (2010) used Wordle, a web-based word clouds
program, to analyze the transcriptions of six focus-group meetings. They claimed that word clouds can be a
supplementary research tool in conducting content analysis. Word clouds can present what the most common words
are with their size reflecting their frequency.

Although several articles mentioned above have conducted studies on content and citation analyses regarding
educational technology research, the citation analyses examined in the highly cited articles in a specific journal have
not been examined in detail. Hence, the current study systematically analyzed the research type, research topic, first
author’s country, international collaboration, participant levels, learning domain, research method, and frequently
appearing keywords among top 20 highly cited articles in the ET&S during 2003-2010.

Method
Materials

The data were based on the highly cited articles published in the ET&S from 2003 to 2010. The ET&S was chosen
for two reasons. First, the ET&S has a high impact factory of 1.066 in Thomson Scientific 2010 Journal Citations
Report, from the Web of Science database. Second, the ET&S is one of the leading Social Science Citation Index
(SSCI) journals in the field of educational technology. The ET&S, a quarterly journal, began publishing referred
journal articles in 1998, and has been on the SSCI list since 2003. All the journal’s articles are freely accessing
online at http://www.ifets.info.

The search period was set during 2003 to 2010 because the ET&S was not indexed in the Web of Science database
until 2003. It is important to note that the articles in the Web of Science database show more consistency in quality
under restrict peer-review and an objective evaluation process (Braun, Schubert, & Kostoff, 2000; Wohlin, 2007).
Hence, identifying the characteristics of the highly cited articles published in the ET&S during the past eight years
will provide a macroscopic and systematic examination for readers to have a holistic and accurate interpretation of
the research influence in the field of educational technology.

The Web of Science database (http://www.isiknowledge.com/), published by the Institute for Scientific Information
(now Thomson Reuters), was the literature source in this study. The reason to use the Web of Science database was
that it is the most important and frequently used source database in conducting bibliometric studies in various
research fields (Gil-Montoya et al., 2006; Lee, Wu, & Tsai, 2009; Li et al., 2009; Mao, Hwang, & Ho, 2010; Ozcinar,
2009; Tsai & Wen, 2005).
6

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Procedures

The authors went through three steps to analyze the characteristics of the highly cited ET&S articles. In the first step,
the authors searched the name of the ET&S journal in the Web of Science database for the timespans 2003-2010,
2003-2006, and 2007-2010, respectively. Afterwards, the authors refined the search by specifying articles under the
category of document types. The database produced total citation numbers of articles published in the ET&S in
different time intervals, using the update data as of November 30, 2011. Since this study focused on examining the
highly cited articles until 2010, a calculation of the number of citations per article was computed by subtracting the
citations of the year 2011. The criterion for selecting the highly cited articles was those articles published in the
ET&S and cited at least 15 times.

In the second step, the authors identified research types, research topics, first author’s country, participant level,
learning domain, research methods, and frequently appearing keywords among all the articles obtained from the
results of the first step. The process of classifying research type and research topics was jointly coded by two raters
(one of the authors and one research assistant with master’s degree in cognitive psychology). The two raters first
discussed the coding criteria, and then separately coded three articles listed on the highly cited articles during 2003-
2010. The agreement rate between raters for all coding results was 90%, suggesting that the coding classification
used in this study was stable and reliable. Disagreements were resolved via three face-to-face discussions between
the two raters.

In the final step, the authors used word clouds (http://www.wordle.net) to validate the previous analysis in manual
coding implemented in the second step. Wordle is a web-based visualization program to generate word clouds. The
authors first typed all the keywords listed on the highly cited empirical studies, and the program automatically
generated a graphic on a new web page.

Data analysis

In addition to manual coding, the authors used word clouds to be a supplementary research tool to support traditional
content analysis methods (McNaught & Lam, 2010). Word clouds reveal the frequencies of the different words
within the body of text. The more frequent the word, the more important is the concept (McNaught & Lam, 2010).
Since word clouds deal with each word as the unit of analysis, the authors used this supporting tool to validate the
finding of research topics obtained from manual coding.

Results
Identification of the highly cited articles published in the ET&S

The results of the citation analysis of the top 20 highly cited articles from the years 2003-2010, 2003-2006, and
2007-2010 included self-citations. Appendix-1, Appendix-2 and Appendix-3 present the results, which have been
ranked in order by the number of citations, in different time intervals. This yielded 20, 20, and 23 articles for the
periods 2003-2010, 2003-2006, and 2007-2010, respectively. The reason for retrieving more than 20 articles was the
tied number of citations among the last four highly cited articles. A detailed list of the highly cited articles in rank
order by total number of citations can be found in the Appendix.

Research types

All the data retrieved from the Web of Science database were identified into four categories: system and/or model
design, empirical study, theoretical paper, and other. The four categories were modified from those suggested by Lee
et al. (2009). The category of system and/or model design included articles that report a new system and/or model
applied in a new learning context, without statistical analyses. Empirical study category included articles that report
results obtained from what the research methods were (quantitative, qualitative, or mix-method), who the participants
were, what the participants did, and what measures were utilized. Theoretical paper category included articles that
propose “a new theory or theoretical framework” in the field of educational technology (Lee et al., 2009, p. 2002).
7

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Since some articles could not meet the definition of the three categories, they were classified into the category “other.”
Table 1 presents the frequencies and percentages of research types after the two coders manually classified the top 20
highly cited articles in different time intervals. During 2003-2010, 40% (n = 8) were under the category of system
and/or model design, 30% (n = 6) were empirical studies, 15% (n = 3) were theoretical papers, and 15% (n = 3) were
under the category of other. From 2003 to 2006, 38.1% (n = 8) were system and/or model design, 23.8% (n = 4) were
empirical studies, 14.3% (n = 3) were theoretical papers, and 23.8% (n = 5) were under the category of other. During
2007-2010, 17.4% (n = 4) were system and/or model design, 52.2% (n = 12) were empirical studies, 8.7% (n = 2)
were theoretical papers, and 21.7% (n = 5) were under the category of other. The comparison of different research
types in highly cited articles is presented in Figure 1.

Table 1. Frequencies and percentages of research types in top 20 highly cited articles during 2003-2010, 2003-2006, and 2007-2010
Research types Frequencies (Percentages)
2003-2010 2003-2006 2007-2010
System and/or model 8 (40%) 8 (38.1%) 4 (17.4%)
design
Empirical study 6 (30%) 4 (23.8%) 12 (52.2%)
Theoretical paper 3 (15%) 3 (14.3%) 2 (8.7%)
Other 3 (15%) 5 (23.8%) 5 (21.7%)
Total 20 (100%) 20 (100%) 23 (100%)

(a)

(b) (c)
Figure 1. (a) Distribution of the research types during 2003-2010; (b) Distribution of the research types during 2003-
2006; and (c) Distribution of the research types during 2007-2010

Co-authorship

The results indicated that the vast majority of the highly cited articles were co-authored with one or more
collaborations. For example, the percentages of co-authored articles with either the same country or different country
were 90% (n = 18), 80% (n = 16), and 83% (n = 19) for the periods 2003-2010, 2003-2006, and 2007-2010,
respectively.

During 2003-2010, 85% (n = 17) of the highly cited articles were written by more than one author. Yang (2006) from
Taiwan, Liu (2005) from Taiwan, and Nichols (2003) from New Zealand were the only three single authors among

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
the highly cited ET&S articles. To divide different time intervals, the authors found that four highly cited ET&S
articles were written by single authors between 2003 and 2006. They were: Yang (2006) from Taiwan, Nichols (2003)
from New Zealand, Liu (2005) from Taiwan, and Anohina (2005) from Latvia. Four highly cited ET&S articles were
also written by single authors from 2007 to 2010, namely Paquette (2007) from Canada, Dron (2007) from UK, Liu
(2007) from Taiwan and Yang (2009) from Taiwan.

International collaboration

Of the top 20 highly cited articles published in the ET&S during 2003-2010, 20% (n = 4) articles had international
co-authorship. The articles with international collaboration were: Koper and Olivier (2004) between The Netherlands
and U.K. with 84 citation counts, Aroyo and Dicheva (2004) between The Netherlands and U.S.A. with 29 citation
counts, Avgeriou et al. (2003) between Greece and Cyprus with 25 citation counts, and Aroyo et al. (2006) among
The Netherlands, Germany, Belgium, Sweden, and Austria with 18 citation counts. Researchers from the Netherland
were the most active to collaborate with scholars with other countries in publishing internationally co-authored
articles.

The results of four highly cited ET&S articles in international collaboration during 2003-2006 were identified to
those of the articles between 2003 and 2010. The four articles were: Koper and Olivier (2004), Aroyo and Dicheva
(2004), Avgeriou et al. (2003), and Aroyo et al. (2006). Four highly cited ET&S articles during 2007-2010 were
identified with international collaboration. They were: Jovanovic et al. (2007) between Serbia and Canada with 12
citation counts, Teo, Luan, and Sing (2008) between Singapore and Malaysia with 9 citation counts, Hastie, Chen,
and Kuo (2007) between Australia and Taiwan with 9 citation counts, and Chen, Kinshuk, and Wei (2008) between
Taiwan and Canada with 7 citation counts. Researchers from Canada and Taiwan were the most active to engage in
international collaboration in publishing articles.

Research topics under system and/or model design articles

To further analyze research topics in highly cited articles under system and/or model design during 2003-2010, the
authors identified that research topics were adaptive learning (Karampiperis & Sampson, 2005; Henze, Dolog, &
Nejdl, 2004; Aroyo et al., 2006), mobile and ubiquitous learning (Yang, 2006; Kravcik et al., 2004) , e-learning
(Aroyo & Dicheva, 2004), and collaborative learning (Yang, Chen, & Shao, 2004).

During 2003-2006, the research topics were mobile and ubiquitous learning (Yang, 2006; Kravcik et al., 2004),
adaptive learning (Karampiperis & Sampson, 2005; Henze, Dolog, & Nejdl, 2004; Aroyo et al., 2006), e-learning
(Aroyo & Dicheva, 2004), collaborative learning (Yang, Chen, & Shao, 2004), and assessment criteria (Yin et al.,
2006).

During 2007-2010, three research topics were identified: ontology (Jovanovic et al., 2007; Boyce & Pahl, 2007),
personalized learning (Wang et al., 2007), and ICT integration (Wang & Woo, 2007).

Characteristics of the highly cited empirical studies published in the ET&S

The following content analyses of the highly cited empirical studies were identified on the basis of research topics,
author’s country, participant level, learning domain, research methods, and frequently appearing author keywords.

Research topics

Based on the highly cited empirical studies during 2003-2010, the authors used manual coding to identify four
research topics. Table 2 shows the four research topics: collaborative learning (Hernandez-Leo et al., 2006; Zurita et
al., 2005), game-based learning (Holzinger et al., 2008; Virvou et al., 2005), mobile learning and ubiquitous learning
(El-Bishouty et al., 2007), and technology adoption (Sugar et al., 2004).

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Table 2. Distribution of research topics of the highly cited empirical studies published in the ET&S during 2003-2010
Author/Year Research topic Citation counts
Hernandez-Leo et al. (2006) Collaborative learning 36
Holzinger et al. (2008) Dynamic media 30
Virvou et al. (2005) Game-based learning 28
Zurita, et al. (2005) Collaborative learning with mobile devices 22
Sugar et al. (2004) Technology adoption 20
El-Bishouty et al. (2007) Ubiquitous learning 15

After manually coding four highly cited empirical studies during 2003-2006, the authors found three main research
topics: collaborative learning (Hernandez-Leo et al., 2006; Zurita, et al., 2005), game-based learning (Virvou et al.,
2005), and technology adoption (Sugar et al., 2004) (Table 3).

Table 3. Distribution of research topics of the highly cited empirical studies published in the ET&S during 2003-2006
Author/Year Research topic Citation counts
Hernandez-Leo et al. (2006) Collaborative learning design 36
Virvou et al. (2005) Game-based learning 28
Zurita et al., (2005) Collaborative learning 22
Sugar et al., (2004) Technology adoption 20

Several research topics were classified in the highly cited empirical studies during 2007-2010. The authors
categorized mobile and ubiquitous learning (El-Bishouty et al., 2007; Chen & Hsu, 2008; Liu, 2007), e-learning
(Yukselturk & Bulut, 2007; Demetriadis & Pombortsis, 2007), dynamic media (Holzinger et al., 2008), forum
analysis (Hou et al., 2008), technology adoption (Teo et al., 2008), blended learning (Delialioglu & Yildirim, 2007),
Web 2.0 (Yang, 2009), and collaborative learning (Huang et al., 2009) (Table 4).

Table 4. Distribution of research topics of the highly cited empirical studies published in the ET&S during 2007-2010
Author/Year Research topic Citation counts
Holzinger et al., (2008) Dynamic media 31
El-Bishouty et al., (2007) Ubiquitous learning 15
Chen & Hsu (2008) Mobile learning 11
Teo et al., (2008) Technology adoption 9
Hou et al., (2008) Forum analysis 9
Yukselturk & Bulut (2007) e-learning 8
Liu (2007) Mobile learning 8
Yang (2009) Web 2.0 7
Delialioglu & Yildirim (2007) Blended learning 7
Demetriadis & Pombortsis (2007) e-learning 6
Makri & Kynigos (2007) Web 2.0 6
Huang et al., (2009) Collaborative learning 6

Authors’ Countries

Table 5 shows the frequencies of all authors’ countries over different time intervals. Based on author’s country, the
following countries were identified in the highly cited empirical studies during 2003-2010: Spain, Austria, Greece,
Chile, U.S.A., and Japan. Among four highly cited empirical studies during 2003-2006, the authors originated from
Spain, Greece, Chile, and U.S.A. With regard to 12 highly cited empirical studies during 2007-2010, 41.7% (n = 5)
of the authors’ country were from Taiwan, followed by Greece and Turkey, both of them with two highly cited
empirical studies. There was one international co-authored empirical study conducted by researchers from Singapore
and Malaysia during 2007-2010.

Table 5. Frequencies of author’s country in highly cited empirical studies during 2003-2010, 2003-2006, and 2007-2010
Country Frequencies
2003-2010 2003-2006 2007-2010
10

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Spain 1 1 0
Austria 1 0 1
Greece 1 1 2
Chile 1 1 0
Taiwan 0 0 5
U.S.A. 1 1 0
Japan 1 0 1
Singapore 0 0 1 (international
collaboration)
Malaysia 0 0 1 (international
collaboration)
Turkey 0 0 2
Total 6 4 12

Participant levels

As revealed in Table 6, all highly cited empirical studies involved post-secondary students and elementary school
students. No other educational levels of participants, such as junior high and senior high students, were found. The
educational levels of the participants involved in six empirical studies published during 2003 to 2010 were:
elementary school level (50%) and college level (50%). During 2003-2006, two empirical studies (Hernandez-Leo et
al., 2006; Sugar et al., 2004) involved college students and two empirical studies (Virvou et al., 2005; Zurita et al.,
2005) used elementary school students. Interestingly, 11 empirical studies during 2007-2010 involved college
students whereas only one study (Liu, 2007) used elementary school students.

Table 6. Frequencies of participant level in highly cited empirical studies during 2003-2010, 2003-2006, and 2007-2010
Participant level Frequencies
2003-2010 2003-2006 2007-2010
College 3 2 11
Elementary school 3 2 1
Total 6 4 12

Learning domain

Table 7 presents different learning domains applied in the highly cited empirical studies over different time intervals.
During 2003-2010, two articles were classified into education domain and two articles were science domain
(including computer and engineering). The two remaining empirical studies were about math and geography. During
2003-2006, two empirical studies (Hernandez-Leo et al., 2006; Sugar et al., 2004) were conducted in education
domain, and the two remaining studies were about math (Zurita, et al., 2005) and geography (Virvou et al., 2005).
During 2007-2010, six empirical studies were conducted in science domain. Four studies (Chen & Hsu, 2008; Teo et
al., 2008; Yang, 2009; Makri & Kynigos, 2007) were identified into education domain, and the two remaining
studies were classified into math (Liu, 2007) and business (Hou et al., 2008) domains.

Table 7. Frequencies of learning domain in highly cited empirical studies during 2003-2010, 2003-2006, and 2007-2010
Learning domain Frequencies
2003-2010 2003-2006 2007-2010
Education 2 2 4
Geography 1 1 0
Math 1 1 1
Science (including computer and 2 0 6
engineering)
Business 0 0 1
Total 6 4 12

11

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Research method

Table 8 shows different research methods applied in the highly cited empirical studies over different time intervals.
During 2003-2010, among highly cited empirical studies (n = 6), four empirical studies utilized mixed method and
two used quantitative method. During 2003-2006, three empirical studies (Hernandez-Leo et al., 2006; Virvou et al.,
2005; Sugar et al., 2004) used mixed method, followed by one empirical study (Zurita et al., 2005) with quantitative
method. No empirical study using qualitative method was found in the two intervals. During 2007-2010, five
empirical studies (Hou et al., 2008; Liu, 2007; Yang, 2009; Delialioglu & Yildirim, 2007; Makri & Kynigos, 2007)
utilized qualitative method, followed by four studies (Chen & Hsu, 2008; Yukselturk & Bulut, 2007; Demetriadis &
Pombortsis, 2007; Huang et al., 2009) with mixed method and three studies (Holzinger et al., 2008; El-Bishouty et
al., 2007; Teo et al., 2008) with quantitative method. Interestingly, the number of empirical studies using qualitative
method obviously increased in the period of 2007-2010.

Table 8. Frequencies of research method in highly cited empirical studies during 2003-2010, 2003-2006, and 2007-2010
Research method Frequencies
2003-2010 2003-2006 2007-2010
Mixed method 4 3 4
Quantitative 2 1 3
Qualitative 0 0 5
Total 6 4 12

Frequent author keywords

According to Mao, Wang, and Ho (2010), author keywords analysis provides researchers with “the information of
research trend” (p. 813). The authors used word clouds to present the frequently used author keywords obtained from
the highly cited empirical studies during 2003-2010, 2003-2006, 2007-2010, respectively.

The Wordle program automatically generated three graphics (Figure 2(a), 2(b), and 2(c)) after the authors typed all
the keywords listed on the empirical studies in different periods of time. The results indicated that “learning” was the
top keyword that appeared in empirical studies over the different time intervals. During 2003-2010, the word clouds
showed that frequently appeared keywords were: collaborative learning, computer-supported collaborative learning,
dynamic media, and educational technology (Figure 2(a). It is important to note that the Wordle program shows
single words, and in this analysis, those words were combined as per the keywords provided by the authors in order
to obtain meaningful analysis. Only four empirical studies were identified during 2003-2006. The frequently
appeared keywords were: computer-supported collaborative learning and educational technology (Figure 2(b)).
During 2007-2010, the word clouds showed that mobile learning, media learning, blended learning, online learning,
instructional technology, and language learning using blog (Figure 2(c)).

(a)

(b) (c)
Figure 2. (a)Word clouds of the keywords listed on empirical studies during 2003-2010; (b) Word clouds of the keywords listed
on empirical studies during 2003-2006; and (c) Word clouds of the keywords listed on empirical studies during 2007-2010
12

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Discussion

The purpose of the current study was to explore the characteristics of the highly cited articles published in the ET&S
during 2003-2010. Appendix-1 presents the top 20 highly cited articles published in the ET&S from 2003 through
2010. Obviously, Appendix-1 demonstrates that the top 20 highly cited articles were mostly about system and/or
model design (40%, n = 8) during 2003-2010. It is not surprising to obtain such findings because most researchers in
the field of educational technology conducted system/model design in a short period of time to report how the
system/model worked in a learning setting.

The distribution of highly cited empirical studies during 2003-2006 and 2007-2010 was quite different. The results
indicated that only four (23.8%) empirical studies were found in the top 20 highly cited articles from 2003 to 2006.
On the other hand, 12 (52.2%) empirical studies were retrieved within the top 20 highly cited articles from 2007 to
2010. The reason could be that the ET&S did not receive many high quality empirical studies before 2007. As a
result, fewer empirical study articles were cited by other scholars during 2003-2006.

The impact or visibility of an article can be identified by the number of citations (Chiu & Ho, 2007). The overall
quality of the highly cited articles published in the ET&S over the past years appeared to be good due to an increase
in mean citation count every year (impact factor in 2010 = 1.067) shown on the Web of Science database. The
numbers of internationally co-authored articles in different time intervals were the same. They did not increase in
recent years. One of the reasons may be that it is difficult to find common research topics among researchers from
different countries. Moreover, it would be reasonable to assume that participants’ different characteristics and
English proficiencies may hinder the possibilities of conducting international collaboration. To increase more
international collaborations in research fields, the policy makers may provide consistent financial support for those
researchers who are interested in publishing international co-authored articles while allocating national funding.

From Appendix-1, it is evident that will be the key trends in the near future. Further, mobile learning technology and
ubiquitous collaborative learning with mobile devices, and game-based learning, and ubiquitous learning were the
core research topics, based on the six highly cited empirical studies during 2003-2010. This is in line with the 2011
Horizon Report (http://wp.nmc.org/horizon2011/). In this report, the application of mobile devices and game-based
learning learning are new research topics with great potential in academia (Hwang & Tsai, 2011; Liu & Hwang,
2010). Hence, the three core research topics definitely echo these researchers’ statements and indicate a future
direction in the field of educational technology.

After splitting into two different time intervals, the authors found different results. During 2003-2006, among four
frequently cited empirical studies, collaborative learning was the hot research topic in this time frame. During 2007-
2010, mobile/ubiquitous learning, e-learning, and Web 2.0 were identified to be the trends in citations among the 12
highly cited empirical studies. Particularly, the topic of collaborative learning became less representative in recent
years. It is possible that the findings of collaborative learning studies have matured in the educational technology
field, which in turn affects the citation counts of these articles. On the other hand, mobile/ ubiquitous learning and
other technology-based learning have become popular research topics over the recent years. For example, two studies
related to mobile/ubiquitous learning conducted by El-Bishouty et al. (2007) and Chen & Hsu (2008) received 16
and 11 citation counts respectively (as of November 30, 2011). It is therefore not surprising that the results of the
present study are consistent with the 2011 Horizon Report due to the fact that mobile devices have become
affordable and wireless network connections are accessible for the public.

The researchers in the field of educational technology generally conduct empirical studies in different learning
domains. The results of analyzing highly cited empirical studies indicated that the research of using technology in
science classes and education programs showed their impact during 2007-2010. In particular, 50% (6 out of 12) of
the highly cited empirical studies during 2007-2010 were found in science curriculum (including computer and
engineering). The studies of using technology in the science classroom published in the ET&S obtained more
attention from the researchers over the past four years. Based on the findings, it is predicted that using technologies
in different learning domains will be foreseeable.

In terms of research methodology, mixed-method was found to be the major research method in highly cited
empirical studies over 2003-2010. By analyzing the research methods used in highly cited empirical studies, the

13

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
authors concluded that mixed-method was popularly applied in the field of educational technology. For researchers,
the reason to apply mixed method design is to collect both quantitative and qualitative data in order to present
complete pictures and in-depth explanation about the findings.

The increase of using qualitative method in educational technology research was observed at different time intervals.
No papers with qualitative research were found during 2003-2006, whereas five (42%) qualitative research papers
were identified during 2007-2010. It suggests that the findings in educational technology research need more in-
depth exploration to investigate users’ thoughts and concerns.

Interestingly, the results of using word clouds to present the frequently used author keywords were similar to the
findings obtained from manual coding. For instance, in analyzing empirical studies, the authors found that ubiquitous
learning, mobile learning, and collaborative learning were highlighted in Wordle.

The findings of this study are constrained by some limitations, which similar studies in future should address. First,
selecting a single journal to analyze the highly cited articles might be skewed towards a certain research field due to
the small number of published articles. It might not be truly representative of the total literature of educational
technology. Analyzing different journals in the same field may have different results regarding the characteristics
identified in this study. Second, the analysis on research topics obtained from the highly cited empirical studies could
be extended to analyze research topics from all articles published in the ET&S. Third, the highly cited articles were
analyzed by total citation counts obtained from 2010 Journal Citation Report. Future study may analyze the h index
in the highly cited articles by comparing individual authors’ h indices and their papers cited in the field of
educational technology. Finally, the citation counts used in this study included self-citations. Use of indications
(rather than indicators, such as citation counts) to evaluate the quality of the highly cited articles lends itself to
further investigation (Aksnes, 2003).

The way forward

It has been very interesting journey through time to see how educational technology has progressed as reported in the
highly cited papers in the Journal of Educational Technology and Society. The field has matured immensely in
certain areas and new directions are opening up. Still, the issue of “learning” has stayed on the top and hopefully, we
would be sensible enough to keep it that way.

The major goal of the journal, since when it was started, has been to open up dialog between those who design the
educational technology and those who use it. The analysis seems to endorse progress in that area even if a lot of
work still to be done. Interestingly, the patterns emerged during the analysis align with the analyses of other
prominent initiatives, such as the Horizon Reports published by the New Media Consortium (http://www.nmc.org),
and the roadmap for education technology compiled by Woolf (2010). For example, Horizon Reports have over past
few years consistently identified research areas related to mobility, collaboration, social media and personalization,
as some of the technologies with the best chances of adoption. Woolf (2010)’s roadmap also identified these as
promising areas to overcome various challenges that are experiences in today’s educational environment. Findings of
the study presented in this paper agree with these analyses and provide indication of a healthy research progress for
the advancement of these educational technology research areas worldwide.

In terms of the coverage of the issues, concerns related to the impact of ICT were very predominant at earlier stage
but later declined, as Web-based learning has more and more integrated into mainstream education and teething
problems have started to sort out.

Educational paradigms and concerns for individual students have continued to dominate the field and the trend
indicates that it will continue to do so.

Infrastructure issues and associated technologies have featured continuously but there is a rapid shift in the field.
Earlier issues of the journal featured areas like hypermedia but the focus then shifted to more advanced entities such
as collaborative technologies, social media, mobile learning and collaborative technologies, and the trend seems to
continue for a foreseeable future.

14

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Overall, the analysis indicates that the field of educational technology is a rapidly evolving field. Both educational
paradigms and technological advancements are affected. However, the changes in technology are at much faster pace
compared to the shifts in educational paradigms. It would be very interesting to see how the landscape develops in
next few years, when the true effects of globalization and ever improving connectivity based technologies, such as
ubiquitous and augmented reality technologies mature.

Conclusions
The distribution of research topics in highly cited empirical studies identified in this study provides insights for
educators and researchers in the educational technology field to develop their future research interests. Moreover, the
results might lead researchers in educational technology to focus their manuscript submissions on the hot research
topics found in this study.

To monitor the research trends in the field of educational technology, the authors used word clouds to analyze author
keywords listed in the highly cited empirical studies and made a cross-validation with the research topics found in
this study. The authors could conclude that the future research direction of educational technology is mobile learning,
ubiquitous learning, and game-based learning. The findings provide directions to better understand the future
potential research topics.

Two questions were worthy of re-thinking after the authors finished this study. Why did the highly cited articles
published in the ET&S with single author receive high citation rates? Future studies might ask the author(s) about
their comments. Another question is that international collaboration illustrates a contribution factor for the highly
cited articles. How did these authors from different countries find their common research topics? The current study
has set the footing and foundation of guiding future studies on these issues.

Acknowledgements
This study is supported by National Science Council, Taiwan, under the contract number of NSC 101-2917-I-110-
001, NSC 100-2511-S-110-001-MY3, NSC 100-2631-S-011-003 and NSC 99-2511-S-110-004-MY3. The authors
would like to thank Mr. Wei-Chieh Fang to jointly code the sections of research types and research topics in this
study. The authors also acknowledge the support of NSERC, iCORE, Xerox, and the research-related gift funding by
Mr. A. Markin.

References
*References marked with an asterisk indicate studies included in the highly cited list in this study.
Aksnes, D. W. (2003). Characteristics of highly cited papers. Research Evaluation, 12(3), 159-70.
Allen, M., Jacobs, S. K., & Levy, J. R. (2006). Mapping the literature of nursing: 1996-2000. Journal of the Medical Library
Association, 94(2), 206-220.
*Anohina, A. (2005). Analysis of the terminology used in the field of virtual learning. Educational Technology & Society, 8(3),
91-102.
*Aroyo, L., & Dicheva, D. (2004). The new challenges for e-learning: The educational semantic web. Educational Technology &
Society, 7(4), 59-69.
*Aroyo, L., Dolog, P., Houben, G.-J., Kravcik, M., Naeve, A., Nilsson, M., & Wild, F. (2006). Interoperability in personalized
adaptive learning. Educational Technology & Society, 9(2), 4-18.
*Avgeriou, P., Papasalouros, A., Retalis, S., & Skordalakis, M. (2003). Towards a pattern language for learning management
systems. Educational Technology & Society, 6(2), 11-24.
Aylward, B. S., Roberts, M. C., Colombo, J., & Steele, R. G. (2008). Identifying the classics: An examination of articles
published in the Journal of Pediatric Psychology from 1976-2006. Journal of Pediatric Psychology, 33(6), 576-589.
Blessinger, K., & Hrycaj, P. (2010). Highly cited articles in library and information science: An analysis of content and authorship

15

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
trends. Library and Information Science Research, 32, 156-162.
Braun, T., Schubert, A. P., Kostoff, R. No. (2000). Growth and trends of fullerene research as reflected in its journal literature.
Chemical Reviews, 100(1), 23-38.
*Boyce, S., & Pahl, C. (2007). Developing domain ontologies for course content. Educational Technology & Society, 10(3), 275-
288.
*Chen, C.-M., & Hsu, S.-H. (2008). Personalized intelligent mobile learning system for supporting effective English learning.
Educational Technology & Society,11(3), 153-180.
*Chen, N.-S., Kinshuk, Wei, C.-W., & Yang, S. J. H. (2008). Designing a self-contained group area network for ubiquitous
learning. Educational Technology & Society, 11(2), 16-26.
Chiu, W.-T., & Ho, Y.-S. (2007). Bibliometric analysis of tsunami research. Scientometrics, 73(1), 3-17.
*Delialioglu, O., & Yildirim, Z. (2007). Students’ perceptions on effective dimensions of interactive learning in a blended
learning environment. Educational Technology & Society, 10(2), 133-146.
*Demetriadis, S., & Pombortsis, A. (2007). E-lectures for flexible learning: A study on their learning efficiency. Educational
Technology & Society, 10(2), 147-157.
Doyle, M. W., & Julian, J. P. (2005). The most-cited works in Geomorphology. Geomorphology, 72, 238-249.
*Dron, J. (2007). Designing the undesignable: Social software and control. Educational Technology & Society, 10(3), 60-71.
*El-Bishouty, M.M., Ogata, H., & Yano, Y. (2007). PERKAM: Personalized knowledge awareness map for computer supported
ubiquitous learning. Educational Technology & Society, 10(3), 122-134.
Flores, L. Y., Rooney, S. C., Heppner, P. P., Browne, L. D., & Wei, M. F. (1999). Trend analyses of major contributions in the
Counseling Psychologist cited from 1986 to 1996: Impact and implications. The Counseling Psychologist, 27, 73-95.
Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 22,
108-111.
Garfield, E. (1983). Citation indexing: Its theory and application in science, technology and humanities. Philadelphia, PA: Wiley.
Retrieved from http://www.garfield.library.upenn.edu/ci/title.pdf
Gil-Montoya, J. A., Navarrete-Cortes J., Pulgar, R., Santa, S., & Moya-Anegon, F. (2006). World dental research production: An
ISI database approach (1999-2003). European Journal of Oral Sciences, 114, 102-108.
*Hernandez-Leo, D., Villasclaras-Fernandez, E. D., Asensio-Perez, J. I., Dimitriadis, Y., Jorrin-Abellan, I. M., Ruiz-Requies, I.,
& Rubia-Avi, B. (2006). COLLAGE: A collaborative learning design editor based on patterns. Educational Technology & Society,
9(1), 58-71.
*Hastie, M., Chen, N.-S., & Kuo, Y.-H. (2007). Instructional design for best practice in the synchronous cyber classroom.
Educational Technology & Society, 10(4), 281-294.
*Henze, N., Dolog, P., & Nejdl, W. (2004). Reasoning and ontologies for personalized e-learning in the semantic web.
Educational Technology & Society, 7(4), 82-97.
*Holzinger, A., Kickmeier-Rust, M., & Albert, D. (2008). Dynamic media in computer science education, content complexity and
learning performance: Is less more? Educational Technology & Society, 11(1), 279-290.
*Hou, H.-T., Chang, K.-E., & Sung, Y.-T. (2008). Analysis of problem-solving-based online asynchronous discussion pattern.
Educational Technology & Society, 11(1), 17-28.
*Huang, Y.-M., Jeng, Y.-L., & Huang, T.-C. (2009). An educational mobile blogging system for supporting collaborative learning.
Educational Technology & Society, 12(2), 163-175.
Hwang, G.-J., & Tsai, C.-C. (2011). Research trends in mobile and ubiquitous learning: A review of publications in selected
journals from 2001 to 2010. British Journal of Educational Tchnology, 42(4), E65-E70.
*Hwang, G.-J., Tsai, C.-C., & Yang, S. J. H. (2008). Criteria, strategies and research issues of context-aware ubiquitous learning.
Educational Technology & Society, 11(2), 81-91.
*Jovanovic, J., Gasevic, D., Knight, C. , & Richards, G. (2007). Ontologies for effective use of context in e-learning settings.
Educational Technology & Society, 10(3), 47-59.
*Karagiorgi, Y., & Symeou,L. (2005). Translating constructivism into instructional design: Potential and limitations. Educational
Technology & Society, 8(1), 17-27.

16

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
*Karampiperis, P., & Sampson, D. (2005). Adaptive learning resources sequencing in educational hypermedia systems.
Educational Technology & Society, 8(4), 128-147.
*Klamma, R., Chartti, M.A., Duval, E., Hummel, H.,. Hvannberg, E.T., Kravcik, M., Law, E., Naeve, A., & Scott, P. (2007).
Social software for life-long learning. Educational Technology & Society, 10(3), 72-83.
Klein, J. D. (1997). ETR&D – Development: An analysis of content and survey of future direction. Educational Technology
Research and Development, 45(3), 57-62.
*Knight, C., Gasevic, D., & Richards, G. (2006). An ontology-based framework for bridging learning design and learning content.
Educational Technology & Society, 9(1), 23-37.
*Kravcik, M., Kaibel, A., Specht, M., & Terrenghi, L.(2004). Mobile collector for field trips. Educational Technology & Society,
7(2), 25-33.
*Koper, R.,Olivier, B. (2004). Representing the learning design of units of learning. Educational Technology & Society, 7(3), 97-
111.
Lee, M.-H., Wu, Y.-T., & Tsai, C.-C. (2009). Research trends in science education from 2003 to 2007: A content analysis of
publications in selected journals. International Journal of Science Education, 31(15), 1999-2020.
Lee, Y., Driscoll, M. P., & Nelson, D. W. (2004). The past, present, and future of research in distance education: Results of a
content analysis. The American Journal of Distance Education, 18(4), 225-241.
Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers. Trends in Ecology & Evolution,
20(1), 28-32.
Li, L.-L., Ding, G. H., Feng, N., Wang, M.-H., & Ho, Y.-S. (2009). Global stem cell research trend: Bibliometric analysis as a
tool for mapping of trends from 1991 to 2006. Scientometrics, 80(1), 39-58.
*Liu, C.-L. (2005). Using mutual information for adaptive item comparison and student assessment. Educational Technology &
Society, 8(4), 100-119.
Liu, G.-Z., & Hwang, G.-J. (2010). A key step to understanding paradigm shifts in e-learning: Towards context-aware ubiquitous
learning. British Journal of Education Technology, 41(2), E1-E9.
*Liu, T.-C. (2007). Teaching in a wireless learning environment: A case study. Educational Technology & Society, 10(1), 107-
123.
*Makri, K., & Kynigos, C. (2007). The role of blogs in studying the discourse and social practices of mathematics teachers.
Educational Technology & Society, 10(1), 73-84.
Mao, N., Wang, M.-H., & Ho, Y.-S. (2010). A bibliometric study of the trend in articles related to risk assessment published in
Science Citation Index. Human and Ecological Risk Assessment, 16, 801-824.
*McInnerney, J.M., & Roberts, T.S. (2004). Online learning: Social interaction and the creation of a sense of community.
Educational Technology & Society, 7(3), 73-81.
McNaught, C., & Lam, P. (2010). Using wordle as a supplementary research tool. The Qualitative Report, 15(3), 630-643.
*Nichols, M. (2003). A theory for eLearning. Educational Technology & Society, 6(2), 1-10.
Noyons, E. C. M., van Raan, A. F. J. (1998). Monitoring science developments from dynamic perspective: Self-organized
structuring to map neural network research. Journal of the American Society for Information Science and Technology, 49(1), 68-
81.
Ozcinar, Z. (2009). The topic of instructional design in research journals: A citation analysis for the years 1980-2008.
Australasian Journal of Educational Technology, 25(4), 559-580.
*Paquette, G. (2007). An ontology and a software framework for competency modeling and management. Educational
Technology & Society, 10(3), 1-21.
Rourke, L., & Szabo, M. (2002). A content analysis of the “journal of distance education” 1986-2001. Journal of Distance
Education, 17(1), 63-74.
Smith, L. C. (1981). Citation analysis. Library Trends, 30, 83-106.
Shih, M., Feng, J. & Tsai, C.-C. (2008). Research and trends in the field of e-learning from 2001to 2005: A content analysis of
cognitive studies in selected journals. Computers & Education, 51, 955-967.
*Sugar, W., Crawley, F., & Fine, B. (2004). Examining teachers’ decisions to adopt new technology. Educational Technology &
Society, 7(4), 201-213.

17

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Taylor, E. W. (2001). Adult Education Quarterly from 1989 to 1999: A content analysis of all submissions. Adult Education
Quarterly, 51(4), 322-340.
*Teo, T., Luan, W. S., & Sing, C. C. (2008). A cross-cultural examination of the intention of use technology between Singaporean
and Malaysian pre-service teachers: An application of the technology acceptance model (TAM). Educational Technology &
Society, 11(4), 265-280.
Tsai, C.-C., & Wen, L. M. C. (2005). Research and trends in science education from 1998 to 2002: A content analysis of
publication in selected journals. International Journal of Science Education, 27, 3-14.
Tsai, C.-C., Wu, Y.-T., Lin, Y.-C., & Liang, J.-C. (2011). Research regarding science learning in Asia: An analysis of selected
science education journals. The Asia-Pacific Education Researcher, 20(2), 352-363.
Uzunboylu, H., & Ozcinar, Z. (2009). Research and trends in computer-assisted language learning during 1990-2008: Results of a
citation analysis. Eurasian Journal of Educational Research, 24, 133-150.
Uzunboylu, H., Eris, H., & Ozcinar, Z. (2011). Results of a citation analysis of knowledge management in education. British
Journal of Educational Technology, 42(3), 527-538.
*Virvou, M., Katsionis, G., & Manos, K. (2005). Combining software games with education: Evaluation of its educational
effectiveness. Educational Technology & Society, 8(2), 54-65.
*Wang, Q., & Woo, H. L.(2007). Systematic planning for ICT integration in topic learning. Educational Technology & Society,
10(1), 148-156.
*Wang, T.I., Tsai, K.H., Lee, M.C., & Chiu, T.K. (2007). Personalized learning objects recommendation based on the semantic-
aware discovery and the learner preference pattern. Educational Technology & Society, 10(3), 84-105.
White, M. J., & White K. G. (1977). Citation analysis of psychology journals. American Psychologist, 32, 301-305.
Wohlin, C. (2007). An analysis of the most cited articles in software engineering journals – 2000. Information and Software
Technology, 49, 2-11.
*Wolpers, M., Najjar, J., Verbert, K., & Duval, E. (2007). Tracking actual usage: The attention metadata approach. Educational
Technology & Society, 10(3), 106-121.
Woolf, B. P. (2010). A Roadmap for Education Technology. Retrieved from
http://www.cra.org/ccc/docs/groe/GROE%20Roadmap%20for%20Education%20Technology%20Final%20Report.pdf.
*Yang, S.-H. (2009). Using blogs to enhance critical reflection and community of practice. Educational Technology & Society,
12(2), 11-21.
*Yang, S. J. H. (2006). Context aware ubiquitous learning environments for peer-to-peer collaborative learning. Educational
Technology & Society, 9(1), 188-201.
*Yang, S. J. H., Chen, I. Y.-L., & Shao, N. W. Y. (2004). Ontology enabled annotation and knowledge management for
collaborative learning in virtual learning community. Educational Technology & Society, 7(4), 70-81.
*Yin, P.-Y., Chang, K.-C., Hwang, G.-J., Hwang, G.-H., & Chan, Y. (2006). A particle swarm optimization approach to
composing serial test sheets for multiple assessment criteria. Educational Technology & Society, 9(3), 3-15.
*Yukselturk, E., & Bulut, S. (2007). Predictors for student success in an online course. Educational Technology & Society, 10(2),
71-83.
*Zurita, G., Nussbaum, M., & Salinas, R. (2005). Dynamic grouping in collaborative learning supported by wireless handhelds.
Educational Technology & Society, 8(3), 149-161.

18

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Appendix 1
Top 20 highly cited ET&S papers (by citation counts in total, as of November 30, 2011) during 2003-2010
Rank Citation Title Author(s) Country Published Research
counts year /page type
number
1 84 Representing the learning design of units Koper, R., The 2004/7(3), Theoretical
of learning Olivier, B. Netherlad 97-111 paper
s, U.K.
2 45 Context aware ubiquitous learning Yang, S. J. H. Taiwan 2006/9(1), System
environments for peer-to-peer 188-201 evaluation
collaborative learning
3 36 COLLAGE: A collaborative learning Hernandez- Spain 2006/9(1), Empirical
design editor based on patterns Leo, D., 58-71 study (mixed
Villasclaras- method)
Fernandez, E.
D., Asensio-
Perez, J. I.,
Dimitriadis,
Y., Jorrin-
Abellan, I.
M., Ruiz-
Requies, I.,
Rubia-Avi, B.
4 36 Adaptive learning resources sequencing Karampiperis, Greece 2005/8(4), System
in educational hypermedia systems P., Sampson, 128-147 evaluation
D.
5 31 Dynamic media in computer science Holzinger, A., Austria 2008/11(1), Empirical
education, content complexity and Kickmeier- 279-290 study
learning performance: Is less more? Rust, M., (quantitative
Albert, D. method)
6 30 Reasoning and ontologies for Henze, N., Germany 2004/7(4), System
personalized e-learning in the semantic Dolog, P., 82-97 evaluation
web Nejdl, W
7 29 The new challenges for e-learning: The Aroyo, L., The 2004/7(4), System
educational semantic web Dicheva, D. Netherlan 59-69 introduction
ds, U.S.A.
8 28 Combining software games with Virvou, M., Greece 2005/8(2), Empirical
education: Evaluation of its educational Katsionis, G., 54-65 study (mixed
effectiveness Manos, K. method)
9 25 Towards a pattern language for learning Avgeriou, P., Greece, 2003/6(2), Other
management systems Papasalouros, Cyprus 11-24
A., Retalis,
S.,
Skordalakis,
M.
10 24 Ontology enabled annotation and Yang, S. J. Taiwan 2004/7(4), System
knowledge management for collaborative H., Chen, I. 70-81 evaluation
learning in virtual learning community Y.-L., Shao,
N. W. Y.
11 22 Dynamic grouping in collaborative Zurita, G., Chile 2005/8(3), Empirical
learning supported by wireless handhelds Nussbaum, 149-161 study
M., Salinas, (quantitative
R. method)
12 20 Criteria, strategies and research issues of Hwang, G.-J., Taiwan 2008/11(2), Other
context-aware ubiquitous learning Tsai, C.-C., 81-91
Yang, S. J. H.
13 20 Examining teachers’ decisions to adopt Sugar, W., U.S.A. 2004/7(4), Empirical
new technology Crawley, F., 201-213 study (mixed
Fine, B. method)
14 19 A theory for eLearning Nichols, M. New 2003/6(2), 1- Theoretical

19

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms
Zealand 10 paper
15 19 Using mutual information for adaptive Liu, C.-L. Taiwan 2005/8(4), Other
item comparison and student assessment 100-119
16 18 Mobile collector for field trips Kravcik, M., Germany 2004/7(2), 25- System
Kaibel, A., 33 evaluation
Specht, M.,
Terrenghi, L.
17 18 Interoperability in personalized adaptive Aroyo, L., The 2006/9(2), 4- System and
learning Dolog, P., Netherlan 18 model
Houben, G.- ds, evaluation
J., Kravcik, Germany,
M., Naeve, Belgium,
A., Nilsson, Sweden,
M., Wild, F. Austria
18 16 An ontology-based framework for Knight, C., Canada 2006/9(1), 23- Theoretical
bridging learning design and learning Gasevic, D., 37 paper
content Richards, G.
19 16 A particle swarm optimization approach Yin, P.-Y., Taiwan 2006/9(3), 3- System
to composing serial test sheets for Chang, K.-C., 15 design
multiple assessment criteria Hwang, G.-J.,
Hwang, G.-
H., Chan, Y.
20 16 PERKAM: Personalized knowledge El-Bishouty, Japan 2007/10(3), Empirical
awareness map for computer supported M.M., Ogata, 122-134 study
ubiquitous learning H., Yano, Y. (quantitative
method)

20

This content downloaded from 86.59.13.237 on Sat, 10 Dec 2016 05:48:41 UTC
All use subject to http://about.jstor.org/terms

You might also like