You are on page 1of 9

DEVELOPING PROJECT ASSESSMENT INSTRUMENTS FOR

SPEAKING SKILL OF THE EIGHTH GRADE STUDENTS


OF JUNIOR HIGH SCHOOL
Gede Mahendrayana, A. A. I. Ngurah Marhaeni, Nyoman Dantes
English Education Study Program, Post-graduate Program
Ganesha University of Education
Singaraja, Indonesia
e-mail: gede.mahendrayana@pasca.undiksha.ac.id;
agung.marhaeni@pasca.undiksha.ac.id; nyoman.dantes@pasca.undiksha.ac.id
Abstract
This study aimed at developing project assessment instruments for speaking skill of the eighth grade
students of junior high school. Specifically, this study is concern with: (i) finding out the types of project
assessment needed to be developed (ii) developing prototype of developed project assessment
instruments (iii) examining the quality of the developed project assessment instruments. This study used
a research design of Gall, Gall and Borgs model (2003) in developing project assessment instrument.
First, a need analysis was done in the form of distributing questionnaire to the 13 English teachers of
junior high school, interview, and documentation. Based on the result of analysis, it was decided to
develop project assessment instruments consisted of two main aspects, such as (1) linguistic aspect, and
(2) non-linguistic aspect. After the project assessment instruments were developed, expert and user
judgments were done to examine the validity and the quality of the instruments. The result of expert and
user judgments showed that the developed project assessment instruments have very high validity and
belong to excellent instruments.
Key words: Project Assessment Instrument, Speaking Skill

INTRODUCTION
The purpose of education is to provide
learning experiences including cognitive,
affective, and psychomotor significantly,
which serves to prepare learners to face the
era of globalization where the competition is
getting tougher. It means that education is
something that must be had in order to be
able to survive and compete in this era of
globalization. Through education, every
individual should be given opportunity for
lifelong learning in order to develop the
knowledge, skills, and attitudes as well as
to be able to adjust to the outside world
which is increasingly competitive and
complex.
In education world, language is the
key to success in studying all fields of study
(Depdiknas, 2004a). It means that the
language has very important role in learning
of all fields of study and in the interaction
between teachers and learners in the
classroom. Language can also serve as a

tool to convey ideas between teachers and


students in learning both inside and outside
of the classroom. One way to convey these
ideas is in the form of speaking.
Speaking is a key to communication.
According to Brown (1994) speaking is an
interactive process of constructing meaning
that involves producing, receiving, and
processing information. He also states that
form and meaning depend on the context in
which they occur, including the participants
themselves, their collective experiences,
the physical environment, and the purpose
of speaking. Speaking enables students to
receive information from people through
their conversation, after which they should
understand the information and respond to
it or communicate their understanding of the
information. Among the macro skills of
language, it has been widely recognized
that speaking, particularly in a second or
foreign language, is the most difficult

language skill to assess. This is due, for the


most part, to difficult matching of the testing
goals and the appropriate instruments and
tasks for assessment (Luoma, 2004).
According to Brown (2004) generally, there
are at least five aspects of speaking skill
that need to be considered in assessing
speaking skill, namely: (1) comprehension,
(2)
grammar,
(3)
vocabulary,
(4)
pronunciation, and (5) fluency.
Moreover, School Based Curriculum
(SBC) suggests that during the teaching
and learning process of speaking, the
teachers should be conducting an authentic
assessment. Authentic assessment is a
form of assessment in which students are
asked to perform real-world task that
demonstrate meaningful application of
essential knowledge and skills. Students
performance on a task is typically scored on
a rubric to determine how successfully the
student has met specific standard (Mueller,
2005). Marhaeini, 2012 defines authentic
assessment as a process of collecting data
of students, either during the learning
process or at the end of the learning. In this
case, the authentic means real as what
happens in real life. Therefore, authentic
assessment refers to an assessment
requiring the students to do authentic task
that represent the application of their
knowledge and skills.
One of the authentic assessment
types which is appropriate to assess
speaking skill and good to be applied in the
application of SBC is project assessment.
The project assessment is a type of
assessment which investigates real topics
in deep investigation (Marhaeni, 2012).
Project is defined as complex tasks based
on problem encountered by students,
conducted in certain periods of time and
culminated in realistic products that might
be in the form of presentation, exhibition,
publication, performance, etc. (Thomas,
2000). The project is supposed to be longterm, requires teamwork among students,
and results in a substantial final product
(Thompson & Beak, 2007, as cited in Cruz
& Vik, 2007). It means that not every task
can be considered as project. The work
includes investigation on a phenomena or a
case which starts from planning, data
collection, organization, data analysis and

report. In using project as a basic in the


learning
process,
students
have
opportunities to use several skills (e.g.
problem-solving, creativity, teamwork, selfmanagement, respect, as well as language)
at different work stages, so the work and
language skills are developed (Brunetti,
Petrell, Sawada, 2003; Solomon, 2003).
In the implementation of School
Based Curriculum, the teachers are
suggested to use authentic assessment,
particularly for project assessment, for its
many benefits that standardized test do not
have. The benefits that can be given by
project assessment to the students are (1) it
can expand students thinking which is
useful to face the problem of the life, (2) it
can foster the students to practice applying
the knowledge, attitudes, and skills in daily
life in an integrated, (3) this technique of
assessment in accordance with the
principles of modern didactic which is need
to be considered in teaching, (4) it can build
learners confidence, self-esteem, and
autonomy as well as improving students
language skills, content learning, and
cognitive abilities (Solomon, 2003; Srikrai,
2008; Stoller, 1997), and (5) it can make
learning becomes fruitful for learners
because they exhibit their abilities to plan,
manage, and accomplish projects through
their content knowledge and language skills
(Kloppenborg & Baucus, 2004).
However, the implementation of
authentic assessment which is expected is
very different from what happened on the
real field. The result of observation that had
been done in some junior high school in
Buleleng Regency showed that There were
some aspects about the readiness of
teachers
in
implementing
authentic
assessment,
particularly
for
project
assessment,
which were
far from
expectations. It can be seen from the
implementation of authentic assessment in
the classroom, in which the students
participation in assessment activities was
not optimal. The problems which are faced
teachers
in
implementing
project
assessment in the teaching learning
process particularly for speaking skill are:
(1) the irrelevance of learning activities
toward expected skills affected the
irrelevance of assessment type which was

used to assess the expected skill, (2) the


unclear of assessment rubric provided
causing the measurement of students
language ability became incomplete (3) the
teachers
mastery
about
project
assessment was still lacking, and (4) the
lack of availability of project assessment
instruments for speaking skill.
Based on the above description, it can
be said that there was a huge gap in the
ability of teachers to implement authentic
assessment,
especially
for
project
assessment, integrating with the learning
process. On the other hand, authentic
assessment is a device that must be
mastered by teachers both in the design
and its implementation. The teachers are
not able to implement project assessment
for speaking skill properly is due to the lack
of
concept
mastery
about
project
assessment and the lack of availability of
project
assessment
instruments
for
speaking skill. Therefore, this research was
conducted to develop project assessment
instruments that can be used by teachers in
managing teaching speaking skill in the
school.
Based on the background of study
described above, this study aimed at: (1)
finding out the types of project assessment
needed to be developed, (2) developing
prototype of developed project assessment
instruments (3) examining the quality of the
developed project assessment instruments.
RESEARCH METHOD
This study was categorized into a
research and development study. It was
aimed at developing project assessment
instruments for speaking skill. According to
Borg and Gall (1983) research and
development is a process of developing and
validating educational product. Moreover,
Gall, Gall, and Borg (2003) state that
research and development refers to
evaluating material in which the findings of
the evaluation are used to develop a new
product. This study used Gall, Gall and
Borgs (2003) model which was simplified to
be a modified model of research design.
There were seven stages involved in the
modified research design of Gall, Gall and
Borg (2003), namely: (1) need analysis, (2)
planning and designing the prototype, (3)

developing project assessment instrument,


(4) expert judgment, (5) revision, (6) Field
try out, and (7) final product revision.
The research subject in this study was
the sources where the data were obtained.
There were three subjects of this research,
namely English teachers of junior high
school (SMP) in Buleleng Regency,
syllabus of English Subject for grade eight,
and the eighth grade students of Junior
High School. Meanwhile, the research
object of this study was the project
assessment instruments for speaking skill
that were developed in this study.
The data for need analysis were
collected by using method of questionnaire,
interview,
observation,
and
syllabus
analysis.
The
questionnaires
were
distributed to 13 English teachers in
Buleleng Regency while also doing
unstructured interviewed with them. The
interview was done for the confirmation of
the data from the questionnaires if those
data valid or not. Meanwhile, in order to
examine the quality of the developed
project assessment instruments, expert and
user judgments were done toward the
project assessment instruments that had
been developed.
Based on the method of data
collection described above, there were
three instruments which were used in
collecting the data in this study, such as
questionnaire for need analysis, interview
guide, and questionnaire to measure the
quality of developed instruments. A form of
questionnaire for need analysis was
designed in order to be used to collect data
from English teacher of junior high school.
The questionnaire consisted of 40 items
which were divided into five aspects,
namely (1) teachers knowledge about
teaching speaking, (2) teachers knowledge
about speaking assessment, (3) teachers
knowledge about project assessment, (4)
teachers expectation of project assessment
for speaking skill, and (5) teachers
readiness in using project assessment for
speaking skill. Besides a questionnaire, A
form of interview guide was also designed
to collect the data from English teacher of
junior high school. There were 10 questions
prepared in the interview guide that include
five aspects as in the questionnaire for

need analysis. Meanwhile, the second


questionnaire was designed to measure the
quality
of
the
project
assessment
instruments which were developed. The
questionaire was developed based on the
criteria of good assessment instrument in
which a good assessment instrument
should be valid and reliable (Marhaeni,
2012).
The data from the questionnaire and
interview for the need analyzed were
calculated to found the result of
questionnaire in form of percentage. From
this percentage, it was the basic for drawing
conclusion of the research in the form of
descriptive explanation. Different with
questionnaire and interview for the need
analysis, the data from the expert judgment
were analyzed quantitatively using the
formula of Gregory validity (adopted from
Gregory, 2000). Meanwhile, The data from
the user judgment were analyzed by using
Nurkancana and Sunartana (1992) formula
for identifying the quality of the developed
project assessment instruments.
FINDINGS AND DISCUSSION
Project Assessment needed by EFL
Teachers for Speaking Skill
As stated before that this study used
the modified research design of Gall, Gall
and Borgs (2003) model in which at the first
stage there should be focused on doing
need analysis related to the implementation
of project assessment in teaching speaking.
There were three activities done in the need
analysis, namely (1) syllabus analysis, (2)
administering questionnaire, and (3)
interview.
The result of syllabus analysis
showed that there were 12 speaking topics
from four standard competencies were
determined to be developed with project
assessment instruments. The standard and
basic competencies were taken from
content standard of junior high school in
BSNP (2006). From twelve topics, there
were five topics for the first semester of the
eighth grade of junior high school and
seven topics for the second semester of the
eighth grade of junior high school. Those
topics are: (1) Asking, Giving, Rejecting
Helps and Things; (2) Asking for
Someones Opinion; (3) Inviting, Receiving,

and Rejecting Invitation; (4) Descriptive


Text; (5) Recount Text; (6) Asking, Giving,
Rejecting Helps and Things; (7) Asking,
Giving, Rejecting for Information; (8) Asking
for Someones Opinion; (9) Agreeing and
Disagreeing; (10) Starting and Closing
Conversation on Telephone; (11) Recount
Text; and (12) Narrative Text. All those
topics were chosen with the consideration
that the form of the conversations and texts
for these topics have range of materials
which are not too wide and not too narrow
and appropriate developed with project
assessment.
Moreover, the finding of questionnaire
and interview showed that speaking is a
difficult and complex competency since
speaking involves the process of thinking,
organizing, and expressing the ideas orally
in English. For the currents level of
students language proficiency, teaching
them to speak was very difficult. This
finding was in line with Richards and
Renandya (2002) stating that speaking a
language is especially difficult for foreign
language learners because effective oral
communication requires the ability to use
language
appropriately
in
social
interactions. According to them, speaking is
one on the elements of communication in
which communication is the output modality
and learning is the input modality of
language acquisition.
However, the teachers believed that
speaking assessment should be done
authentically. The speaking assessment
should link the students to the real world.
So, when the students leave the class, they
were ready to face the real world. This
finding supports as what was stated in
School Based Curriculum in which the
teachers should be conducting an authentic
assessment in the learning process.
Authentic assessment is a form of
assessment in which students are asked to
perform real-world task that demonstrate
meaningful
application
of
essential
knowledge and skills.
Even though authentic assessment
was not a new issue for the teachers, in
fact, the result of the questionnaire and
interview also showed that the teachers did
not have enough knowledge about project
assessment, as one kind of authentic

assessment. This finding was in line with


the result of the study conducted by
Wahyuni (2012) in which it was found that
the teachers only have a little knowledge of
authentic assessment. However, the
teachers have a high expectation to use
project assessment in the teaching of
speaking. Considering to these findings, it
could be concluded that project assessment
instruments for speaking skill were needed
to be developed as guidance for assessing
speaking skill.

The Prototype of Project Assessment for


Speaking Skill
After the need analysis were
conducted
and
a
conclusion
was
determined based on the results of need
analysis, then a prototype of project
assessment instrument was designed. The
prototype was used to develop project
assessment instruments for speaking skill
of the eighth grade students of junior high
school. The following Figure shows the
design of the prototype of project
assessment instruments which were
developed in this study.

Figure 01. The Prototype of Project Assessment Instrument for Speaking Skill

After the prototype of project


assessment instruments was designed, the
project
assessment
instruments
for
speaking skill of the eighth grade students
of junior high school were developed based
on the prototype above. There were twelve
project assessment instruments developed
in this study. The type of the developed
project assessment in this study is
considered
appropriate
for
teaching
speaking competency. Each instrument
consisted of blueprint, assessment rubric,
and scoring guide which was developed
based on the topic and the speaking task
that had been determined. The project
assessment instrument that had been
developed in this study consisted of two
main aspects, namely (1) Linguistics aspect
and (2) Non-linguistics aspect. These to
aspects were carefully decided by
considering the finding in the school and
discussing them with the experts.
The linguistic aspects of project
assessment instrument are intended to
assess the student speaking which cover
five aspects namely: (1) pronunciation, (2)
grammar, (3) vocabulary, (4) fluency, and
(5) comprehension. Each of this aspect was
developed as guidance for the students in
finishing their speaking successfully. It is in
line as what was stated by Brown (2004)
that there are at least five components of
speaking skill that need to be considered in
assessing speaking skill, namely: (1)
comprehension, means that communication
certainly requires a subject to respond to
speech as well as to initiate it; (2) grammar
meaning that use the sentence pattern
correctly; (3) vocabulary, means that use
the related vocabulary; (4) pronunciation
which means that pronounce the words with
the correct sounds, uses correct tones; and
(5) fluency meaning that speaking is fluid
and has emotion and pause properly.
On the other hand, Non-linguistic
aspects of project assessment are used to
assess the students progress in the
process of doing the project task. It is based
on the statement proposed by Stoller (1997)
in which the learning which based on the
project should be oriented on the process
and product. There are four aspects
involved in the non-linguistics aspect of
project assessment instrument; those are

(1) self-management, (2) communication,


(3) respect, and (4) group cohesiveness.
These non-linguistic aspects support the
statement proposed by Brunetti, Petrell,
Sawada, 2003 and Solomon, 2003, in which
in conducting the project assessment the
students have opportunities to use several
skills (e.g. problem-solving, creativity,
teamwork, self-management, respect, as
well as language) at different work stages,
so the work and language skills are
developed.
These all aspects are used to assess
the students progress in doing the
speaking task as well as their speaking
performance. The students were asked to
accomplish the project task for each topic of
speaking competency. In accomplishing the
project task given, the students were
involved in the three stages of the project
task completion, such as (1) planning
phase, (2) developing phase, and (3) Final
phase. In the planning phase, the students
were allowed to determine the topic and
prepare the material that would be used in
doing the project task. Therefore, in the
assessment rubric of project assessment
instrument, it was determined two indicators
of non-linguistic aspects to assess the
planning phase in doing the project task.
The next stage after the planning phase
was developing phase, in which the
students started to carry out the project task
given in group and accomplish the task in
certain time period. In order to assess the
students in accomplishing the project task,
there were eight indicators of non-linguistics
aspects were determined in the assessment
rubric of project assessment instrument
especially for developing phase.
For the final phase, the students
should perform their speaking competency
based on the speaking task given as well as
assessment task that had been determined
by the teacher. It is in line with Thomas
(2000) who defines a project as complex
tasks based on problem encountered by
students, conducted in certain periods of
time and culminated in realistic products
that might be in the form of presentation,
exhibition, publication, performance, etc.
Therefore, the five indicators of linguistic
aspect were determined in the assessment
rubric of project assessment instrument

especially for final phase to assess the


students competency in speaking.
Besides develop project assessment
instruments, there was also developed a
compilation of lesson plan which consisted
of twelve lesson plans which involved all
topics of speaking competency that had
been determined in the syllabus analysis.
Each lesson plan takes one of the topics of
speaking competency which are included
this study. These lesson plans could assist
the teachers in giving description where and
when the project assessment instruments
can be appropriately used in the learning
activities. Therefore, each lesson plan was
designed by providing learning activities
and speaking task for each topic developed
in this study.

shows the range score from 50 52. And if


it is inserted to the interval coefficient
proposed by Nurkancana and Sunartana,
the scores belong to the first category that

The Quality of the developed Project


Assessment Instruments for Speaking
Skill of the Eighth Grade Students of
Junior High School
In order to identify the quality and the
validity of the project assessment
instruments that had been developed, two
expert judges and two user judges were
asked to evaluate them. The result from the
expert judges revealed that the developed
self-assessment was categorized as
instruments with a very high validity. This
conclusion was obtained from the data
analysis in which each instruments has 1.00
point of content validity. If it is inserted into
the interval coefficient, this point of content
validity belongs to the first category (0.76
1.00 1.00). This category is interpreted as
very high validity. Marhaeni (2012) mention
that there are two aspects as to be
requirement
of
good
assessment
instruments, namely: validity and reliability.
After the result of expert judgment was
found in which all instruments have very
high validity and reliability. It means that the
developed project assessment instruments
can be used for teaching speaking
competency since the developed project
assessment instruments have fulfilled the
criteria of good assessment.
Meanwhile, the result from the user
judges revealed that the quality of the
developed project assessment instruments
was in the category of excellent
instruments. The result of the data analysis

CONCLUSION AND SUGGESTION


Based on the result of the study as
well as the findings which were found in the
field, it can be drawn some conclusions.
First, the project assessment instruments
for speaking skill of the eighth grade
students of junior high school needed to be
developed involving assessment task,
blueprint of assessment rubric, project
assessment rubric (linguistic and nonlinguistic), and scoring guide of project
assessment. Besides, there were also
needed to develop lesson plan for each
topic to allocate where and when the project
assessment
instruments
can
be
appropriately used in the learning activities.
Second, a prototype was developed based
on the need analysis and the grand theory
used in this study. This prototype was used
as basic to develop the project assessment
instruments. Third, the result of evaluation
from expert judgment and user judgment
showed that the validity of the developed
project assessment instruments was very
high.
Meanwhile, the quality of the
instruments is considered as excellent
quality since the data analysis from the user
judgment categorized the instruments into
the first category of Nurkancana and
Sunarta formula.
Based on the conclusion presented
above, it can be proposed some
suggestions in order to improve the
implementation of authentic assessment,

is 48 X 60, where X is the score from


the user judges. This second category is
categorized as excellent instruments. From
this
result,
the
developed
project
assessments is considered as excellent
instruments
for
teaching
speaking
competency for the eighth grade students of
junior high school.
By seeing the result of the evaluation
from the expert judgment and user
judgment, it can be concluded that the
project assessment instruments which were
developed in this study are valid and can be
used in the classrooms.

especially for project assessment in


teaching learning process, particularly for
teaching speaking.
For the teachers, it is suggested to
use the project assessment instruments
that have been developed in this study for
teaching speaking in the class. Since the
instruments accompanied with the lesson
plan, teachers will not have problems in
implementing it. Since the development of
project assessment instrument was just on
the stage of evaluation validity and quality,
it is also suggested for the others
researchers to do further study to find out
the significant effect of this instruments.
Besides, it is also suggested for the policy
makers to start the implementation of
authentic assessment such using the
developed project assessment instruments
of this study. This is an attempt to build an
effective and successful learning as well as
to find out the effectiveness of the
developed project assessment instruments.
REFERENCES
Borg,

W.R. and Gall, M.D. 1983.


Educational
Research:
An
th
Introduction (4 Ed.). New York, NY:
Longman, Inc.

Brown, H.D. 1994. Teaching by Principle:


An Interactive Approach to Language
Pedagogy. New Jersey: Prentice HallInc.
Brown, H.D. 2004. Language Assessment.
Principle and Classroom Practice.
Longman.
Brunetti, A.J., Petrell, R.J., Sawada, B.
2003. Team Project-based Learning
Enhances Awareness of Sustainability
at the University of British Columbia,
Canada. International Journal of
Sustainability in Higher Education, 4,
210.
BSNP. 2006. Standar Isi untuk Satuan
Pendidikan Dasar dan Menengah:
Standar Kompetensi dan Kompetensi
Dasar SMP/MTs. Jakarta: BSNP.

Cruz, H.A. and Vik, G.N. 2007. Using


Project-Based Learning to Connect
Theory to Practice in Teaching
Accounting Communication. Journal
of
Association
for
Business
Communication Annual Convention.
Depdiknas.
2004a.
Materi Pelatihan
Terintegrasi Bahasa Inggris. Jakarta:
Depdiknas Dirjen Pendas Direktorat
Pendidikan Lanjutan Pertama.
Gall, M.D., Gall, J.P., and Borg, W.R. 2003.
Educational
Research:
An
th
Introductioin (7 Ed.) Boston: Allyn
and Bacon.
Gregory, R.J. 2000. Psychological Testing
History: Principle and Application (3rd
Ed.) USA: Allyn and Bacon. Inc.
Kloppenborg, T.J. and Baucus, M.S. 2004.
Project
Management
in
Local
Nonprofit organizations: Engaging
Students in Problem-Based Learning.
Journal of Management Education,
28, 610 630.
Luoma, Sari. 2004. Assessing Speaking.
New York: Cambridge University
Press.
Marhaeni.
A.A.I.N.
2012.
Language
Assessment for Masters Program of
English Language Education. Post
Graduate
Program,
Ganesha
University of Education.
Mueller,
J.
2005.
The
Authentic
Assessment
Toolbox:
Enhancing
Student learning through online
faculty development, published in the
Journal of Online Learning and
Teaching
(2005).
http://jonathan.mueller.faculty.noctrl.e
du./ (downloaded on October 10th
2013)
Nurkancana, W. and Sunartana PPN. 1992.
Evaluasi Hasil Belajar. Surabaya;
Usaha National.

Richard, J.C. and Renandya, W.A. 2002.


Methodology in Language Teaching:
An Anthology of Current Practice.
Cambridge: Cambridge University
Press.
Solomon, G. 2003. Project-based Learning:
A Primer. Journal of Technology &
Learning 23, 20-27.
Srikrai, P. 2008. Project-based Learning in
an EFL Classroom. Journal of
Humanities and Social Sciences,
Khon Kean University, 25, 85 111.
Stoller, L.S. 1997. Project Work: A Means
to Promote Language Content,
Forum,
35(4),
2

18,
http://eca.state.gov/forum/vols/vol35/n
o4/p2.htm (downloaded on July 2nd
2014)
Thomas, J.W. 2000. A Review of Research
on
PBL.
http://www.bobpearlman.org/BestPrac
tices/PBL.Research.pdf. Downloaded
on July 2nd 2014.
Wahyuni, L.G.E. 2013. Analysis of the
Authenticity of Teachers Made
Assessment and Its Contribution on
Students English Achievement (A
Study in Vocational School in
Buleleng Regency. Thesis Language
Education Study Program, Post
Graduate
Program,
Ganesha
University of Education, Singaraja,
Bali.

You might also like