You are on page 1of 25

Developing the APTIS Testing

System
Professor Barry OSullivan
Senior Advisor, English Language Assessment
Overview

Developing theories

Localisation

Computerisation

Conclusions
conceptualising the test

DEVELOPING THEORIES
System Design
Thinking about Test Performance
Task
Performance Social Factors
Parameters

Target Personal
Language Characteristics

Cognitive
Task Language PERFORMANCE
Characteristics
Then Theres Scoring

Accuracy

Selection &
Training Consistency

Answer Key or
Rating Scale

Machine or
Human Interpretation
Validation Theory
Decisions, Decisions

Why are we
testing?
How Many Decisions?
Social Factors
Interlocutor [who;
number]
Task Relative Age, gender
Performance etc.
Personal
Characteristics
Parameters Familiarity & Status
Etc. Physical
Time & Support
Psychological
Knowing Assessment
Criteria Experiential
Physical Conditions, etc.
Target Cognitive
Language Characteristics
Vocabulary &
Maturity
Grammar Ability
Function, Mode etc. Etc.

Task Language
Vocabulary &
Grammar
TEST Scoring System
Raters & Rating
Scales & Keys
Genre, Mode, etc.
SCORE Analysis & Grading
Etc.
Why all the Fuss?

Curriculum
explicit
implicit

Decisions

Delivery Assessment
room design judgemental
materials developmental
teachers
theories of language progression

FOCUS ON READING
The Reading Model
How it Works
C Inter Text
[full text level]

Inter Text
B [paragraph level]

Text [external]

Text [within]

A Paragraph

Word/Sentence
Introducing

CORE
[Grammar & Vocabulary]

Writing

Reading

Speaking

Listening
the british council approach

LOCALISATION
Theory to Practice Test Design
Appropriate Test
Content Taker
Appropriate
Topic

Appropriate Test
Task Task
Appropriate
Language

Appropriate Scoring
Scoring System
LOCALISATION
Defining Localisation

The process of ensuring that all construct-irrelevant


variance related to the test taking population is
controlled for in the design and delivery of a test

Testing a well defined population within a well defined context


in order to make decisions that will apply only to that context
Localisation Case Study

Curriculum

Delivery Assessment
Case Study - Approach

Context Localisation
Population 15 & 17 yrs Language Reviewed
Secondary School [inc. instructions]
Mixed ability, Gender Reading Topics
Various L1 [multilingual] Listening Topics
Similar Culture Writing Topics/Tasks
Similar Curriculum Speaking Topics
Similar Delivery Speaking Images
the growing importance of the machine

COMPUTERISATION
The Computer & Test Practice
Research
Language
Data Analysis
Task Performance
Rating Performance

Item Development
Test Taking Specification
Test Scoring Knowledge Writing
Rater Training Receptive Skills Trialling
Test Break Up Productive Skills Banking
Scoring Archiving
Analysing/Awarding
Score Reporting System Management Test Delivery
Back Office Computer
Reporting to Clients Tablet
Other
The Computer & Test Practice

Item Development
Specification
Writing
Trialling
Banking
Archiving
whats the point?

CONCLUSIONS
The Message?
Every decision impacts on ALL stakeholders, though
especially on the test taker

Without a clearly defined ABILITY MODEL the test is


suspect

Without a clearly defined VALIDATION MODEL the test


is suspect

It is close to impossible to deliver any test without


technology
Professor Barry OSullivan

barry.osullivan@britishcouncil.org

You might also like