Page 1
www.e-mesh.com
teachers’ centre
articles
testing
DESIGNING LANGUAGE TESTS
The Test of Interactive English
By Gronia de Verdon Cooney
Teachers know about tests – they are part of their
arsenal. Some teachers know more than others, but
then, some teachers are more attack-minded than
others. They can call up tests as reinforcements
against learners who refuse to toe the line, or keep
them in reserve as a hidden threat, an ambush to be
unleashed on the recalcitrant. Friday tests, unit tests,
end of term tests – these are the side-arms of the
teacher’s kit, often supplied primed and ready for use
by the publishers, while the big guns are provided by
the monumental Examination Boards. Most students,
faced with an internationally recognised, universally
available language examination, have no trouble
identifying with the six hundred: Their’s (sic) not to
reason why, Their’s but to do and die, Into the valley of
Death … If you think this is going too far, ask yourself
why we talk about a ‘battery of tests’.
Tests, of course, come in many shapes and sizes.
Placement tests do just what the name asserts: they
enable administrators to group students in more or
less appropriate classes. They are frequently fairly
blunt instruments, but this does little harm because
learners can usually be moved very quickly to a more
suitable group if necessary. All a placement test does
is string students out like birds on a telephone wire,
some closer together and some farther apart. In itself,
it tells us relatively little about students as individuals.
To find out about students’ strengths and weaknesses,
we need a diagnostic test. This tells us what the
learners can, and more importantly for the teacher,
can’t do. Placement tests rarely have a diagnostic
function; one exception is the Oxford Placement Test.
Familiar to many teachers is the achievement test,
taken on Fridays, or as a progress test after five units,
or some such interval. This kind of test measures
whether the students have learned what the teacher
has been teaching, or, more to the point, whether the
learners can reproduce it accurately enough to satisfy
the teacher that they know it, which, as we all know, is
not always the same thing.
Then there is the proficiency test, which extrapolates
from a sample of language the learners’ ability to use
the language. These tests are usually independent of
particular courses; they ask not ‘How well can the
students remember the contents of book x?’ but rather
‘How well have the students mastered the systems of
the language?’ – although it is sometimes hard to
differentiate between these two questions!
The final category of test types is the performance
test, which asks ‘What can we see and hear the
students do with the language in a given situation?’
Workplace related tests usually fall into this category.
When teachers have to write tests, they are usually
progress or achievement tests for a particular class.
The teacher knows all the students individually and
any anomalies in the final mark can be massaged by
using continuous assessment or discretionary
marking. Few teachers have the chance – or even
desire – to be involved in designing a test to be
available on a national level for large numbers of
unknown students. This is what I want to talk about
now.
In 1995 the Advisory Council for English Language
Schools in Ireland started a test development project.
The stated aim was to design a test that would be
available to all students taking EFL courses in Ireland,
encourage students to interact with the local
environment, and encourage ‘best practice’ in
classroom teaching methods; and that would facilitate
both learner autonomy and lifelong learning as
promulgated by the Council of Europe. For six months
after the initial meeting (a weekend seminar with
Richard West) the discussion raged. Hotly debated
questions included:
Page 2
www.e-mesh.com
teachers’ centre
articles
testing
•
Who is the test for? Young learners?
Teenagers? Adults? University students?
•
How long does it take?
•
What age group is it suitable for?
•
How many sub-test are there? How many
‘papers’?
•
Are there multiple choice questions?
•
Are
there
reading
and
listening
comprehension questions?
•
How do we test grammar?
•
How many levels does the test discriminate?
•
Are there separate tests at Beginner,
Elementary, Intermediate, and Advanced
levels?
•
Can we have separate versions for ESP?
•
Who writes the descriptors and marking
schemes?
•
How many test items do we need?
Gradually it became clear that the group shared
certain ideals:
•
Grammar tests are bad;
•
Discrete-point
multiple-choice
questions
induce anxiety; furthermore, the examiner is
always right!
•
Tests don’t have to be traumatic;
•
Students know why they are learning English
– we don’t’.
This realisation lead to the development of a single,
learner-centred, interactive, task-based, student-led
test of language performance – the Test of Interactive
English (TIE). All candidates enter for the same test
and carry out the same tasks; their result is like a rung
on a ladder – their performance is assessed and the
result is expressed in terms of the six bands of the CEF
scale.
Most teachers nowadays seem to subscribe to a
‘learning by doing’ philosophy of language learning,
so instead of saying that ‘candidates must learn certain
things’ we decided that ‘learners must do certain
things’ – in other words, a task-based test that is
congruent with the tasks and activities of most
language classrooms.
•
The candidates read a book. This is not a set
book: the candidates may choose any book
of interest or relevance to their life and
studies.
•
Then, candidates follow a news story in the
English language media over a short period
of time (most students come to Ireland for
less then four weeks).
These two tasks involve students in reading and
listening, and they are motivated to persevere because
they have chosen interesting topics.
•
Finally, candidates investigate something
that has attracted their attention, which may
be local history or an Irish aspect of
something they already know. This ’project’
usually involves interviewing people, using
research resources (libraries or internet) and
any other activities that are relevant to the
task in hand.
Thus the students have employed the skills of reading,
listening and speaking, and recorded the results in
writing in a dossier or log book which is evidence of
exam preparation. We chose these tasks because
these are the things students do naturally when they
come to Ireland to study. We wanted the test to be
relevant to the real, day-to-day lives of our potential
candidates.
Now, readers who are still thinking in terms of
traditional language examinations will be asking
themselves how we write comprehension questions for
all the books that candidates might present, or how we
manage listening comprehension on all the news
media. The answer, of course, is that we don’t. In fact,
we don’t actually test listening and reading at all. It is
sufficient for the purpose of the Test of Interactive
English that the candidates have done them. What we
do test is speaking and writing: the candidates talk
and write about their personal responses to the
materials they have prepared, with the aid of their
dossiers (because we are not testing their memory but
their ability to use language). While our consultants
Richard West and Dave Allan have enthusiastically
endorsed this decision, people more familiar with
traditional tests find the TIE format somewhat radical.
Students, however, love it! This brings me to a
description of the test itself.
The test is in two parts – oral and written. The Oral
Test is a 30-minute interview in which an interlocutor
guides two candidates in a discussion of the materials
they have prepared. Candidates are expected to
interact with each other. This is another function of the
dossier: it serves as a starting point for conversation.
Half an hour may seem a very ling time, but this was a
conscious decision that has been vindicated as we
Page 3
www.e-mesh.com
teachers’ centre
articles
testing
have conducted the test in the last few years. In the
first place, 12.5 minutes per candidate (allowing 5
minutes for interlocutor management talk) is not such
a long time, and secondly, some candidates are so
nervous at the start of the test (into the valley of Death)
that it takes them 15 minutes to relax and start
performing to the best of their ability.
The Written Test lasts an hour, during which
candidates write two texts of approximately one page
each, one on their prepared topics and the other an
unprepared authentic-type text such as a letter or an
article, on a topic of general interest.
In both parts of the test the emphasis is not so much
on getting the answer right as on communicating
effectively on topics of interest, importance or personal
relevance. And how is all this marked? We were
fortunate in the timing of our project because the
Council of Europe had just produced ‘Modern
Languages: Learning, Teaching, Assessment. A
Common European Framework of Reference.’ This
document became our bible, not least because it
contains descriptors of 48 or so elements that are part
of language ability, set out on a 6-band scale.
A large proportion of the test development process
involved our selecting the criteria that we wanted to
measure in the TIE. Our advisors told us that
examiners could not comfortably remember more that
five or six criteria, so we selected five for each part of
the test (Oral and Written) of which two are common to
both. These criteria are:
Oral Test:
Interaction
Communicative Effectiveness
Phonological Control
Written Test:
Production
Task Achievement
Orthographic Control
Common:
Accuracy of Structure and Lexis
Range and Complexity of Structure
and Lexis
As we trialled and piloted the test, we modified the
descriptors until we were happy that they described
accurately the elements of language performance that
we expected from candidates at the 6 levels in the
Common European Framework (A1-A2-B1-B2-C1-C2,
also known as Breakthrough – Waystage – Threshold –
Vantage – Effectiveness - Mastery).
The Advisory Council, who established the TIE,
operates under the aegis of the Department of
Education and Science in Ireland, so in effect the TIE
is run on behalf of the state education system. At the
moment (autumn 2000) it is available to all students
following an EL course in Ireland. It takes an average
of 10-14 days to prepare, though students who are
here for longer may spread the preparation over a
longer period. This does not affect the outcome of the
test since what is being measured is language
performance, not depth of subject knowledge.
Certificates are sent out about two weeks after the test
date.
The easy part of designing the Test – being inventive
and creative - is now behind us, and it has taken five
years. Ahead of us lies the ongoing developmental
work – marketing and financial planning; explaining to
teachers how the test fits in with their teaching; and
recruiting, training, monitoring and retraining
examiners, as well as the day to day administration of
actual tests. We also have to look at systems for
recording tests in case of queries from students or
teachers, and we need to start in-depth statistical
analyses so that we can be absolutely sure that the
test continues to be valid and reliable.
The original test development group has almost
dissolved as the members have gone on to other
things, and business management structures are
being put in place. From one point of view, the ‘fun’
part is over; from another, the challenge is only
beginning. Either way, all the people who were
involved in the first five years of the Test of Interactive
English can rest assured that, far from adding to the
battery of tests mentioned at the beginning, they are
responsible for putting the rose in the barrel of the
testing rifle. English language tests will never be quite
the same again.
Gronia de Verdon Cooney is an experienced and highly-qualified
freelance teacher and teacher trainer. Her interests include
strategies and multiple intelligences, writing and testing. She was
until recently Chief Examiner and Academic Manager of the Test
of Interactive English and is working with the Centre for Language
and Communication Studies (CLCS) in Trinity College Dublin. She
is a member of IATEFL and BIELT, and is Chairman of the Forum
for English Language Teachers (FELT Ireland).