Exam production
Making, delivering, assessing and validating exams
How an exam is developed
At Cambridge ESOL we constantly review the exams we provide to ensure they are accurate, relevant and fair.
We listen carefully to the feedback we receive from test takers, teachers, centres and other professionals in the language learning and assessment world. Sometimes this feedback leads us to identify areas where there is an unsatisfied demand for language assessment and a need to create an entirely new exam.
Creating a new exam which is a thorough and meaningful test of a new area of assessment is a complicated process requiring careful research and testing. Some of the world's leading experts in language assessment contribute to the work of Cambridge ESOL and the quality of our exams is built on their combined expertise, experience and understanding of assessment and learning.
New exams have to go through a rigorous series of processes including:
- commissioning the exam — where the scope and time frame for the new exam is mapped out
- pre-editing — checking that the material written by the exam question writers broadly fits the commissioning brief
- editing — ensuring the quality and accuracy of the test paper
- pretesting/trialling — trying out the test paper to ensure that it tests candidates in the way expected
- pretest/trial review — checking the results of the pretesting and trials to see if further editing is needed
- test construction — the new exam is checked to ensure the level and focus of the test are correct and that there are no overlaps with other existing tests
- overview — a final critical review of the whole process before the question paper production process begins.
Only when we are completely satisfied that the test has passed all of these stages, will it be released to the public.
In many cases it takes two years from the first stages of planning to students sitting the first session of the new test.
Delivering exams
For a test to be fair and accurate, everyone who takes it has to have the same opportunity.
To ensure our tests are fair we have a number of important security measures in place to ensure that exam papers are safely distributed from our headquarters in Cambridge to the thousands of places around the world where our tests take place.
Exam papers are delivered to test centres by a number of secure methods — some receive them by secure post, and in some countries they are sent to a courier for delivery only on the day of the test.
Exam papers frequently need to be stored for brief periods of time before the exam, as do the completed exam scripts before they are returned to Cambridge ESOL for assessment. All exam centres are required to have secure storage facilities and are inspected regularly to ensure they meet our strict criteria for security.
Many of our certificates can now be earned through taking a computer-based version of the test. In these cases, the exam is sent electronically to the centre ahead of the test, but can only be opened when an encryption key is sent on the day of the test itself.
Assessing exams
Cambridge ESOL's reputation for quality does not rest on our examination alone. Quality is a process which begins with the research and design of tests and ends with an accurate assessment of each candidate's entry. Cambridge ESOL's quality management system is certificated to the ISO 9001:2000 standard.
The assessment of a candidate's work has to be as consistent and reliable as every other aspect of the examination process.
Upon arrival at Cambridge ESOL, all completed written papers are randomly allocated for marking — this ensures that regardless of where they have come from, all papers are marked fairly.
The examiners who mark the tests are themselves subject to strict monitoring of their own performance to ensure they are accurate and consistent.
Speaking tests (with the exception of IELTS) use two assessors to ensure accuracy and these are randomly recorded for monitoring purposes. All IELTS Speaking tests are recorded for subsequent monitoring.
Some of our computer-based tests (such as BULATS) are able to give an instant assessment, which is of great value to those who are testing large numbers of people for educational or recruitment purposes.
Validating exams
Cambridge ESOL examinations have a long-established and well-deserved reputation for excellence. More than 2 million people in 130 countries take our exams every year: it is essential to the quality of our exams and the continued reputation of Cambridge ESOL, that each one of those candidates experiences a test that consistently meets the high standards we set.
To achieve this, we have the largest dedicated research team of any UK-based language assessment organisation. The work they do in validating our exams — ensuring their quality, fairness and relevance — is vital to our continued high reputation.
We work closely with leading specialists in the following disciplines:
- testing and assessment
- statistical analysis and item-banking
- applied linguistics
- corpus linguistics
- language learning/pedagogy.
Together they carry out a variety of research projects for all of our exams.
Because Cambridge ESOL is a department of a not-for-profit organisation it is perhaps unique in sharing the findings of much of its research with other academics and specialists in the language testing community.
We regularly give presentations to conferences, submit papers to leading academic journals, and we also publish our research ourselves in our own publications, the quarterly journal Research Notes and in a series of books entitled Studies in Language Testing.
