Are closed questions enough for national curriculum tests?

An assessment is an opportunity for pupils to demonstrate what they can do and what they have learnt. It is up to assessment writers to provide clear, unambiguous items, assessing a range of content and skills in different ways. But research shows that if a test is limited to particular item types or in the range of skills it assesses, pupils will be limited in their responses.

The purpose of the National Curriculum Tests (NCTs) is to measure pupils’ attainment in maths and English against the national curriculum. To do this, NCTs contain a range of question types that can broadly be divided into two categories: open-response items (where the pupil creates their own response) and closed-response items (where the pupil selects their response from a set of given options). Closed-response items are good at measuring recall, understanding and some simple process skills. Open-response items, are more time-consuming to mark (and it is harder to automate that marking), but they are better able to measure application, analysis and evaluation skills.

With regards to reading, NCTs tend to use longer, open-response questions to assess more complex skills, such as inference supported by evidence. For example, in the 2019 year 6 reading test, pupils read a story about two friends and were asked to give two impressions of the relationship between them, using evidence to support their answer. For three marks, pupils needed to provide two of the five acceptable points listed in the mark scheme and at least one piece of textual evidence. Markers had the freedom to mark any acceptable point as correct, as long as it was justified with evidence.

If this were assessed using a closed-response item, a pupil may spend time contemplating a list of multiple choice options that they would have phrased differently to the assessment developer. In addition, closed-response items would have given only one or a small number of correct responses, whereas the open-response item allowed multiple acceptable points for pupils to describe their interpretation in their own words.

Maths NCTs assess a range of skills from knowledge and understanding to more advanced problem solving and evaluation. One paper comprises entirely open-response items, enabling pupils to demonstrate their calculation skills and the specific methods and approaches identified in the curriculum (e.g. long multiplication and division). In ‘show your working’ problems, pupils are encouraged to show their problem-solving approach. For example, in the 2019 year 6 mathematics paper, pupils were challenged with:

“A machine pours 250 millilitres of juice every four seconds. How many litres of juice does the machine pour every minute?”

The mark scheme gave three methods, each leading to a correct solution. Two marks were awarded for a correct method and calculation, while one mark could be awarded for a correct method. ‘Show your working’ questions, therefore, provide pupils with an opportunity to demonstrate how they tackle extended problems that require them to formulate and carry out a method, which a closed-response question would not allow.

While it is possible to write closed-response questions to assess higher-order skills such as analysis and evaluation, the language demand increases, the incorrect options become more realistic and plausible, and pupils need more cognitive skills to understand what they are being asked to do. In reading tests, the options can also give away answers to other questions, so the test may need more texts to assess the same range of skills, which can cause pupil fatigue and impact pupils’ performance.

There is a temptation, particularly in on-screen tests, to use purely closed-response item types. But while they can be scored quickly and provide teachers with some diagnostic information, they limit the range and skills that can be assessed. The purpose of an assessment is critical in deciding which item types are needed.

To give teachers the necessary insights into their pupils’ deeper skills (such as critical thinking, analysis and problem solving) open-response items are required. The national curriculum requires pupils to be taught these higher-order skills, therefore any test seeking to assess the curriculum should contain some open-response item types.