RIDE 2025 Day 2 afternoon: Closed book exams or not?

The title of the fourth and final keynote presentation at RIDE 2025 was, unusually, posed as a question for debate. In introducing the speaker, Inspera’s Ishan Kolhatkar, CODE director Linda Amrane-Cooper first described that company as the University of London’s technology partner for exams. Each year, the University runs a ‘mind-boggling’ number of exams – approaching 100,000 – and the Inspera platform has been integral to their success. So it is interesting, at least, that a speaker from Inspera would be discussing the ‘controversial’ question of whether there is still a place for exams (or at least for closed-book exams) in contemporary higher education. 

Introduction 

Ishan began his talk by addressing the question ‘why are you here?’, addressed to himself, with a potted history of his own career. He had started as a barrister – a background that came up often in his talk, with examples and anecdotes taken from the law – before moving into academia as a law lecturer and then, in 2017, to be Deputy Dean of Learning & Teaching at BPP University where he transformed assessment by implementing Inspera. His current job title at Inspera, Global Client Evangelist, was not one that he had chosen for himself, but he likes it. 

He explained that he wanted to make us think about how we assess students. He deliberately chose a topic and title that would encourage us to critically evaluate our own positions on this question and (as a lawyer would) the evidence to back them up. We each of us start with a position that derives from our personal experience. His own position, of disliking exams and closed-book exams in particular, arises at least in part from his finding them difficult as a student on account of his poor memory. In contrast, assessment in the Bar Course - that is, the necessary step between a law degree and pupillage for intending barristers – directly assesses candidates on a range of practical, relevant skills including advocacy and case conferences.  Academia, however, is full of people with good memories who survived and thrived in the current exam-based system and who are more likely to have positive memories of closed-book exams. It is always useful to ask the question of whether our individual preferences for assessment types correlate with what we were ‘good at’ and ask whether this is fair to all students. 

Provocation 

Academics’ most common response to Ishan’s thoughts on closed-book exams can generally be summed up as ‘But Ish, I need people to know things!’. This is often at least partly true, and there are some justifiable reasons for sticking to closed-book exams. He presented the case, however, that there are far fewer of these reasons than there are currently closed-book exams. We need to think about the context in which we want students to gain, and demonstrate, knowledge. As an example, he told a story in which he needed to paint a wall (which he’d never done) and found that the book-knowledge, or even video-knowledge, available on his phone was almost useless.   

Closed-book exams, then, identify which students can take good notes and then regurgitate them correctly: they do not test critical thinking. Any critical thinking that has occurred came much earlier, when the students took the notes in the first place and also, often, when they planned their revision around what they thought (often correctly) would ‘come up’. Are we over-assessing students’ memories, and how much of the syllabus is it absolutely essential that students know ‘off the top of their heads’? It is likely to be much less than what they are assessed in that way. In his own field of law, there is far too much material to be memorised, and if a potential client ever meets a lawyer who ‘never looks anything up’ they would be advised to run away. 

So, what are the alternatives? There are, of course, open-book exams, but these are still to some extent tests of memory as it is impossible to look up everything and still finish the exam in time. Students need to remember enough of the first principles to know how to search efficiently for the information required. And there are also oral exams. These have many benefits, not least for the often-beleaguered markers, but they also have drawbacks. Many students find them nerve-wracking, and they are difficult to schedule, particularly if the student cohort is large. Technology, and also flexibility in the exam format and schedule, can help here. 

Discussion  

Ishan then introduced two questions for discussion, asking delegates in the room to discuss them in their tables and those online to add comments to the padlet. These were: 

1. Is there a place for closed-book examinations in higher education? 
2. If so, what are the circumstances in which they are
   1. The only option 
   2. A better option than other assessment instruments 

The discussion in groups led into a long and lively general discussion. Linda began by picking up on some of the points from the padlet. One contributor summed up a commonly-made point in suggesting that, before deciding to keep or replace a closed-book exam, you should ask whether the skills that the students are being asked to demonstrate are authentic: that is, whether they would be expected to memorise and recall this knowledge ‘in the field’, or in their workplace. Often, closed-book exams only test the knowledge itself, not its application. “I’m fairly sure that I’d be fired if I did things without … checking sources and making sure I had all the information I need” wrote another commentator.  

Delegates in the room suggested circumstances where real-world situations do require memorisation. The first quoted example was of clinical medicine: particularly surgery, where a surgeon would not be able to stop an operation to refer to Gray’s Anatomy. According to Ishan, that case was the one most often cited, and he equated the doctor’s real need to learn anatomy with the schoolchild’s need to learn to spell, as base knowledge. Interestingly, however, another delegate mentioned a real-life example of a doctor who needed to call in a specialist from another hospital to fill in a gap about the function of a particular nerve. English literature is an example of a discipline at the other end of the spectrum: graduates can wonder how much more critical thinking they would have learned if they had not had to memorise so much text. 

Another delegate suggested that it is important to define what is meant by ‘closed-book exam’. There are many ways of testing unseen, memorised knowledge other than the traditional exam in which candidates are required to write essays. There are cases where essay-writing skills will have practical uses, such as when lawyers are required to write down arguments, but these are limited. 

Other questions that were raised centred on the types of skills we expect our students to acquire, and on the role of professional bodies. If we use closed-book exams we are by definition only assessing the cognitive domain, and an exam-based system arguably places too much emphasis on that. Students also need to learn ‘soft skills’ and our universities too often turn out students who have excellent subject knowledge but are poorly prepared for the workplace. And professional bodies, too, often insist that students must pass exams before they can be accredited. Some accrediting bodies for accountants or engineers even set a minimum percentage of the assessment as either ‘an in-person, invigilated exam or an online proctored exam’. Ishan commented that regulators of the legal professions work in a similar way, and that ‘that’s just the way it is’. Linda added that one of the reasons why the University of London sets so many exams is that the regulators in the countries where our students are based require them, and we need to make sure that all our students get degrees that are recognised in their local context. 

In response to a request for his personal recommendations, Ishan suggested that institutions would do well to consider their assessment strategy as a whole and work out where both open- and closed-book exams fit within it. Two institutions he knows of that have adopted this approach successfully are the Coventry University in the UK and the University of Queensland in Australia. This provides a useful opportunity for academics to look at the assessment strategies available across all the disciplines, rather than sticking with those they have used in their own departments for years. It does, however, have the disadvantages of being time-consuming and expensive. 

Finally, a delegate from Nigeria picked up on one of the comments on PollEverywhere, that knowledge was ‘subjective and … colonial’, and that closed-book exams, in particular, make assumptions about what knowledge is valued. However, there are still a few circumstances in which a closed-book exam will be the most appropriate choice. Ishan concluded a fascinating discussion by agreeing that, while we shouldn’t ‘consign closed-book exams to the bin’, we need to think more carefully about why we have the ones we do. 

Linda ended the session by thanking Ishan for a provocative, thoughtful keynote and for doing exactly what she had asked, which was ‘… to get [delegates] thinking, talking and arguing…’. In her opinion, it had been the best post-lunch session ever.