In my last blog I laid out the background to my thinking around the use of multiple choice as an assessment tool in English. My focus is on its use as a formative vehicle, in particular its application to the teaching of reading. To this end I have been experimenting with setting regular multiple choice assessments with my GCSE English class, preparing them for an examination of Steinbeck’s Of Mice and Men next summer.
This post will outline the nitty gritty of my approach, explaining how I have been using these short assessments to inform my weekly planning. Whilst I will concentrate a great deal on the methodology, I will also look at the results and the manner in which I have been interpreting them in order to inform my subsequent instruction. What I am realising more than anything, is that this is still very much a work in progress – but one that I think has considerable scope for future development.
So, to the practicalities. My year 11 class are a challenging set 3 out of 5. All the students are predicted ether a B or a C, though many still have a great deal of ground to do to obtain these grades. And this is where I planned to take advantage of the multiple choice format: to help me target areas to focus on in order to maximise learning and the impact on progress. I have taught Of Mice and Men many times before, so I am familiar with what to expect from students and how to approach the teaching. This familiarity has proven to be very useful when it comes to designing the assessments, which are not only time consuming to produce, but also reliant on a thorough understanding of the text.
I decided to break up the unit into six weekly teaching cycles. Using the mark scheme I identified everything the students would need to know about the novel in order to obtain at least a grade B. This included a detailed knowledge of the characters, their relationships, the plot, the settings, the different contexts, a wide range of authorial methods and the overarching themes. Each week I set out to address a different focus, with the book’s six chapters nicely matching my intentions. I wanted the focus to build in complexity and consist of a cycle of learning that included reading and discussing, analysing one or two key passages and then writing in a structured way. I wanted the multiple choice questions to check the level of understanding for each focus and give me feedback that would inform my teaching for the following week.
The first week focused on chapter one, in particular the initial presentation of George and Lennie, and the role of the setting in establishing their friendship. Whilst students would obviously learn more as the book progressed, I wanted them to know the following after reading the first chapter:
– what happens in chapter one and what we learn about past events
– where and when the story is set
– who George and Lennie are and the differences between them
– the nature of their friendship and their shared dream
– how Steinbeck uses adjectives and verbs to portray their characters
– how Steinbeck uses animals to construct Lennie’s character
– the symbolic importance of the setting
In the first three lessons we read and discussed chapter one, and the students completed activities designed to help them identify and explain the language used to construct the characters and how to write about these details analytically. At the beginning of the fourth lesson the students completed the first multiple choice assessment, which focused on testing their understanding of some of the areas identified above.
After reading a number of blog posts and research papers about the effective design of multiple choice questions, I realised that my questions would need to bear in mind the following if they were going to be successful:
– they would need to assess understanding of a teaching sequence
– they would need to be phrased in a clear and unambiguous way
– they would need to have all options as plausible answers
and most importantly:
– there would need to be clear cognitive reasoning behind each option, which would enable me to use the results diagnostically
Here is a link to my first assessment on chapter one: http://tinyurl.com/pb9u5y3
On the whole students did quite well, with the average class mark just under 7 out of 10. Considering that I had made the questions quite challenging and had deliberately stayed away from explicitly discussing the chosen passages in advance, I felt this was a decent reflection of their learning. After looking through the results, a few things started to become apparent. Firstly, it was clear that I had phrased one or two questions rather poorly. In question two, for instance, it was clear that the word ‘morosely’ caused a problem. This reminded me of the importance of unpicking unfamiliar vocabulary and the need to be more mindful with my question phrasing.
Of much more interest, however, was the students’ poor showing on question three, which I had not considered to be terribly difficult. This question asked students to select the most fitting explanation for the ‘verb ‘exploded’ and the adverb ‘triumphantly’ in the quotation: “They run us outa Weed,” Lennie exploded triumphantly. Only a handful of students got what I had set as the right answer, which initially left me puzzled. Even though I thought had carefully rationaled the logic of each possible option, I realised that the conclusion I hoped students would reach – that Lennie did not fully understand the consequences of his actions and was therefore dangerous – required a far deeper level of deduction than I had initially anticipated. I had taken it for granted that students would arrive at the full, ominous significance of Lennie’s naïve explosion of joy.
The more I thought about it the more it became clear the level of deductive reasoning and inference that would be required to reach this depth of understanding, the kind of understanding I wanted the class to be able to demonstrate. I came up with the following chain of reasoning:
- Lennie remembers that they were run out of Weed.
- Lennie is excited that he has remembered.
- Lennie cannot control his excitement at remembering.
- Lennie likes to please George – they are friends.
- Lennie must have a childish nature if he is pleased by all this.
- Lennie has clearly forgotten the main reason why they were run out of Weed: his inappropriate actions.
- He does not recall, because he does not understand that he’s done anything wrong.
This therefore makes him potentially dangerous, as he does not understand what he did wrong, even though it had terrible consequences.
Despite having taught this novel for many years and helped students to gain very good grades for it in exams, I had never really thought about aspects of their understanding in this degree of detail. The multiple choice assessment, or rather the students’ responses to it, had forced me to reflect on the way that I taught the text and challenged some of my approach to the teaching of reading.
The next lesson I went through this example with the class, explaining to the students the sequence of inference and deduction that I thought they needed to go through in order to arrive at the most sophisticated response. Whilst I am fully aware that part of the joy of literature is the multiplicity of interpretation, I genuinely want my students to be able to read and understand to this level of sophistication. All the responses that I set may well be plausible and in that sense correct, but I believe the option that suggests it shows Lennie’s potential danger is ultimately the most sophisticated, requiring the greatest insight and evaluative skill and therefore worthy of the highest standards of academic excellence.
I have since conducted two more assessments, with three further tests to follow. In my third, and final, blog on this subject, I will analyse the salient points from my students’ responses to these questions, and start to arrive at some tentative conclusions for how I intend to use multiple choice in the future. I will also offer some reflections on broader applications of the format.
For now, I am still learning what it can do for me.