There is a place for multiple choice in English – Part II

Image

In my last blog I laid out the background to my thinking around the use of multiple choice as an assessment tool in English. My focus is on its use as a formative vehicle, in particular its application to the teaching of reading. To this end I have been experimenting with setting regular multiple choice assessments with my GCSE English class, preparing them for an examination of Steinbeck’s Of Mice and Men next summer.

This post will outline the nitty gritty of my approach, explaining how I have been using these short assessments to inform my weekly planning. Whilst I will concentrate a great deal on the methodology, I will also look at the results and the manner in which I have been interpreting them in order to inform my subsequent instruction. What I am realising more than anything, is that this is still very much a work in progress – but one that I think has considerable scope for future development.

So, to the practicalities. My year 11 class are a challenging set 3 out of 5. All the students are predicted ether a B or a C, though many still have a great deal of ground to do to obtain these grades. And this is where I planned to take advantage of the multiple choice format: to help me target areas to focus on in order to maximise learning and the impact on progress. I have taught Of Mice and Men many times before, so I am familiar with what to expect from students and how to approach the teaching. This familiarity has proven to be very useful when it comes to designing the assessments, which are not only time consuming to produce, but also reliant on a thorough understanding of the text.

I decided to break up the unit into six weekly teaching cycles. Using the mark scheme I identified everything the students would need to know about the novel in order to obtain at least a grade B. This included a detailed knowledge of the characters, their relationships, the plot, the settings, the different contexts, a wide range of authorial methods and the overarching themes. Each week I set out to address a different focus, with the book’s six chapters nicely matching my intentions. I wanted the focus to build in complexity and consist of a cycle of learning that included reading and discussing, analysing one or two key passages and then writing in a structured way. I wanted the multiple choice questions to check the level of understanding for each focus and give me feedback that would inform my teaching for the following week.

The first week focused on chapter one, in particular the initial presentation of George and Lennie, and the role of the setting in establishing their friendship. Whilst students would obviously learn more as the book progressed, I wanted them to know the following after reading the first chapter:

–       what happens in chapter one and what we learn about past events

–       where and when the story is set

–       who George and Lennie are and the differences between them

–       the nature of their friendship and their shared dream

–       how Steinbeck uses adjectives and verbs to portray their characters

–       how Steinbeck uses animals to construct Lennie’s character

–       the symbolic importance of the setting

In the first three lessons we read and discussed chapter one, and the students completed activities designed to help them identify and explain the language used to construct the characters and how to write about these details analytically. At the beginning of the fourth lesson the students completed the first multiple choice assessment, which focused on testing their understanding of some of the areas identified above.

After reading a number of blog posts and research papers about the effective design of multiple choice questions, I realised that my questions would need to bear in mind the following if they were going to be successful:

–       they would need to assess understanding of a teaching sequence

–       they would need to be phrased in a clear and unambiguous way

–       they would need to have all options as plausible answers

 and most importantly: 

–       there would need to be clear cognitive reasoning behind each option, which would enable me to use the results diagnostically 

Here is a link to my first assessment on chapter one: http://tinyurl.com/pb9u5y3 

On the whole students did quite well, with the average class mark just under 7 out of 10. Considering that I had made the questions quite challenging and had deliberately stayed away from explicitly discussing the chosen passages in advance, I felt this was a decent reflection of their learning. After looking through the results, a few things started to become apparent. Firstly, it was clear that I had phrased one or two questions rather poorly. In question two, for instance, it was clear that the word ‘morosely’ caused a problem. This reminded me of the importance of unpicking unfamiliar vocabulary and the need to be more mindful with my question phrasing.

Of much more interest, however, was the students’ poor showing on question three, which I had not considered to be terribly difficult. This question asked students to select the most fitting explanation for the ‘verb ‘exploded’ and the adverb ‘triumphantly’ in the quotation: “They run us outa Weed,” Lennie exploded triumphantly. Only a handful of students got what I had set as the right answer, which initially left me puzzled. Even though I thought had carefully rationaled the logic of each possible option, I realised that the conclusion I hoped students would reach – that Lennie did not fully understand the consequences of his actions and was therefore dangerous – required a far deeper level of deduction than I had initially anticipated. I had taken it for granted that students would arrive at the full, ominous significance of Lennie’s naïve explosion of joy.

The more I thought about it the more it became clear the level of deductive reasoning and inference that would be required to reach this depth of understanding, the kind of understanding I wanted the class to be able to demonstrate. I came up with the following chain of reasoning:

  1. Lennie remembers that they were run out of Weed.
  2. Lennie is excited that he has remembered.
  3. Lennie cannot control his excitement at remembering.
  4. Lennie likes to please George – they are friends.
  5. Lennie must have a childish nature if he is pleased by all this.
  6. Lennie has clearly forgotten the main reason why they were run out of Weed: his inappropriate actions.
  7. He does not recall, because he does not understand that he’s done anything wrong.

This therefore makes him potentially dangerous, as he does not understand what he did wrong, even though it had terrible consequences.

Despite having taught this novel for many years and helped students to gain very good grades for it in exams, I had never really thought about aspects of their understanding in this degree of detail. The multiple choice assessment, or rather the students’ responses to it, had forced me to reflect on the way that I taught the text and challenged some of my approach to the teaching of reading.

The next lesson I went through this example with the class, explaining to the students the sequence of inference and deduction that I thought they needed to go through in order to arrive at the most sophisticated response. Whilst I am fully aware that part of the joy of literature is the multiplicity of interpretation, I genuinely want my students to be able to read and understand to this level of sophistication. All the responses that I set may well be plausible and in that sense correct, but I believe the option that suggests it shows Lennie’s potential danger is ultimately the most sophisticated, requiring the greatest insight and evaluative skill and therefore worthy of the highest standards of academic excellence.

I have since conducted two more assessments, with three further tests to follow. In my third, and final, blog on this subject, I will analyse the salient points from my students’ responses to these questions, and start to arrive at some tentative conclusions for how I intend to use multiple choice in the future. I will also offer some reflections on broader applications of the format.

For now, I am still learning what it can do for me.

Advertisements

Is there a place for multiple-choice in English? Part I

Image

This is a blog about my experience of using multiple-choice questions as a form of formative assessment in English. It will be split into three parts. This initial post lays out my present thinking around the format and how I think it could be of use to teachers. In the next few days I well post on explaining exactly how I implemented the assessments within a cycle of teaching, before I finally conclude with my reflections about its impact and suggestions for further development.

Whilst the focus of my experimentation has been in English – using multiple choice to guide my instruction of teaching Of Mice and Men to a middle set year 11 class I hope that some of the insights that I will be sharing will be of use to other curriculum areas, particularly those essay based subjects where there is lots of reading to do and lots of knowledge to acquire, and which may not initially appear to lend themselves to the multiple choice format.

Much of my thinking here has been inspired by the chapter on assessment in Leadership Leverage, and by recent blog posts on the subject by Daisy Christodoulou http://bit.ly/17y37nP and Harry Fletcher-Wood http://bit.ly/HnELmq  Through these readings and my own subsequent reflections, I have become increasingly convinced that multiple choice can offer teachers an important and, perhaps even, necessary method for providing ongoing formative data that can reliably inform short and medium term planning.

As I have already suggested English is not usually associated with the use of multiple choice as a means of assessing understanding, at least in this country where I have not certainly not seen it employed in any meaningful way. I suppose I can recall instances where I have seen the format used to crudely test basic understanding, such as in a short quiz at the beginning of a lesson to re-cap events in a novel or the names of key characters and settings. But this blog is not interested in that kind of use of multiple-choice approach; instead, it’s about something far more complex and nuanced than that, something that I am only beginning to really understand and appreciate for myself.

In America, students who study English have far more exposure to the format than students in this country, where state schools regularly use multiple choice questions as part of their standardised reading assessments. Students in this country, however, tend to have their reading assessed through extended written responses – a situation that is only going to increase in light of the recent national curriculum developments. Now, I am certainly not in any way proposing that one model of assessment is inherently better than the other, or that I favour a move away from extended writing. I don’t. As it happens, I think students should be encouraged to write at length and be given every opportunity to do so in a meaningful way across the curriculum.

What I am interested in exploring and learning from the American system is the way that the multiple-choice assessment model can provide teachers in this country with a robust formative vehicle to ensure that when students do come to their extended writing they are more likely to flourish because they have the requisite deep knowledge and understanding that the skill of writing relies upon. Ultimately, you have to know a great deal about a book, a poem, a text or a topic in order to truly be able to synthesise, evaluate and analyse in your writing, criteria which are usually found in the highest bands of reading assessment mark schemes.

Now, if you had asked me last year what I thought about the use of multiple choice assessment in general, let alone in English, I would probably have baulked, thinking either that the format was too simplistic to be any way useful, or that it simply was not compatible with the demands of gauging ability or progress in English. After all, both the English GCSE and A level exams test reading via the extended essay, so what possible benefit could the format hold for English teachers outside of this. This model of assessment – one that sees knowledge and understanding expressed through the medium of the written word – lies at the heart of the ethos of many an English curriculum. It explains why most schools run some kind of half termly key assessment task: an extended essay that assess reading or writing, which is then leveled with the results duly entered onto a spreadsheet.

But this model of assessment seems to me to have a number of significant flaws. Firstly, and most importantly, it is a lag measure. In other words, it gives summative information about student achievement and progress after instruction has ended, when it is simply too late to do anything about it. The class has moved onto to the next text or unit of work, with their scores or levels firmly in the rear view mirror, disappearing into nothingness. No doubt, some schools – those not overly bogged down in the mire of propping up year 11 outcomes –probably do use this data more wisely, either to adapt medium and long term planning or to intervene for whole classes, groups or individuals. I’m guessing that this is probably not the norm, though.

Yet, I think that even this kind of reflective and pragmatic approach is in itself problematic, particularly with the vagaries of levels factored into the equation. Either the data itself used to inform intervention is erroneous – from my experience it is hard to ensure the consistent application of levels, particularly in large departments – or because the skills descriptors themselves are too generic and cloudy to offer meaningful information that can inform further instruction. The intervention ends up addressing everything by repeating what didn’t work the first time, or it focuses on the wrong thing. Rarely is the result of this kind of remedial action retested, particularly at KS3 where there is rarely the time or the will. I am surely not alone in thinking that these half term key assessments are often more for us and our accountability systems than they are for the students who will have to live with the results of our teaching for the rest of their lives – for better or for worse.

And here is where I am starting to think that multiple choice might provide a much-needed helping hand, providing us English teachers with the means of increasing the chances that when our students do come to do extended written responses they write much better because they know what they are writing about: that they have the necessary knowledge and understanding to access those higher band reading criteria. Whilst I seem to have been exclusively focusing here on using multiple choice questions for developing reading, in truth because there is such an intrinsic relationship between knowledge about something (the reading) and the ability to express that understanding clearly and effectively (the writing), the format is ultimately likely to develop both skills simultaneously – Joe Kirby’s metaphor of the double helix remains pertinent: http://bit.ly/148KSRd

My next blog will explore the set up and practicalities of how I have been trying to implement the use of multiple-choice questions in my teaching and offer some of my initial findings.