The Elements of Language – Lessons learned

Right_Way_Wrong_Way_-_Road_Si_2737953

It has been interesting to read the recent online discussions between David Didau and Daisy Christodoulou about the merits and pitfalls of different assessment models. Many of the issues they raise are ones that anyone who has invested time in creating an alternative to National Curriculum Levels has almost certainly encountered for themselves. This is probably the case even more for those working in schools that have piloted these approaches and seen flaws emerge that were not necessarily apparent from the outset. It is easier to envisage an alternative to levels, but perhaps harder to make it work in practice.

This post is about my current thinking in relation to assessment at KS3. It reflects the specific context of my school and the types of challenges and opportunities that we face in the months and years ahead. I wrote the last of my two previous posts on our English assessment model, The Elements of Language, about a year ago and since then my thinking has moved forward quite a bit, partly as a result of our experiences to date, but more as a consequence of us moving towards a significantly enhanced CPD programme next year, which will include substantial and enshrined professional development every week. This significant investment of time should enable departments to collaborate on planning, share their understanding and interpretation of assessment data and get the chance to look closely at student work together – the actual results of what happens in the classroom.

The current beliefs that underpin my approach to assessment can be summed up as follows:

  • performance descriptors are often too vague and unreliable for drawing useful inferences
  • performance descriptors can often mask student underachievement and gaps in learning
  • specific statements of the learning to be mastered organised in a logical sequence are generally more useful
  • in some subjects it is hard to reduce certain aspects of achievement down to a manageable amount of specific statements about learning
  • in practice, a mastery approach to assessment can be time-consuming for teachers to implement and can detract from planning better lessons
  • threshold concepts are a useful way of mapping out transformational pathways to achievement for both students and teachers alike
  • most assessment should primarily aim to inform the next steps, whether in the classroom or more widely across a department or year group
  • any inferences drawn from assessment should be acted upon as quickly as possible
  • assessment can be a useful means of ensuring students learn and make necessary progress, with the caveat that learning takes time and progress does not look the same in every subject
  • looking at and discussing actual student work with colleagues is a powerful way of understanding the impact of classroom teaching and reaching a shared understanding of what success looks like and how to get there
  • assessment is more robust if its draws upon a range of different forms and provides multiple opportunities for that learning to be demonstrated e.g. MCQ, essay, short answers

If, as Dylan Wiliam suggests in the comments at the bottom of Daisy’s recent blog, ‘an assessment is nothing more, or less, than a procedure for making inferences’, then it is wise to make sure that whatever is used in place of levels, ensures these inferences are as reliable as possible and are acted upon as quickly as is necessary. I think that what I am proposing here achieves both these ambitions and, perhaps more importantly, provides a means through which subject professionals can engage in meaningful discussions about student learning, where gaps or misconceptions can be identified and appropriate action can be taken.

Learning from past mistakes

On reflection, I made several errors in my earlier iterations of the Elements of Language. My first mistake was to include knowledge acquisition within the overall assessment framework – knowledge and vocabulary were distinct thresholds of the reading and writing Elements respectively. Whilst I am still very much committed to the centrality of knowledge development, I can see that there are probably better, more robust ways of assessing students’ acquisition of it. Broadly, I am working on the idea that in English – and perhaps other humanities subjects such as history and religious studies – there should be a core knowledge component. This component would be assessed at strategic points throughout the year, using an efficient format such as multiple-choice that provides accurate formative data on whether students have learnt the requisite knowledge or not. I suppose this is a variation on the principle of knowledge organisers, though in my thinking the notion of core knowledge would probably be a bit more detailed as well as closely linked to a systematic programme of vocabulary instruction. There will be more on what I mean about this over the coming months.

My second error was to place the notion of mastery too much at the forefront of the assessment framework – the ‘rubric’ seen by parents, teachers and students articulated what was to be learned in a very explicit way. I now believe that it is probably better for any overarching framework to contain more generalised articulations of the different thresholds (see example below) so that it is clear what stages of transformational learning students need to pass through in order to achieve genuine mastery, say with regards to developing an ability to control writing or adopting an academic voice.  More specific items of learning to be mastered are, I think, better served sitting behind these threshold definitions, encoded as objectives but acting more as standards to be achieved by the end of each academic year. It is possible in my revised model to have different sets of standards depending on where students are at the beginning of the year, thus ensuring rigorous objectives are well matched to different starting points. I should stress that I really only see the notion of standards applying to maths and English at KS3, who have the time, resource and sense of urgency in terms of securing core competences.

From threshold concepts to classroom teaching

In my proposed assessment model specific to-be-learned items would be drawn from the threshold objectives (which, remember, are operating as standards) and these individual learning items would be pursued relentlessly by each teacher until an agreed level of mastery is achieved, in or around the 80% figure. In this model the threshold concepts have effectively been broken down into objectives which have then been mapped out across the units of work for the year. These objectives, or standards, would be assessed in a holistic way only once or twice every year – suitable periods of time in which inferences about long term learning are more likely to be valid.

On a day to day basis the standards across a unit of work would be reduced down yet further into specific learning items that would need to be mastered across a sequence of lessons. This sequence is a manifestation of our version of a teaching and learning cycle, one which we are introducing next year and that I will try and blog about in due course. Bodil Isaksen is right when she explains how the lesson is the wrong unit of learning. I think it is far better to see learning planned across longer periods of time, rather than in discrete one off lessons where there is insufficient time to properly introduce, deconstruct, revisit or assess in a meaningful way. For me, the notion of a sequence or teaching and learning cycle feeds directly into the collaborative subject-based CPD we are planning for next year. Departments will be able to regularly review the relative strengths and weaknesses of a teaching sequence and teachers will be able to get closer to understanding how their students learn.

Worked example:

Below is a copy of my revised Elements of Language for writing, where you will notice I have reduced the amount of threshold concepts from five to four and slightly reconfigured some of the others.

Picture2

In the document below you will notice how the Year 7 standards (where as I suggested above, there might be some students who work to different standards in accordance to their starting point) has been drawn from the overarching threshold definition to coded objectives mapped out across the year’s units of work. Some of the information has obviously been simplified here for illustrative purposes.

Picture3

The document below outlines how the codified objectives across units are then broken down even further into specific to-be-learned items across a teaching and learning cycle.

Picture4

I think this model – where the unifying idea of threshold concepts is used to inform a mastery approach in the classroom – has the potential to be a very powerful driver of learning, particularly as it will be wedded to systematic and collaborative review by departments working collaboratively to better understand student learning.

In truth, there is a lot more to this assessment mode than I have explained, especially around its implementation and wider application in other subject areas. I am, however, minded about the length of this post, so if anyone wants to ask me a question in the comments below or tweet me a query, I would be more than happy to go into more detail.

Thanks for reading.

Advertisements

The Elements of Language: what we are using in place of levels

Image

In my last post I blogged about our department’s plans for a new KS3 English curriculum, which we are looking to phase in gradually starting this coming September.

This curriculum change is part of a wider set of reforms, in part a response to the shifting national picture, but in the main the result of a desire to transform the reading and writing competences of the students at our school. Changing the texts and sequence in which they are studied is a necessary first step, but this alone will not lead to significant rises in attainment if that content is not well taught or if there are not robust methods of assessment to purposefully guide instruction or to meaningfully evaluate its impact.

And so to the subject of this post: the nature of the model of assessment that we will be using to drive our ambitious plans forwards. It is not perfect, but what I am convinced about is that despite its inevitable shortcomings, it will prove to be a much better method of assessment than the ambiguous and imprecise system of levels that we are currently using. It will support learning, rather than distort it.

Formative and Summative

There are essentially two strands to this assessment model. One is concerned with measuring the progress of students’ over time (summative) the other, the more important, is a tool to support the class teacher in their ongoing understanding of student learning (formative). Michael Tidd is excellent on this distinction. The first supports the reporting process; the second supports the learners. Under this new assessment framework there will be one extended reading and writing assessment at the end of each year, which will take the form of an examination.

From these assessments students will be given an overall percentage for their performance over the two parts, which will then be compared against their starting point and their target for the end of the year. Regrettably, we think a baseline test is necessary. Whilst I sympathise with the valid arguments about retesting students at the beginning of year 7, we want to fully understand exactly what is behind the normalised numbers we will be receiving from our feeder schools. I appreciate this is not ideal, but for us, as I hope you will see, it is necessary: we want to know what our students can and cannot do so we can adapt our subsequent instruction accordingly.

The Elements of Language

The Elements of Language (see below) is the terminology that will allow us to articulate what we actually mean when we talk about effective reading and writing. Divided into 10 elements – five for reading and five for writing with corresponding assessment objectives – each element is embodied by a single word. So, for example, for writing there is A02 Control and A03 Style, whilst for reading A06 Knowledge and AO7 Interpretation. Together The Elements of Language define our notion of literacy and provide a genuine vehicle for a cross curricular focus on developing reading and writing – a shared language for talking about literacy and a practical means for understanding what it looks like.

Image

The Elements of Reading and Writing

The Elements of Language are divided into The Elements of Reading and The Elements of Writing. Each element has a corresponding Assessment Objective and has four stages of progression (see below). Within these four stages there are three clearly defined statements about the knowledge and understanding required to master. As much as possible we have tried to avoid vague skills definitions, which are unhelpfully imprecise, particularly as a means for helping students to understand next steps and to guide future instruction. This was more difficult to achieve with The Elements of Reading, which use some evaluative terminology in order to avoid an overwhelming number of specific statements.

Image

Assessment that drives learning

The creation of these three distinct objectives within each stage of progression is deliberate. It is designed to enable every unit we teach to work on one specific aspect from each overarching objective (or element) and to carry out this coverage in a coherent, systematic and rigorous manner. Across each term (we will run termly units) teachers will be focusing on teaching ten specific areas for improvement, along with responding to the learners’ needs as required. A simple tracker like the one below will help the teacher to maintain a firm grasp of whether students are learning the different objectives or not. Students will receive a 1 if they partially meet the objective criteria, a 2 if they fully meet it and a 0 if they fail to meet the criteria at all. These judgements will be made at the discretion of the individual teacher; they will not be tied to a specific piece of work.

Image

Because there are ten Elements of Language, and an on-going monitoring system that makes a 0-2 judgement, students’ progress can be easily transferred into percentages, both individually and per objective. We believe this highly visual and transparent terminology will give the teacher a clearer and more specific set of information they can act upon to inform their planning and to respond to the needs of their learners. It will also allow the co-ordinator to see if there are patterns of underachievement and if intervention is required. The specificity of our statements makes the understanding of English and how to get better at it much clearer: the students either demonstrate an understanding and application of a particular element or they don’t. This information will be available to teachers across the curriculum, particularly in the essay-based subjects as part of a shared planning model.

Interim and end of year assessment

Each term our students will complete one extended reading and one extending writing task, as well as a contextualised speaking assignment. Both extended writing tasks will be redrafted multiple times using the gallery critique model in an effort to establish a culture of excellence. Students’ work will receive regular, specific feedback; it will improve accordingly, along with their levels of motivation and self-perception. This work will not be graded. We are completely doing away with the notion of a half term assessment or APP task, believing instead that there are better ways to assess on-going knowledge and skill acquisition (see below) and that real progress takes a longer period of time to manifest– namely a year or perhaps even longer.

The Use of Multiple Choice Questions

I have already blogged here and here about the benefits of the multiple choice format, primarily as a means of informing teaching, but also as an effective method of managing the demands of marking – a real problem for so many, many teachers. As I have already outlined, the only extended pieces of writing that will be subject to specific assessment will be the end of year examination. Termly pieces will be produced but not be judged in isolation. Rather they will be used to evaluate whether a particular strand of an assessment objective has been met and if re-teaching or consolidation is required. Learning will need to be shown as secure as opposed to being performed in a one off piece.

Across a term, multiple-choice assessment will test the extent to which the focused elements have been learnt, or are on their way to be being learnt. Some aspects of reading and writing are easier to test using this format than others. The Elements of Language that perhaps lend themselves the best to multiple choice are Vocabulary (A01), Control (A02), Style (A03), Knowledge (A06) and Interpretation (A07).  Just to be clear, I am not suggesting these tests in and of themselves prove learning has occurred. They don’t. They provide an indication of the learning process and, most importantly, they provide a reliable guide for future instruction. Every class will sit these assessments and results will be used by individual teachers, as well as across the department to inform joint planning.

Limited, inconsistent, secure and exceptional

The end of year assessments (along with the year 7 baseline test) will be marked using our new KS3 mark scheme (see example for reading below). This mark scheme is broken into five different standards of performance, which we have termed ‘limited’, ‘inconsistent’, ‘competent’, ‘good’ and ‘exceptional’. These different standards – as much as humanly possible – match the four incremental phases of development within the separate Elements of Reading and Writing. I am aware that this system runs the risk of the ‘adverb problem’ as highlighted by Daisy Christodoulou here. I have wrestled with this conundrum for a while now: what is the best way to effectively judge a holistic piece of extended writing where different aspects (or elements) of English are synthesised? This mark scheme is my attempt at a response.

Image

Whilst I am not completely sure that it fully resolves the dilemma, I hope the way the standards are articulated at each level, and the relative specificity of the individual objectives, will make the marking clearer and more reliable. Obviously, robust standardisation and moderation procedures will also be necessary, as will exemplification at each standard. And this is exactly what we intend to do: exemplify what we mean by ‘exceptional’, ‘good’ and so on. To do this we plan to take the most accomplished student in the year above and use their exam response to set the standard for what is excellent’, which we can then rework downwards for ‘good’, ‘competent’, ‘inconsistent’ and ‘limited’. When a better response is produced this will become the new ‘exceptional’, thus ensuring the bar for what we expect from our students is always rising.

As with the termly tracker, at each of the five stages there are 2 marks available, 0 for not met, 1 for partially met and 2 for fully met. Again, like with the ongoing monitoring, the end of year assessments will be converted into percentages by combining the raw reading and writing marks. These final percentages will produce a transparent measure that will show the extent to which progress has been made or not been made. At this stage we are not fully decided upon what would represent a realistic, yet challenging, percentage target for the year. I expect it will be something like 10-15%, though this will most probably become clearer once we have implemented the assessment model and refined its workings.

A note on starting points

‘Exceptional’ is, of course, what we would like all of our students to be by the end of KS3. If they achieved the criteria that we have laid out then we truly would have instigated step change. Yet, we are realistic enough to know that this will not be possible for all, at least in the short term, perhaps even ever. To this end we want to make it clear that the minimum we expect our students to be is ‘good’ readers and writers, particularly those that come in at or around the normalised mark of 100 – what is deemed to constitute ‘secondary ready’. In our eyes this pretty much equates to our scale of ‘competent’. And this is why all those who come into our school at secondary ready will follow the second assessment pathway marked ‘competent’. Those below this will follow the greyed out area labelled ‘inconsistent’. There are no criteria for ‘limited’, since by its very definition ‘limited’ implies a considerable lack of requisite knowledge and understanding. We don’t need to define this.

And that is pretty much our new assessment model. It is still in draft format, so I’m sure there will be some glaring errors, typos, omissions and the like. We will also be making amendments and tweaks over the coming months.

We feel that we have come up with a model of assessment that is right for the students of our school and one that will actually help drive improvement, not get in the way of it.

I hope it is of use in some way elsewhere.

Why Wilshaw is probably right

Image

Like so many other departments there is often a tendency to give a disproportionate amount of time and resource to year 11, particularly at key points within the year and with certain groups of pupils. I’ve worked at three very different schools and in each case it’s been largely the same: the most support, the greatest amount of intervention and the best teachers are dedicated to one fifth of the main school population. In many respects this is understandable, and in such a high stakes accountability culture it is probably unavoidable. This is our present reality.

But every year I wonder if the situation could be different. I vow to change things in my department by investing more heavily in teaching and learning at KS3 in an effort to ensure that when pupils get to Year 11 they are much better prepared, more independent and do not need quite so much support or intervention. And every year it’s the same. I am never able to deploy my best, most experienced teachers to Years 7 and 8 because of circumstances beyond my control, such as unforeseen late departures throwing a spanner in the allocation works, or having to assign more experienced colleagues to marginal (but often tricky) GCSE groups. These teachers, who more often than not have other responsibilities, then don’t have the hours left for any KS3 teaching – a problem that has meant that I myself as Subject Leader have not had a year 7 or 8 class for over two years!

Don’t get me wrong: we review the success of KS3 at the end of each year – the curriculum, teaching, text choices, schemes of work, resourcing, methods of assessment, etc. I cannot tell you how many times we’ve tried to make APP work. It doesn’t, and we are currently in the process of designing our own method of assessment that does away completely with National Curriculum Levels, and gives pupils and teachers more meaningful feedback on progress. KS3 pupils are certainly not badly taught; there are some cracking teachers who teach years 7, 8 and 9: enthusiastic, committed and incredibly inventive colleagues who inspire their pupils every day.

But these teachers are often still in their professional infancy, and as might be expected from professionals still learning their art, they have not yet reached the height of their powers. Whilst the pupils in their classes make good progress, I wonder if they would progress even more in the hands of someone at the absolute top of their game, the 8 years or so teaching experience that Andy Hargreaves and Michael Fullan cite – in their exposition of the importance cultivating Professional Capital in raising standards – as the time where most teachers are at their peak.

Likewise, how much better would early years pupils do if I threw as much additional resource into their development as I currently do with Years 10 and 11 – resource that would probably have a greater long-term impact. I  I realise that teaching for 8 plus years does not automatically make you a great teacher. It doesn’t, and we all know of teachers whose years of working with teenagers have clearly had a corrosive effect on their passion and effectiveness. Yet one would hope that these instances are not the norm, and that with every year spent teaching young people – making mistakes, but fewer, and learning from them – teachers grow stronger: that their experiences make them better equipped to help pupils to make progress more quickly. In my experience this level of experience and knowhow is rarely let loose on Years 7 or 8, pupils who would probably benefit the most from the experience of teachers on top of their game, having arrived at secondary school eager to learn and capable of establishing a different ethos.

And this is why Sir Michael Wilshaw is probably right when he says that the brightest primary pupils don’t make enough progress in the majority of state secondary schools. Put bluntly, because of the accountability measures put in place by successive governments determined to lay claim to how their lot increased standards, the currency of more able students has had less value than those on the margins of whole school headline figures. This has had two perverse effects. One, that a disproportionate amount of time, expertise and resource is given over to one or two Year groups, and two within those Year groups higher ability pupils are less likely to receive the same level of support and intervention to help them excel. The best, most experienced teachers are often assigned to GCSE classes, where their brief is to get vulnerable (and valuable) groups of pupils over the line. Some pupils might never get to see the best teachers in a given department, unless of course by the time they get to Year 11 there are in danger of not making progress. 

To a certain extent, this is changing with the increasing use of more rigorous (and fairer) expected progress measures for subjects, together with holistic value added becoming a more of a yardstick for whole school success. Of course, this will help force schools to play closer attention to the progress of all pupils and not just those on the cusp of crucial outcome measures. But what is perhaps needed even more than this is a change in the culture of school accountability itself – a system that always places pupils, rather than numbers or statistics, at the centre of education. Because whilst schools continue to be held accountable to a limiting range of statistics and pitted in competition with each other, their limited resources will always be directed towards those areas that yield the greatest (most transparent) gains.

These gains may or may not be in the best interests of the pupils themselves.