The Elements of Progression: threshold concepts meet mastery learning

Screenshot 2015-11-15 18.05.04

Last Saturday I was lucky to present at the first ResearchEd Literacy event in Swindon, organised by David Didau at the fantastic Swindon Academy. It was an exceptional day with some brilliant talks and a fabulous keynote by Ray Land on threshold concepts, sadly lost to posterity due to a technical problem. You will have to trust me, it was an exceptional introduction to the day!

I have been meaning to write my session up all week, but have not had the time to do so. Next week looks just as busy, so instead of waiting a couple more weeks, I am going to take the easy route and post my slides and a link to a recording of my session instead.

My accompanying slides

In many ways the content of my session was not entirely new: I have blogged here, here and here about our KS3 assessment model, and have also written two posts addressing the issue of threshold concepts in English here and here. Despite being relatively proud of these two posts, they are by far the least read efforts I have written. I guess, Miltonic Vision Part I: Trivium 21C, Threshold concepts and the power of ‘powerful knowledge’ was never going to really capture the imagination!

In the main my presentation summarised where we are with assessment as a department and indeed as a school, where the English model has been adopted by other subjects. It is certainly not a perfect model, and I am aware of some of the issues that will need to be amended in the future in order to increase both reliability and validity. What I do think, however, it that it is a much better, more purposeful means of assessing progress and driving up achievement than was ever the case with national curriculum levels.

A big, belated thank you to David Didau and Tom Bennett for putting together such a brilliant event, and to Ruth Robinson and Nick Wells for being so welcoming and well organised. It was a magnificent day.

Advertisements

ResearchEd 2015 – a sharper focus on what works

Screenshot 2015-09-08 18.49.53

I didn’t go to the inaugural ResearchEd conference in 2013. I did, however, get a flavour of the day from my Twitter feed and then, subsequently, from the videoed sessions kindly put together after the event. Ben Goldacre set the tone for an enthusiastic response towards the idea of teaching as more of an evidence-based profession. This desire for a greater research base to inform educational decision-making was perhaps understandable: for too long teachers had been subjected to every unsubstantiated whim of national policy makers, and had born the work load brunt of over exuberant SLTs eager to demonstrate how they were meeting the latest outstanding criteria laid out by Ofsted. Research provided a possible means to challenge the status quo and build something better from the ground up.

As is often the case, early enthusiasm can be followed by cynicism and doubt. And so it came to pass that at last year’s conference, which I was lucky enough to attend, there was a definite air of caution towards the trumpeting of research in education. If Ben Goldacre symbolised ResearchEd13’s rallying cry for education to become more like the medical profession, then Dylan Wiliam struck a more cautious note for teaching and evidence becoming bedfellows. His provocatively titled, ‘Why teaching will never be a research-based profession, and why that’s a Good Thing’ was a message heard in several other sessions I attended that day. Whilst most teachers generally felt an evidence-informed profession was a desirable thing – that was why there were there! – there was uncertainty about exactly what form it should take.

At Saturday’s fantastic third annual conference, held at South Hampstead High School in North West London, it felt like some of these doubts from last year had gone away, or rather that – at least in the sessions I went to or heard about – there was a growing confidence about how research could play a successful (and practical) role within education, one that also takes account of the legitimate concerns articulated previously – the lack of replicated findings; the prevalence of poorly constructed studies, compatibility with craft knowledge, etc. As distinct as each session was, it seemed that the different facets of the profession had begun to work out what their relationship to research and evidence should be, and that there was something close to consistency from these different stakeholders. More importantly, there was greater clarity on the practical benefits of research for teachers in the classroom and for school leaders looking to establish the conditions for great teaching in their schools.

I certainly picked up something valuable from every presentation, an idea or approach that I can and will take back to my school and apply in my context. I’ve picked out four of the sessions that I attended and identified what I took away from the session, which is likely to be idiosyncratic and reflect my own school’s concerns.

Can we learn anything from ‘top performing’ education systems? Lucy Crehan

I was gutted to miss Lucy speak at the Festival of Education earlier in the year. I had dragged @teachertweeks half way across the school to hear Lucy, only to meet with a shut door and a queue of disgruntle edu-punters. I was not going to make the same mistake again and so arrived earlier to get a seat. Lucy’s talk concentrated on the ways in which international evidence is often used incorrectly by governments to make policy decisions. Lucy has visited a number with different education contexts, including Finland, Canada, New Zealand and Singapore, and spent time digging beneath the surface headlines to find out a lot about how their systems work. Her accounts of different jurisdictions are fascinating and will no doubt make a great read when she gets round to writing them up in her crowd-sourced book, Cleverlands. There were many interesting observations about both the nature of different education settings and how data from these environments can be and has been misapplied by governments looking simplistically at other systems for solutions to their own educational problems.

# Takeaway point 1

In high performing systems that operate benchmark standards for student achievement Lucy talked about the amount of early intervention to address underperformance, often before the achievement gap gets too wide to do anything about. There are no higher standards (or objectives) for higher attaining students, though in reality they do learn, do and know more than their weaker peers. This struck a chord with me and helped me think more about how we get to grips with intervention whilst there is still time.

Hack your own teacher-researcher career – Becky Allen

I think it is fair to say that Becky is not really a big fan of everyone becoming teacher researchers; she believes ‘almost all teachers should never do education research’ – it is just too small scale and often poorly constructed to provide anything of any real value to the wider profession. In truth, she is not against the notion of the enquiring practitioner per se, just that it may not be the best of use of a teacher’s time and that it is never going to be the kind of thing that brings about system-wide change or the type of research that her organisation Education Datalab is interested in looking at. What Becky does like, and speaks passionately about, is big data: the size and scale of studies that can and should be used across systems to provide teachers with an evidence base to help them inform their practice. Becky’s talk outlined how those intent on contributing towards this kind of research could go about it, avenues open to teachers today that were not available to her when she forced to take a sabbatical to pursue further studies in research. Becky is an inspiring individual, though I tend to look more favourably on the potential of individual disciplined enquiry than she does.

# Takeaway point 2

The 10 step process that Becky used to explain how a willing teacher could get more involved in large scale research is a fantastic sequence of steps that I will definitely draw upon to help develop the lead learners at my school, and to encourage more teachers to engaged with research in all its different forms. We may not be conducting large-scale studies here, but we can get better at our own classroom enquiry.

IMG_2544

The swimming pool and the marathon: prioritizing cognitive and character development – Eric Kalenze

Personally, I thought Eric was a bit of a star turn. Not only was his talk excellent, but his passion for the event itself and his obvious amazement at what Tom and Helene have created over the last few years was a clear reminder of the extraordinary power of the ResearchEd initiative. Sometimes it takes an outsider to help you appreciate exactly what you have got right in front of you, even if it means that you have to unwittingly be part of a group selfie to prove to the folks back home that you exist! Eric’s talk drew upon one of the chapters of his recent book, Education is Upside Down, which though I have not yet started, am reliably informed is excellent. It is clear from Eric that the American education system is facing many of the challenges that are only too familiar to those working on these shores, such as the overcorrection to curriculum time for teaching soft skills caused by the Dweck-Duckworth juggernaut. Eric skilfully explored the pitfalls (and lack of evidence) of trying to teach non-cognitive skills in an isolated way, reminding us that even the notion of certain character traits may themselves be circumspect. For example, how many people have ever cleared their drive of snow out of intrinsic motivation? Eric doesn’t. His drive remains snow-free to ensure a wife happy and keep a marriage intact.

# Takeaway point 3

I learnt a lot from Eric about how a warm smile and upbeat manner can allow you to get in some pretty devastating critiques without ever appearing polemic or dismissive. His talk also helped me to strengthen my understanding that the best way to teach non-cognitive skills is through cognitive activities and that this is not an either/or situation – that we ‘have to embrace both poles…in full knowledge of the essential contradictions’ between cognitive and non-cognitive outcomes.

Exam marking and re-marking: what do we know and how should we use what we know? Amanda Spielman

This time last year I was Amanda Spielman’s bitch! Let me explain. Amanda is a meticulous presenter and likes to continually move backwards and forwards across her in-depth slides to highlight comparisons, show trends and rearticulate previous points in light of new information presented. Without a clicker, she enlists a nearby manual alternative. Last time round it was me; this year it the honour fell to one of her colleagues at Ofqual. For the love of God, someone please give this woman a clicker for future events. Daisy Christoholodou has already written about the complexity of this session, and it is certainly true that you need to be on it throughout her talk or you lose the thread of her carefully constructed points. I must admit, whilst she lost me at some parts, I nevertheless understood the main thrust of her presentation that the exam marking system and Enquiries About Result (EARs) process, such as it stands, is much more robust than anecdotal account often suggests and, unless we are prepared to invest absurd amounts of time and resources, is probably as reliable and valid as it gets. Amanda outlined some fairly extensive research that Ofqual have conducted into different EAR methods. Even up against single or double blind remarking models, the current EAR model held up strongly, showing the same or greater reliability.

# Takeaway point 4

Perhaps not practical, but I did take away from this session the view that oversight of the assessment process is in much better hands than we are often led to believe. It is clear that amongst the profession there is not really anywhere near enough understanding about assessment procedures and how exactly the whole process works. It was reassuring, though not without its problems, to learn that the Ofqual research into EAR processes revealed an unconscious bias amongst examiners against lowering students’ grades on remark, perhaps influenced by the importance of the grade for the students’ future. Much is being done behind the scenes to make exam outcome as valid and accurate as possible; we need to understand that if the assessment system continues to assess the range of skills and understanding we value, it probably can never truly be perfect.

IMG_2550

I have to confess I didn’t stay to the end. I have made a vow to myself that this year I am going to strike a better work / life balance. Getting home to eat with my family won over seeing the likes of Jack Marwood, Sam Freedman and, of course, Tom Bennett.

To be honest, I am not sure how much more I could have gained from this fantastic event, which seems to be continuing to go from strength to strength.

Thank you, Tom and Helene.

ResearchED Brighton: inside out not bottom up

Screenshot 2015-04-19 09.30.21

I have been to several ResearchEd events, but I have to say that I thought yesterday’s conference in Brighton was the best one, at least in terms of the amount and quality of ideas I took away with me. The high standard of the speakers certainly helped, as did the deliberate decision to make the event more intimate. It really did make a difference to be able to ask questions of the speakers and to share reflections during breaks. Once again, a big well done and thank you to Tom Bennnet and Hélène Galdin-O’Shea, and to the university of Brighton hosts for offering up such a splendid and amenable venue.

If previous ResearchED events have been characterised by a bottom up approach to the use of research in schools, today seemed to be more about working from inside out – a slightly nuanced adjustment to the metaphor of grassroots teacher professional development that I think better captures the way in which inquiry – in all its different guises – helps to grow the individual and, in turn, develop the organisation. However you frame the metaphor of what’s going in educational circles at the moment, these events sure do beat the stale training days in expensive hotels of yesteryear.

The keynote session was delivered by the charismatic figure of Daniel Muijs. His very pertinent presentation was about the extent to which it is possible to reliably measure teacher effectiveness. Drawing upon a range of international research, including some of his own as well the large-scale study into measuring teacher effectiveness conducted by the Bill and Melinda Gates Foundation, Mujis outlined the complex issues surrounding evaluating the performance of teachers. It was very clear that whilst for every measure there are advantages to be had, these often come at a considerable cost and lead to many significant undesirable consequences.

Screenshot 2015-04-19 09.19.25

Whilst the negative effects of using lesson observation for summative judgements are legion, Muijs did outline some of the ways in which it is possible to make them more effective, particularly if you are willing to invest the time, care and resource necessary to develop a coherent framework, such as the Charlotte Danielson model, and to train observers adequately on how to use it effectively. Even then, for observation to meet adequate standards of reliability and validity somewhere between 6-12 observations per teacher per year are required. I doubt there are many schools up and down the country willing or able to invest that much resource into observing every member of staff throughout the course of the year. The conclusion was that whilst some kind of balance of measures is probably best, this is still far, far from being perfect.

I was glad I stayed in the main hall for the next session, even though that meant missing out on what I later heard was an excellent session by Becky Allen on avoiding some of the pitfalls of testing, tracking and targets. In the main lecture hall Louise Bamfield and Paul Foster introduced the Research Rich Schools Website, a result of an initiative from the National College for Teaching and Leadership, which commissioned a group of teaching school alliances to develop a framework research and development tool in collaboration with the RSA. I haven’t had chance to properly investigate the site yet, but it promises to be an excellent resource, not only for designated Research Leads, but more broadly for teachers and organisations interested in developing their engagement with research and inquiry a stage further. The different levels of emerging, expanding and embedding seem helpful for supporting schools who are at different phases of development.

The next session was led by Andy Tharby on the ways in which his school, Durrington, have formed a partnership with Brighton University to support their teachers in running robust small-scale research projects. Originally the talk was to be co-presented by Brian Marsh, the school’s ‘critical friend’ from the university and from what I gathered a great bloke and fantastic storyteller. Unfortunately, Brian had to pull out at the last minute, but Andy carried on undeterred. Perhaps I am a little biased – I rate Andy’s blog and think he is excellent company – but it was really interesting to learn how his school are building up their engagement with research by matching it at different levels to teacher interest and expertise. Whilst he admits it is still in its embryonic stage, the many benefits of having a professional researcher to support, challenge and guide classroom teachers in conducting their own classroom inquiry were clear.

I don’t usually think of educational conferences in terms of their comedy value, but James Mannion’s presentation was a hoot! A combination of his own humourus and engaging style and the benefits of a smaller, more interactive audience, made this session both informative and enjoyable. James has spent the past 6 months or so working on developing an efficient and meaningful way to bridge the gap between educational research and classroom practice. He believes that ‘all teachers should systematically be engaged with professional inquiry’ and has developed a platform for this happen. The Praxis pilot platform, ‘launched’ at the previous Research Leads conference in Cambridge, provides an excellent online space for teacher to upload their own research inquiries, where they can then be shared and critiqued by others.

What I particularly like about James’s project is the way in which he has thought extremely carefully about how to make the whole process as efficient and as user-friendly as possible. There is an inquiry planner which follows a helpful format for thinking about and organising small-scale research.

  • Title
  • Context
  • Research Question(s)
  • Brief literature review
  • Avenue of inquiry
  • Research methods (how are you going to collect data? )
  • Findings / analysis
  • Conclusions
  • Evaluation

Screenshot 2015-04-19 09.20.20

Whilst I am not fully convinced about the overall aim of getting all teachers to be systematically engaged with professional inquiry (perhaps I simply need to know more about the terms of this statement), I find the sentiment behind it laudable and the effort expended on the project nothing short of remarkable. I can already think of several ways of incorporating James’s platform into the professional inquiry options on offer at my school. James will probably disagree, but I do see value in having a continuum of research options available for classroom teachers to engage with as part of their professional development. For James the word Praxis, as defined by Freire as ‘reflection and action upon the world, in order to transform it’ has much less baggage in educational circles than concepts like Lesson Study, practitioner-led research and disciplined inquiry. I am not so sure, and as Nick Rose pointed out, if anything it contains more of a trace of Marxist ideology. Anyway, for some, the small-scale teacher friendly Praxis model will be great, for others, models implied by the terms ‘disciplined inquiry’ and ‘lesson study may be more appropriate. Perhaps it is all semantics.

My day ended with Nick Rose’s wonderful session on different research tools he has developed to better facilitate teacher inquiry. In his role as research lead and leader of the coaching programme at his school, Nick has produced a number of excellent resources to better support the coaching process and help teachers to better understand what is going on in their classrooms. Some of these tools, all of which Nick stressed were for formative purposes only, included a classroom climate log, the use of student surveys and structured prompts to encourage focused self reflection on targeted areas of professional development.

For me, Nick’s session provided a lovely counterpoint to the findings about lesson observation made in Daniel Muij’s keynote, namely with regards to the different possibilities afforded to the profession from using observation as a formative practitioner tool rather than a high stakes judgement mechanism. I liked many of structured observation protocols Nick has developed on the back of Rob Coe’s work in relation to ‘thinking hard’ about subject content and poor proxies for learning. It was clear how these teaching and learning behaviours could be used as more proximate indicators of learning than the ones more commonly associated with Ofsted framework, particularly within a supportive coaching framework.

Those of you familiar with Nick’s fantastic blog, Evidence into Practice, will already know that Nick is an astute and incredibly meticulous thinker. His real life presentation style is equally impressive and I came out of his session with my head bursting with ideas. I can’t remember being so intellectually stretched by the complexity and range of ideas on offer in a session before, so when Nick announced at the end that ‘he has only just got started with this work’, I joined with everyone else in spontaneous laughter. Has there ever been such an example of ironic self-deprecation before? Probably not.

This was a wonderful day with wonderful people.

Thank you to all at ResearchED.