Ben Fuller is Lead Assessment Adviser at Herts for Learning
This Friday our eager anticipation will be over and the new-look RAISEonline reports, showing the 2016 unvalidated data for Key Stages 1 and 2, will be released. (Interactive reports available from Friday 21^{st} October; Summary reports available from the following Tuesday.) Information has already been provided explaining the new-look tables and charts we are going to see.
Understanding the new pupil groups table may prove to be a challenge for some users of the data, with its 3 different types of national comparison all shown within the one table. (For some groups, such as gender, the group in your school is compared with the same group nationally. For other groups, such as disadvantaged children, your children are compared with the national opposite group i.e. the non-disadvantaged children. (This makes sense as this is the achievement gap that we are all trying to close.) And, for reasons that I can’t figure out at all, pupils who have Special Educational Needs or Disabilities are compared with all children nationally.)
There may be other challenges in unpicking these new reports which I have not yet considered.
However the main area I thought I would write about today is the Key Stage 2 progress scores.
This is an area that requires some careful explanation – and it is a critical area for many schools, as these progress scores are a key element in determining schools as being “below the floor standard” or “coasting” (when the data is validated in December).
The first thing to appreciate is that progress measures represent a child’s journey from their end of Key Stage 1 attainment to their end of Key Stage 2 attainment. Therefore any child who is missing data at either end of the journey (i.e. no KS1 data from 2012 or no KS2 test results from 2016) cannot be included in the progress measure.
Secondly, the calculation of prior attainment (KS1) has been changed – maths is now weighted equally with the elements of English when producing the Average Point Score (APS) for a child. So the formula for APS is:
(R + W + 2*M) / 4
where R, W and M represent the point scores for reading, writing and maths respectively.
The resultant APS figure is what is used to determine the prior attainment band that each child sits in. The KS2 Progress model will then treat each child from any given prior attainment band in the same way – although those children could be very different.
Here’s an example. The following 4 (fictitious) children have very different profiles of attainment across the 3 subject areas – but they all have a resultant APS of 18 points.
Reading | Writing | Maths | APS | |
Ali | L 2b (15 points) | L 2b (15 points) | L 3 (21 points) | 18 |
Beth | L 3 (21 points) | L 3 (21 points) | L 2b (15 points) | 18 |
Charlotte | L 2a (17 points) | L 2c (13 points) | L 3 (21 points) | 18 |
Daisy | L 3 (21 points) | L 2a (17 points) | L 2a (17 points) | 18 |
(These are not the only combinations that will produce an APS of 18. You could for example have a child who achieved Level 3 for both reading and maths and Level 1 for writing – but that would probably be quite rare.)
Now, to calculate any individual child’s progress measure, you need to compare their KS2 test outcomes with the national mean average outcome for children in the same prior attainment range. This document from the DfE provides this information on pages 16/17. (NB there are ready reckoners available in the RAISEonline Library which you can use to do all these calculations for you.)
The document tells us that the national average KS2 scaled scores for children with an APS of 18 are as follows –
Reading: 106.8
Writing*: 104.7
Maths: 106.3
* Yes I know Writing wasn’t a test. Scaled scores have been allocated to each of the categories used in teacher assessment for the purpose of the progress measure. Discussion of this particular issue can wait for another day…
So, let’s say that Ali, with her 2b in reading at KS1, achieved a scaled score in her KS2 reading test of 104. We might be very pleased with that. But her progress score would be -2.8, because the expectation, based on her KS1 APS of 18, is 106.8.
Other children with a 2b in reading at KS1 achieving the same scaled score of 104 could have positive progress scores – for example, a child who was a 2b in all three subject areas at KS1 would have an APS of 15. The average KS2 reading scaled score for that prior attainment is 100.6. So if such a child scored 104 they would have a positive progress score of +3.4.
So, to clarify, it’s not about comparing subject to subject (e.g. comparing KS1 reading to KS2 reading). The prior attainment measure takes into account all three subject areas at KS1.
One reason why I have used an APS of 18 to illustrate this point is that that score represents the cusp of the ‘high prior attainment’ group, in the RAISEonline reports that display your KS2 data split into the 3 prior attainment categories. (‘Middle’ is from 12 to 17.9 and ‘Low’ is below 12.) Focusing on Ali in the table, she clearly has high prior attainment in her maths, but was very much an ‘average attainer’ in reading and writing. Beth’s data is the opposite. But all four children in my example are counted as high prior attainers for all subjects at KS2.
However, interestingly, the information telling us what to expect in RAISEonline this year shows that progress scores will also be shown by pupil groups determined by their prior attainment in each subject area.
Curiously though, this table appears to imply that this subject-specific prior attainment grouping will not be limited to the subject in question, i.e. looking at KS2 reading outcomes broken down into groups based on children’s KS1 reading assessment – it suggests that we can also look at the KS2 reading outcomes broken down into groups based on children’s KS1 results in writing or in maths. I can’t think of a logical explanation for this and can’t help but wonder if something has been lost in translation here, between the team that decides the basic principles behind the data they want to show and the team in charge of putting it all together.
In addition to all of the above, we can also expect some new look scatterplots in RAISE, in which progress (rather than attainment) is plotted against prior attainment, and some EYFS to KS1 transition matrices. (Just when you thought transition matrices were a thing of the past, as they have gone at KS2…)
All of which is making me think that, when earlier this year I was confidently telling colleagues that I thought that RAISEonline would be much simpler and more concise this year, my confidence might have been misplaced and I may have to eat my words. It looks like they have succeeded in creating another beast of a document. Thank goodness for FFT.