Ben Fuller, Lead Assessment Adviser at Herts for Learning

Over the last half-term, my email inbox has noticed a bit of a rise in the number of queries along the lines of “I’m not sure I get page x of the new RAISEonline report – can you help?” or “What is this particular table telling me?”

I thought it might be helpful, with the permission of the enquirers, to share some of these brain teasers, along with my responses, as the chances are many others might have been wondering similar things about their own data (but perhaps were too afraid to ask!)

So, here is part 1 of what may well develop into a sequence of ‘Frequently Asked Questions’ (although, in truth, they might not have been asked frequently – but they have been asked at least once!)

Question: (relating to Key Stage 4, although the principle could apply equally to KS2 or KS1)

In the table below, how is it possible that the attainment figures for disadvantaged students are negative for every Prior Attainment Group compared to others nationally, and yet the overall figure for disadvantaged students shows a positive difference?

(i.e. our low PA group has a figure of -2.16, our middle PA group -1.06 and our high PA group -4.50 but overall we have a positive figure of +0.23?)

attainment8

Answer:

The explanation to this query comes down to the fact that the distribution of disadvantaged pupils in this school is different to the national distribution – this school has a large proportion of its disadvantaged students at the top end of prior attainment (6 out of 14 students i.e. 43%). This skews the overall picture. Those high PA pupils have a high attainment score (60.50) which has a strong positive impact on the overall attainment score for the whole group of disadvantaged (magnified because they are such a large proportion of the overall group) , bringing it up to 52.79 (above the national average for non-disadvantaged students). However when compared to students in the same prior attainment group nationally, their score is lower.

The opposite scenario can also occur in a school where the distribution of disadvantaged students is skewed towards the lower end of prior attainment. In this scenario, it would be possible that each PA group could be achieving better than the equivalent national figure, but the overall figure could still be lower than the overall national figure.

Indeed all of the above could also apply to the ‘All’ groups split by Prior Attainment, as well as to Disadvantaged groups. (It just happened to relate to Disadvantaged in this example.)

So there you have it. I hope that makes sense.

More brain teasers to follow. (Well, it is nearly Christmas.)