Monday, September 26, 2011

7 reasons why Kafka would have loved assessment

I’m in Prague this week, speaking to the world’s test providers . Prague was Franz Kafka’s city, whose unfinished masterpiece The Trial tells of the accusation, arrest and relentless pursuit of Josef K. He doesn’t know what he’s done wrong but the whole world seems determined to put him to trial and find him guilty. This is not far removed from the modern obsession with testing. Young people are in a perpetual world of exams and are not sure why the world is so determined to accuse them and find them guilty of not knowing huge tranches of weird stuff. Here’s just seven Kafkaesque features of modern assessment:
Sell ‘cheating’
We tell people tests are merictocratic but the test community sells ‘tests’ and also courses on how to ‘cheat’ those tests. Companies like Kaplan and others sell expensive courses that tell you literally, how to cheat. Imagine a business school that runs courses on banking, and at the same time sells courses on robbing banks! (Then again….) This is immoral and a sign that the tests are not what they claim to be, immune from improvement through tutoring and practice.
Mindless maths
We a dull & irrelevant curriculum, teach it badly, test it endlessly and wonder why they hate it or fail. PISA set the wrong pace, with STEM close behind, so politicians have become obsessed with the weird world of abstract maths, despite the fact that the vast majority of students will NEVER use the quadratic formula, surds, trigonometry or any algebra at any point in their lives. Abstract maths is easy to test, so the tests drive the curriculum. The majority of learners actually need ‘functional’ maths, fit for practical living, not the tiny minority that go on to do STEM degrees.
Summative is too late
The educational system is structured like horizontal layers of impermeable rock. The learner has to punch vertically upwards through these strata, with exams at every stratum, designed to halt progress for the majority. This obsession with summative assessment also means that formative assessment suffers. Teachers teach to the test. In short, we test too late, when the damage is done.
Tests measure failure
Precious few people get 100% in any test, so testing almost always tests below full competence. Why don’t we test until full competence is achieved, rather than accepting second-best? This is what simulations and games approaches do and therefore offer. Why can’t we go for systems of smart, adaptive assessment that assure competence at every stage?
Test and forget
Most exams test knowledge that will be forgotten within days or at most weeks. Ebbinghaus proved this in 1885, yet we still operate a system that follows the ‘cram, test, forget’ method. Part of the problem is the fact that we have abandoned ‘learning by doing’. Tests favour ‘knowing that’ as opposed to ‘knowing how’. Imagine an Olympics with only a few medals available for a few, pure athletics races and the rest are rubbish.
Luddites
Most tests are still done using pens. Two points: 1) few students and workers use pens in the real world, they use keyboards; 2) not giving students the chance to restructure and rewrite essay answers leads students to memorise and regurgitate set essays and answers. Critical thinking through writing is all about rewriting, so why not give them the ability to word process? We test using primitive technology that actually stops them from showing competence.
Abysmal quality control
Recent A-level exams contained impossible questions in a range of subjects from major test organisations such as AQA, OCR and Edexcel. Their quality control was so abysmal that they hadn’t tested their papers with even a SINGLE student. They claim to have statistically eliminated the problem by adjusting marks. Just how did they measure the distress and distraction in trying to answer an impossible question?
Conclusion
We test to blame, whether its students, schools even entire educational systems, which at times has led to a pathological view of education, and the demonization of state schooling. There’s so much testing going on, that relevance, innovation, skills, honesty and quality have gone down the plughole. We’re stuck with a Kafkaesque approach that is relentless, bureaucratic, accusatory and often tests the wrong things for the wrong reasons, killing the desire to learn. We’ve turned our children into a generation of Josef Ks.

6 comments:

Anonymous said...

You accept yourself that most students use keyboards, do you not believe it is vital students have a theoretical understanding of the tools they rely on?
Without basic algebra the technology that drives our information society is incomprehensible and student becomes disenfranchised.
I fully agree that maths should be an applied subject, but this is achieved by pushing past the ridiculously basic levels which are currently expected.
The equivalent is easily seen in language. Letters are a wired abstract concept with limited value, words start to become useful, sentiences are what defines us as a species and drives our society, but it takes the works of Shakespeare to emotionally engage and inspire.

Donald Clark said...

I'm sure there's a correlation between anonymity and banality. I have a television, radio, car and lots of tools in my life. It is not necessary to have a detailed understanding of how every tool works in order to use such tools. That would be ridiculous.

You're confusing the needs of a few with the needs of the many. On language we are grammatical geniuses at age three. We do not need to know the details of grammar to use language. Neither do we need to know Shakespeare to be emotionally engaged or inspired.

Mark Berthelemy said...

Not sure I agree on the maths side of things Donald.

I think you need to give people the opportunity to go beyond functional maths, otherwise how will they know whether they've got an ability in that area. The assessment is simply a means of proving that ability to others.

What this probably means, in practice, is that we need a functional maths qualification, and then a subsequent abstract maths qualification that not everyone will take.

Given the similarities, I would tend to put abstract maths in the same category as music and languages. They are all about learning how to communicate using a particular method.

Donald Clark said...

Mark - on maths I have no problem with pushing people but we push people to breaking point, at the expense of functional maths. In practice, real functional maths, is left by the wayside in favour of abstract maths leaving the majority floundering around in areas with no practical benefit. At least music and languages have some relevance in terms of practical and personal use.

Alan Barnard said...

This seems to be a very narrow view of assessment, so pity those students who have to do as you say. A very 1970s to 1980s approach. Though even when I was getting my secondary and tertiary education (1970s and 80s) I can now with hindsight see the true educators by their approach then. Now my students have to tell me what they need to know before I will teach, and what they learn is far more important than what they learn from me.
Assessment at the beginning, during and at the end (for high stakes purposes) does drive learning if and only if ALL participants are involved in the assessment thinking from the start.

Anonymous said...

Thanks for this.

I wonder why you think there is a necessity for assessments at all? Who are they for?