Radical assessment: avoiding the production line mentality

by Mario Creatura on May 16, 2011

When an academic decides to write a book, journal paper or article there are many questions that enter the mind: what area to focus on? What evidence to use? What structure to build upon etc.

I can guarantee with some degree of certainty that, when they open up the word processor and begin to write, that there will not be a clock ticking in the background and an absence of notes to guide them. I can also guarantee that they will not have had to rote learn their data or memorise their supporting references. I mean, surely it is logical that a test of memory is far more important than a test of interpretation and critical analysis?

The essays are intended to contribute to the body of knowledge in their field; a piece of work that will be ripped apart by peers which is designed to ultimately show off your academic prowess. But none are expected to be composed from memory, and certainly not to a designated time frame.

The examination of undergraduate students is farcical as a system of assessment. I’ve written at length before about my dismay with the illogical discord between ‘real’ academia and the artificial manufacturing of ‘designer students’ so I won’t dwell on this for too long.

It is counterintuitive to teach our students to be critical thinkers and evaluate sourced evidence to reinforce arguments only to test their aptitude using only the data that they can remember. So I ask a general question: why don’t we, as a sector, work to alter the systems that degrade the quality of the output?

It was with extreme joy that I read in Times Higher Education that by 2012 a Danish university will have taken the risky steps required to allow all students unfettered access to the internet during examinations. As the chief architect rightly says: “What you want to test is problem-solving and analytical skills, and students’ ability to reflect and discuss one particular topic. Internet software would allow lecturers to create tests that were aligned with course content rather than ‘trivia’ quizzes.”

Ms Petersen is absolutely spot on. Academics contractually placed in the role of ‘teacher’ have supplied definitive notes and comprehensive presentation slides to aid the rote learning or information. For too long reading lists have been generated and utilised as essential tools for passing, rather than guides for further experimentation of ideas. The progressive reform of the University of Southern Denmark allows differentiation to easily exhibit itself between those who merely regurgitate standardised responses and those who genuinely grapple and discern the useless from the useful.

For too long the mentality in HE has been that of a production line: x pieces of work at x grade with x number of exam passes = a 2:1 degree certificate. This is part of the rot at the core of the higher education sector. And it must be stopped.

The malaise of hyper-regulated admissions processes not recruiting those genuinely passionate about educational development. The notions of “I’ve paid so I am entitled to get a degree” and “university is for getting a job”. These combined with assessment protocols are choking the ideological power from our collective national intelligence.

The University of Southern Denmark has shown that, although innovative thinking may scare traditionalists, approaches similar to their internet initiatives are crucial to start the mending process and restore the high quality (and now largely mythological) publicly perceived prestige of ‘going to university’.

Can we adapt and improve archaic systems that have been around for hundreds of years? Can we turn our arguably highly educated minds on our own organisations and assess the philosophy behind each of the practices that we use to mark our quality as places of academia?

Schemes with the improvement of critical assessment at their heart should be encouraged and adopted by the global sector at large. The theoretical and ideologically driven desires for a better sector must translate and be applied to real life mechanisms. Otherwise the sector will turn into nothing more than a production line of cloned, identical graduates.

Oh wait…

{ 10 comments… read them below or add one }

dkernohan May 16, 2011 at 11:15 am

Dunno whether assessment is the the issue regarding the production-line nature of some modern HE. Assessment is variable, there is lots of it, in all kinds of styles. The issue comes when academics are either (a) using the wrong style of assessment to assess what they say they are (b) assessing the wrong things.

(a) is fixable, with a bit of staff education.

(b) is the wider malaise. If the idea is that “graduates” have a checklist of “stuff they are able to do” on graduation, and that this stuff should be the stuff that employers think they should be able to do – then we have a major problem.

It’s a trueism that employers know what they want now in terms of immediate business needs, they have no idea what they want in 3-4 years time, and no idea of the skillset they need their staff to have to move between the two.

If HE is about teaching “stuff”, then we are sunk. Stuff goes out of date pretty quickly.

StewartA May 16, 2011 at 12:38 pm

I agree the opening up of assessment can be positive – and the Danish example is a good, progressive one.

I think we need to go further back into core curriculum as well though, and look at the nature of what and how we prepare our students for the evolving real-world challenges… In this I think the example of Stanford’s D.School really takes some beating – innovation and creativity inbuilt for new ways of working and new approaches to solving the world’s challenges..!
dschool.stanford.edu

There’s a process in there, but it’s anything but ‘production line’…

Martin Hughes May 16, 2011 at 1:16 pm

What @dkernohan said.

Many types of job available in a decade or two are not yet in existence. We don’t know what the future holds, so it’s not possible to go beyond a general (though critical) set of considerations. NUS/CBI have just had a go at highlighting this:

http://bit.ly/ijJsUI

In terms of exams, et al, there is no single way to assess. It would be foolish to rule out any form of assessment if it is relevant. It is ensuring the relevancy that can be difficult at times.

The production line is not a norm, but it could become that way. I’d say our job is to steer things away from that as a matter of urgency.

Adam May 16, 2011 at 6:26 pm

An interesting concept, although for such a stratospheric change to occur in the heavily institutionalised and relatively immutable British education system in particular doubtless necessitates a concerted and innovative effort.

The reality that the present system has been around for long, and that it has become deeply embedded in the academic sphere, makes it difficult to envisage such a shift occurring. The logistic challenges of something like this would be huge, but the idea is commendable. This is definitely an area worth exploring, and the best way to do that would be small scale experiments, which would spread awareness and get the ball rolling.

Someone just needs to be the first to jump.

Pat May 16, 2011 at 7:39 pm

Does this argument not defeat itself in the first paragraph? If exams are bad due to factor x, or the absence of y – then surely this argument is applicable to journal papers (They are too small, inpenetrable to most people), books (I have to pay to read them) and so on.

(P.S in UX terms, this comment box is ridiculously small).

So exams are bad – but certain jobs would gain from the ability to recall knowledge accurately and quickly.

The production line is inherent in academia, because without exams the product still remains. If it is not a product, then why a degree? Why three years? The rotten core is due to a reliance on fordist approaches, not the system which is used to build the product.

To go against the production line would be to offer accreditation without attendance. I.E I can just come to exams and be assessed.

That’d be brave, but it’s too against the vested interest to happen.

@iamkarenwright May 16, 2011 at 8:06 pm

Taught MSc modules are a good way of exploring new ways to deliver and assess material before being confident to try in larger UG classes.

aheinemann May 16, 2011 at 10:30 pm

Thankyou for picking up on the Danish school, and @StewartA for d.school. So heres a few in return http://youtu.be/60OVlfAUPJg, http://youtu.be/HD4bpztESWw, http://youtu.be/zDZFcDGpL4U, http://youtu.be/fJFKE8kyz7w.

a culture, a society, a religion based on the abstract idea of utility maximisation, scarcity of means and assumed rationality – that posits these points as principles of life whilst engaging in the rhetoric around them. Problem is that its twisted rhetoric that claims reality, but doesn’t look at the “real-world” results of its pedagogy.

This article is great amongst many others of similar content. I am very interested in education pedagogies, methodologies, processes etc… and am active in my department to try and do something.

What I see though is many faults in the areas that are pointed out that need to be addressed and evolved, and a small but gradual number of reactions, but what I haven’t seen is a willingness to look at the real-worldliness aspect of the case in hand. To explain better with the metaphor used of production lines (which isn’t a metaphor but reality really as suggested), we must change, develop and innovate the production line. But on par – we must multi task our efforts, for if we put pre-processed stuff into the line – if we provide a ‘processed product’ friendly environment after then… Lets work to make education hubs take the first steps, but remember the value system inherent within all parts leans its decisive weight against any innovation if it does not question it also.

Just some thoughts

Newell Hampson-Jones May 17, 2011 at 10:05 am

As someone who is wildly against examinations, I am inclined to agree to the spirit of this, if not quite the solutions.

I also agree that this is unlikely to happen, but not just because the sector will change. External factors will play a very large role.

Look at Michael Gove’s assertion that examinations at the end of 2 years study are the way forward. Firstly, this is a ridiculous idea. Most importantly, though, this means our students are going to enter universities completely reliant on examined assessment.

We need universities to innovate, but it will be very difficult-if not impossible-if the rest of the education sector decides to take steps backwards.

Rachel Dearlove May 24, 2011 at 12:52 pm

I’m not a fan of exams as a form of assessment and I think they are often used in disciplines in which they are inappropriate.

In medicine and many sciences they are appropriate – I want my chemist/doctor/pharmacist to actually really know their stuff as well as to be able to think critically.

But in arts and humanities and many areas of social sciences they just aren’t actually assessing students learning – mainly just their revision techniques and memory.

As with other commenters above, I don’t think traditional assessment is actually the cause of the production line mentality – it’s been around far too long for that. It’s our universities’ cultures that have changed to focus on ends rather than experience.

Mario Creatura October 26, 2011 at 9:58 am

Leave a Reply

Previous post:

Next post: