Module Handbooks and Audit Culture

This article is based on a small research project into module handbooks, or module guides. These are the fairly substantial documents which tutors at my institution (and at many others) give to students at the start of a module. Such handbooks contain some prescribed information, and much else as well. They are usually stapled or thermal-bound, and are essentially mixed texts (1), being simultaneously instruments of audit, and expressions of individual approaches to the craft of pedagogy.

From the point of view of the university management, of course, module handbooks are not mixed at all. They are legal instruments. They give students information, which students cannot then deny having received, and to make this transaction doubly secure the management prescribes what the minimum content of a handbook should be. Here is an extract from one university’s formula for a module handbook. It is not from my own institution, but it is so similar that it might as well be:

Synopsis of Module - Copied from module descriptor Module credit - Abstracted from module descriptor Module Aims - Copied from module descriptor Module Learning Outcomes - Copied from module descriptor Learning, Teaching and Assessment Strategy - A reference to the module descriptor Summative Assessment - Summary of assessments, with reference to Module Descriptor

And so on - there is more of this bureaucratic definition, more of the reliance on synthetic phrases like ‘module descriptor’, more of the cloth-eared carelessness which can’t make up its mind whether ‘module descriptor’ has capitals or not, or a ‘the’ or not. And there is the denial of the experienced reality of a module. As far as the formula is concerned, everything in a module handbook is referred to the ‘module descriptor’, not to the tutor or the students. In the corresponding document from another university, the fact that teaching a module is a transaction between tutors and students is panickily occluded in the instructions about module handbooks:

It is the responsibility of the Module Leader for each module to ENSURE that each student has been provided with a Module Handbook. (italics, and upper-case, in original)

This combines emphasis on What Is To Be Done with a refusal to imagine real people doing it. The ‘has been’ instead of ‘is’ is especially interesting. The language insists that the module leader must do certain things - but resists the image of the module leader in the act of doing them. ‘Is provided’ might image what really happens - the module leader gives the handbooks out, of course. But the present perfect tense redefines the tutor’s role as the bureaucratic function of checking procedure, rather than imagine the tutor doing anything which tutors actually do.

Module handbooks are instruments of audit culture, and of the ‘new managerialism’ which has been very critically discussed in recent issues of the Critical Quarterly, the Cambridge Quarterly and the Oxford Literary Review (2), and which has been subjected to anthropological critique by Strathern and others (3). But it is equally true that the managerialist prescription of handbooks does not succeed in standardising them all that much, beyond insisting on some prescribed content. At my own institution, there is a lot of variation in the size of English handbooks: from eight pages to sixty-two. This variation in length is greater in English handbooks than in many other disciplines, I was pleased to discover. Colleagues’ feelings about their handbooks vary a lot too. Some were resentfully compliant: ‘They insist that we do them, so we’ve got to - but it’s just another burden’. Others saw the handbook as a place to share enthusiasm: ‘It’s a first chance to sell the module - to get students interested in it.’ And there were colleagues who invested a lot of themselves: ‘I put in a lot of work in the module handbook; it’s a chance to show students that I care about them.’ This range of views about handbooks shows in the differences in length, and also in variety of content. As well as the material from the management prescriptions about learning outcomes etc., handbooks often contain accounts of the week-by-week structure of the module, with detailed advice about reading and preparation. Sometimes they have enthusiastic personal statements about the material of the module. In the Critical Quarterly (4), Richardson compares the ‘ideal university’ to the building of a Gothic cathedral, ‘with autonomous workers gaining pleasure from their labours’, and the variations in module handbooks show that even in instruments of audit culture there can be expressions of unalienated labour.

Students’ views about module handbooks

I distributed 60 questionnaires to students at Teesside, in all three years of an undergraduate English programme. There were 26 returns, most of them (16) from third year students. In addition, some members of staff at Royal Holloway (RH) kindly distributed some questionnaires for me there (and sent me some module handbooks), and there were 6 returns, which usefully make a small group for comparison between Teesside students and students at a pre- 1992 university. I shall go through the responses to the questionnaire, picking out issues for discussion where appropriate.

The first section of the questionnaire asked questions about good and bad handbooks. Question 1 asked students to think of a module handbook which they rated highly and, without naming the module, say what its good qualities were. The responses indicate that students value clarity and comprehensiveness of information most highly. The things students praise are: details of what will happen each week; suggested reading for each week; detailed information about assignments, in fact the nuts-and-bolts information about what work to do, when and how. Two representative quotations about good handbooks:

Lists module details, outline, set texts, aims, outcomes, attendance, assessment. It also gives a lecture and seminar week by week programme which is extremely helpful because I get to prepare for it. There is also a bibliography which is essential so I know where to focus my reading. (Teesside Year 3)
Sets out clearly coursework + assessment deadlines. The course outline is clear and easy to read as it is split up into weeks and describes what is expected of each week. It also contains a week by week breakdown, with suggested reading. (RH)

The answers to question (2), which asked about bad handbooks, were mirror images of the answers to question (1). Lack of clarity, vagueness and skimpiness were bad points. One addition was adverse comments on handbooks which were poorly produced. Handbooks which consisted of only a few pages, or were simply the required information from the module descriptor, or were poorly stapled together, were criticised. So were handbooks which were only available online. The picture which emerged was one in which students value the text of the handbook, want to keep it, and are sensitive to the amount of work which seems to have gone into it.

The second set of questions asked how useful module handbooks are, and how students use them. The responses show that students attach a lot of value to module handbooks, and see them as an important part of their learning. Students in year 1 English at Teesside thought that handbooks were very useful, quite useful or not useful at all in the ratio 4:2:0. Students at Royal Holloway, and in years 2 and 3 at Teesside, said ‘very useful’ without exception (with some variation like ‘indispensable’ and ‘couldn’t do without them’). The slight variation between first-year students and the rest suggests that students need to learn to use their module handbooks in the same way that they need to learn to use any other learning resources, but that the effort is worthwhile. Also, more advanced study means that students work more closely with their handbooks, especially with the week by week details and instructions which they value so highly.

Students described their use of module handbooks in very pragmatic ways: ‘To prepare for lectures and seminars’ (Teesside 3); ‘Knowing how to prepare for each week’ (RH); ‘As a weekly timetable’ (Teesside 2); ‘Know what is expected in terms of preparation and essays’ Royal Holloway (RH).

The next section of the questionnaire consisted of a brief content-analysis. The questionnaire set out a list of twelve items which might be found in module handbooks, and asked students to rate these as ‘very good ... good ... fairly good ... poor’. I then turned the responses into numerical scores, giving 3 for ‘very good’, then 2, then 1, with zero for ‘poor’. The aggregate scores are given below, with the items in rank order, not in the order they appeared on the questionnaire.

  1. Week-by-week schedule of topics 84
  2. Elaborated week-by-week schedule, with detailed information about questions for discussion 77
  3. Information about referencing 72
  4. Advice about assessment for the module 71
  5. Essay titles 70
  6. Lengthy list of secondary reading 70
  7. Primary source material 69
  8. Module learning outcomes 67
  9. Generic description of assessment bands 55
  10. Information about plagiarism regulations 51
  11. Pictures 29
  12. Quizzes, games 13

I shall pick out four issues for comment. Note that with 32 returns, and a maximum available score of 96 for any item, the table above has a familiar look because it roughly resembles a table of students’ results, expressed as percentages. Thought of like that, the results are two fails, two lower seconds, and a higher range from good upper second to outstanding first.

The fails
 Ludic or decorative elements of module handbooks score very low. The idea of a quiz in a handbook (they are rare, but occasionally occur) attracted scornful incredulity in the responses. One student added ‘???’ to her ‘poor’ rating, and another asked ‘what would be the point?’ And putting pictures in a handbook, which can be fun to do, doesn’t score much better. In these reactions, the pervading theme of students’ pragmatism is visible - if students regarded pictures in a module handbook as primary source material, that would be likely to score much better.

The lower seconds
The responses value information most when it is specific to the module, not when it is generic. Information about plagiarism is (at Teesside) part of the prescribed content of module handbooks, so it appears in all of them, and in students’ programme handbooks too. There were comments about this, and some students wrote that the information was unnecessary more than once. The same principle appears in the scores for generic assessment bands and module-specific advice about assessment: 55 for the generic information, 71 for the specific.

Learning outcomes

Module learning outcomes are a central issue in the whole debate about audit culture and the ‘commodification of learning’. The demand to state the learning outcomes of a module has been strongly criticised. One criticism is that learning outcomes are overly behaviourist:

When applied to learning, we have the situation where an internal process is described in terms of outcomes which can be construed as quantifiable products amenable to (pre)specification in tangible and concrete form. (5)

Similarly, Goodlad distinguishes between ‘training and the inculcation of low-level skills’ and ‘the tentativeness and “authoritative uncertainty” that is at the heart of the academic enterprise.’ (6) Learning outcomes are appropriate for the former, but not the latter.

Another criticism is that stating learning outcomes makes it impossible to deviate from the script, even when this would be a good teaching tactic:

...the presumption that the teaching and learning that takes place on the course can only be justified to the extent that it satisfies learning outcome requirements has the effect of stifling aspects of the course she is teaching that she sees as vital to its life. (7)

As well as these objections to learning outcomes, Hussey and Smith attack them from a philosophical point of view, and argue that they are really much less meaningful than their supporters claim. Hussey and Smith’s argument is that learning outcomes cannot be precise or objective accounts of what is learnt or demonstrated. Beyond the acquisition of practical skills (for instance in ascending capabilities in IT) they can only be meaningful in the light of already possessed knowledge of what is appropriate at different levels of learning and in different disciplinary contexts. ‘In brief, they are parasitic upon the very knowledge and understanding that they are supposed to be explicating.’ (8) One example is of a ‘critical evaluation’ of a poem (Hussey and Smith’s example is Hardy’s ‘At Castle Boterel’). If student A gives an account which discusses intellectual context, and connects this to a detailed and insightful discussion of the poem’s form, and discusses major issues in criticism about Hardy’s poetry generally, whilst student B writes that it’s old-fashioned boring rubbish, then both have produced a ‘critical evaluation’. The learning outcome cannot explain why student A gets a First and student B fails.

we praise one and dismiss the other only because we know roughly what is to count as a critical evaluation at this level: the descriptors themselves do not tell us this. Both teacher and student need this knowledge to interpret the learning outcomes. The student has to judge what is required by reference to the levels set by the teaching, seminar discussions, reading and so on. The learning outcome itself indicates only the general nature of what is expected. (9)

I find Hussey and Smith’s critique completely convincing (though I think it would apply with even greater force to things like generic level descriptors and generic assessment criteria). The students’ responses, though, suggest that they find some value in learning outcomes, even though I do not. They are evidently reading learning outcomes actively, making meaning from them, even though the author (this author at least) believes there is none.

Week-by-week schedule of topics

If stating learning outcomes for a module has attracted criticism because it stifles creativity in teaching, the same applies even more strongly to the detailed scheduling which students value so highly in their responses. Docherty, for example, objects to week-by-week schedules because they may be used for audit:

If my published schedule said that I would be discussing Racine, say, in week 6, then I had to make sure that when the inspectors arrived in week 6, my seminar was indeed focused on Racine. The fact that students might have become so engrossed in Moliere the week previously that they wished to defer discussion of Racine ... would only redound negatively on our QAA result. (10)

A similar distinction between what is prescriptively stated and what is pedagogically valuable runs through many of the contributions to the same issue of the Cambridge Quarterly. It is visible in Rosslyn’s contrast between the ‘On the face of it’ of a seminar and the ‘at another level’ and ‘At a more complex level’ where the real value of the experience lies. And for Rosslyn the prescription of a seminar is inevitably defeated by the fact that ‘For each student the memorable content of each seminar is likely to be quite different.’ (11) Hussey and Smith concede that preparatory information can be valuable, but object to prescription because it inhibits good teaching:

...unplanned diversions from the intended focus of classroom activities account for over 60% of changes that occur: good teachers seize on such moments. (12)

Yet the students’ responses attach very high value to detailed schedules of what will be discussed and what needs to be read beforehand. My own experience suggests to me that the students are right, and that a seminar is most likely to be valuable if the students are properly prepared, and, crucially, have the confidence which comes with knowing that they are properly prepared. Like most teachers, I greatly value the experience of a seminar shifting gear to a higher level, when tutor and students feel that they are sharing discovery. The detailed schedule is not an inhibitor of this, but an enabler, because of its effect on students’ confidence. Without this confidence, what looks like discovery is too often the timeworn tactic of shifting discussion away from what you don’t know about towards the safer ground of what you do.

The last section of the questionnaire asked about different varieties of language. I invented a module and wrote a paragraph from a handbook for it in three different styles (The point of inventing a module rather than using real extracts was to avoid biasing responses with students’ feelings about modules they had experienced. I chose an unusual field of study for the same reason). The invented extracts were these:

A. Literature and Social Conflict in Nineteenth-Century Russia

This module will deal with the following topics:

B. Literature and Social Conflict in Nineteenth-Century Russia

Welcome to this exciting module! It’s likely that you don’t know much (if anything!) about this fascinating subject, but by the time you’ve done this module, you’ll have learned a lot about some of the most famous (and some of the strangest) writers who ever lived. There are some broad themes in the module, all to do with the very turbulent politics of Russia in the nineteenth century and the ways that various writers engaged with political ideas and movements. We’ll start off by looking at the ‘Decembrist uprising’ of 1825 (it’s called the ‘Decembrist’ uprising because it happened in December!) and then we’ll go on to look at a range of political ideas and movements, and the ways they involved literary figures. One example is the conflict between people who wanted Russia to become a modern democracy like France or England (these were called the ‘Westernizers’), and others who thought that Russia had special qualities which were better than anything in the West (the ‘Slavophiles’). There are lots of exciting (and sometimes weird) ideas in the module, and you’ll find plenty of opportunity to develop your own interests as we move through this fascinating area. There are three essays to write for the assessment for this module, and you’ll need to read all the poems, plays, novels and short stories in the reading list. Then it’s up to you to decide what interests you most.

C. Literature and Social Conflict in Nineteenth-Century Russia

The lecture programme is structured as a spiral curriculum, to foster autonomous learning. Unit 1 approaches the question of political writing first through an examination of political events and the social and political context(s) of Russian literature in the period. The events considered are: the Decembrist uprising; the autocratic reaction 1825- 1855; the Crimean War; the liberation of the serfs in 1861; the assassination of Alexander II; the reign of Nicholas II. Writers considered in this Unit are Pushkin, Lermontov, Herzen and Turgenev. Unit 2 is based on political ideology, and especially theories of the relationship between literature and the autocratic state. Theories which will be examined are: Liberalism, Nihilism, Populism, religious reaction and Marxism. Writers to be examined are Belinsky, Saltykov-Schedrin and Dostoyevsky. The last two weeks of the course will be devoted to an examination of Tolstoy’s theory of history. Throughout the module, students will be expected to engage in collaborative learning activities. Assessment is by three essays, in which students will be expected to demonstrate appropriate learning skills and reflective and critical metacognitive knowledge.

 

These inventions were intended to exemplify some of the varieties of language in module handbooks. The first is bullet-point brutal enough to come from a ‘module descriptor’; the second adopts, and perhaps caricatures, very ‘reader-friendly language’, and the third is a running text version of the first, with some jargon added in the form of ‘collaborative learning’ and ‘metacognitive knowledge’, phrases which are on Wareing’s list of not-understood terms from the language of educational development.(13) The questionnaire asked students to rank the extracts in order of preference, and to add comments. To measure preferences numerically, I gave two points for being put first, one for a second place, and zero for a third place. The totals were:

Passage A: 20

Passage B: 6

Passage C: 22

This preference for anything but passage B was visible in the comments too. Comments on passage A included: ‘This is clear and concise. I know exactly what is expected of me.’ (RH); ‘This is all you need. Bullet points are easy to read - straight to the point.’ (Teesside 2). Passage C was praised similarly: ‘I like the way the topics are detailed and broken down into units.’ (Teesside 3); ‘Quite professional in language - being serious and students should take it seriously’ (RH). There was some comment on the jargon in passage C (one student complained that if plain language had been good enough for A. C. Bradley it ought to be good enough here), but the detail and professionalism of the passage made up for this. Passage B, though, was sharply criticised: ‘Very condescending’ (RH); ‘Completely useless’ (Teesside 3); ‘Long-winded and boring’ (Teesside 1).

The responses show again that practicality is very important to students. Even though much in passages A and C was incomprehensible to them (the Chto Dyelet? and the Narodnaya Volya as well as the jargon in C) the students showed a very high tolerance for the unfamiliar. Perhaps the fact that strange topics were stated directly meant that students felt they could begin work for the module, for instance by putting the strange terms into the library’s search facility.

Conclusions

My original intention for this study had been directed towards the relationship between Them (the management and the QAA) and Us (academic staff); including students in the project changed this model. Much of what students told me ran counter to my own beliefs and wishes. They showed much higher tolerance of, and enthusiasm for, aspects of module handbooks which I and others might dismiss as vacuous managerialism. Are learning outcomes valuable? Valuable to whom? They clearly are valuable to students, despite the fact that the rhetoric of ‘Quality’ says they are.

More generally, and to adopt Readings’ metaphor, can this exercise tell us anything about living in ‘The University in Ruins’? The students’ pragmatism and enthusiasm about module handbooks suggest at least that real work is going on in their part of the ruins. Managerial prescription about module handbooks may be irksome, but it does not succeed in reducing module handbooks to uniformity: far from it. And handbooks are very highly valued by their readers. Module handbooks offer an example of Readings’ tentative prescription for the future: ‘... to dwell in the ruins without belief, but with a commitment to thought...’(14)

 

Notes

1. I am using the term ‘text’ here in the broad sense in which Fairclough uses it. See Fairclough, N. Critical Discourse Analysis Longman, London 1995, pp.4-10.

2. Oxford Literary Review 17 Nos. 1-2 (1995).

3. M.Strathern (ed), Audit Cultures: Anthropological studies in accountability, ethics and the academy, Routledge, London 2000.

4. A. Richardson, ‘God, Ruskin and Management’, in Critical Quarterly 44, 4, (Winter 2002) pp.46-49.

5. G. Howe, ‘Universities in the UK: drowning by numbers’, Critical Quarterly 47, 1-2 (Spring and Summer 2005), p.3.

6. C. S. Goodlad, The Quest for Quality: Sixteen Forms of Heresy in Higher Education, Open UP, Buckingham 1995, pp.48-49.

7. P. Standish, ‘Towards an economy of higher education’, Critical Quarterly 47, 1-2 (Spring and Summer 2005), pp. 54-55.

8. T. Hussey & P. Smith, ‘The trouble with learning outcomes’, Active learning in higher education 3, 3 (2002), p.225.

9. Hussey & Smith, p.226

10. T. Docherty, ‘Clandestine English’, The Cambridge Quarterly, 34, 3 (2005) p.227.

11. F. Rosslyn, ‘Literature for the Masses: the English Degree in 2004’, The Cambridge Quarterly, 34, 3 (2005) pp.313-322.

12. Hussey & Smith, p.229.

13. S.Wareing, ‘It ain’t what you say, it’s the way that you say it: An Analysis of the Language of Educational Development’, English Subject Centre Newsletter 7 (November 2004), pp.13-17.

14. B. Readings, ‘Dwelling in the Ruins’, The Oxford Literary Review 17 Nos.1-2 (1995), p.23.

Newsletter Issue 11 - November 2006

© English Subject Centre

Previous article | Table of Contents | Next Article