Examinations

Description

Formal examinations are a feature of many, if not most, courses in the biological sciences. As examinations often contribute significantly to the overall grade for a course, these 'high-stakes' tests send strong messages to students about the types of learning most valued. The careful design and grading of examinations is therefore critical - yet this presents significant challenges for staff, not least in terms of efficiency where classes are large.

I should say, by the way, that I'm not necessarily advocating in any sense exams, multiple-choice tests, or any particular form of assessment for summative assessment except that if you've got 500 students, and you're the only lecturer, and you've got to assess them in a matter of three or four days, you have limited capacity for the type of assessment you want to do [academic]

In addition, while most examinations involve written work, many of the learning outcomes in the biological sciences involve skills that might be better assessed in another format.

While many staff and students described examinations in terms of written question types and supervised time-restricted completion, a range of other approaches were also discussed. These included both practical and oral examinations. We were told of innovative approaches to providing students with opportunities to prepare for examinations, and to the design of questions that encourages higher-level, critical thinking and analysis.

The role of examinations in the overall assessment design of curricula was described in a variety of ways, including the following.

Ranking - a means of comparing students' performances under the same external conditions
Authentication - avoiding plagiarism and 'cheating', ensuring that students' work is all their own [See also: Addressing plagiarism issues]
'Whole of content' assessment - encouraging students to review the subject /topic as a whole
Efficiency - assessment of large classes in a way which is more time / cost efficient than many other assessment methods

Examinations, test and quizzes share many common features. In this section we will focus primarily on those tasks that tend to be longer, higher stakes, and used as a significant measure of student learning. Short tests and quizzes primarily designed to provide students with feedback on their learning are discussed further in a separate section: Tests and quizzes.

Approaches

There is a good deal of conservatism in the design of examinations - and this is not necessarily a criticism. Examinations as an assessment type have a long pedigree and their role in measuring student learning is widely recognised. It is most commonly through examinations that academic standards are monitored - comparability of examination papers, from one year to the next, and between courses at different institutions in some benchmarking strategies.

Some degree of conservatism does not, however, preclude innovation. While it is true that when asked to discuss enhancing assessment, examinations were not the typical examples given to us, there were exceptions. Students extolled the virtue of oral examinations they had experienced in a third year subject. An academic staff member described how careful question design could serve to both encourage and measure critical thinking skills [Example: Mulder 2], while others spoke of extending true/false questions to require justification and thereby using these to trigger analytical thinking through extended responses.

Examinations are rarely optional. Unlike other assessment types, where students are sometimes offered a choice between different forms of presentation, topics for research projects or styles of written assignment, choice is not a strong feature of examination formats. There are, however, exceptions [Example: Sanderson]. We were told of one course where students can choose to sit tests at the end of each of the three topics comprising the curriculum structure, or to take a final examination worth 100 per cent. Last year, all but three students chose the periodic, topic-based assessment, saying that they benefited from the additional feedback and reduced pressure.

A variety of examination formats are used in the biological sciences and while written examinations are perhaps the most common form, these vary significantly in terms of question type and the degree to which students can use additional resources.

Written examinations

Written examinations typically involve a variety of questions types. In part, this is intended to provide students with a range of ways in which to demonstrate their knowledge. An even stronger influence is the need to design assessment which is effective and valid, within the reality of resource constraints. While open questions and extended essays are favoured by many academic staff and students, there are also the practical considerations surrounding the availability of suitably-qualified assessors and the time required to grade individual papers.

Some of the most commonly used questions types are described below.

multiple-choice questions (MCQs). MCQs are extensively used in examinations, particularly at first year and in large classes. They are typically described as expedient, but not always considered ideal. It should be noted however, that well-designed MCQs are powerful diagnostic tools, capable of identifying students' specific conceptions - and misconceptions - providing valuable feedback for staff and students alike. The design of effective MCQs requires disciplinary-expertise, insight into student learning - and time. Therein lies the challenge. The very resource constraints that make MCQs an attractive option in terms of grading pose equivalent challenges in terms of question design.
When used in examinations, MCQs are usually presented as large question sets in order to reduce the likelihood of students scoring well by chance alone. They are often used in combination with other question types such as short-answer questions.
true/false questions. While true/false (T/F) questions can effectively diagnose students' level of understanding of complex concepts, their use in examinations can be problematic. First there is the issue of reliability - with just two options, guesswork and chance are even more an issue than is the case with MCQs. In addition, the risk of ambiguity in the typically-complex statements used in T/F questions is high. We were told of two different approaches to overcoming these limitations. The first approach was to require a written justification for the choice of true or false. In this case, the questions were not dissimilar to short-answer questions. A second approach described to us involved penalty for incorrect answers. In this case, there was no deduction if fewer than 30 per cent of the responses to a particular question were correct - the reasoning being that this would help control for questions which proved confusing or ambiguous.
short-answer questions. Questions requiring a brief written response are widely used in examinations in the biological sciences. Such questions may require an explanation or description, and often allow for the use of diagrams in addition to text. The emphasis is upon demonstrating understanding, rather than writing skills - 'note-style' writing is often accepted, and errors in grammar and spelling tolerated.

images and diagrams. The use of images and diagrams in examination questions is common in the biological sciences. Such devices are used to assess knowledge of structure and function, and students' ability to analyse the data representations that form the 'language' of the particular discipline. For example, the simplest are questions requiring students to name or describe the indicated parts of a photograph, micrograph or diagram. Other tasks require more interpretation, such as explanation of data presented in a graph or the interpretation of a three dimensional structure from a series of sections. Depending upon the conditions under which the examination is taken, video can also be used to good effect.
extended answers or essays. Extended written responses are also a feature of examinations in the biological sciences, particularly in later years when the class sizes tend to be smaller and where increasing emphasis is placed upon students' ability to integrate and develop hypotheses. In timed examinations, it is usual to indicate the amount of detail required by indicating the amount of time recommended for a particular question. Some essay-style questions are multi-part, providing students with a structure and focus. In particular, we were told of an approach to essay question design for a third year class which reflects the research focus of the lecture, incorporates current literature, and thereby provides an authentic setting for the assessment task [Example: Mulder 2].
It is quite common for students to be offered choice in extended response or essay questions. For example, 'answer any 3 of the following 5 questions'. Choice is less common for MCQs or short-answer questions.

Examinations also vary in terms of students' access to examination questions in advance.

previously unseen. This remains the most common approach to examinations - the examination paper is secure and confidential until the time of the examination. It is common practice, however, to provide students with opportunities to practice for examinations. This may involve publication of past examination papers, specific 'practice examinations' [Example: Peat 6], or class time devoted to discussion and feedback on practice questions. A novel approach described to us involves a web-based activity with which students are able interact and test their understanding of assessment criteria for examination-style essay questions [Example: Gleeson].
provided in advance. In some courses, students are provided with an extensive list of possible examination questions and informed that a subset will appear, verbatim, on the examination. In a few courses, the entire paper is provided in advance. Questions provided in advance tend to involve extended answers or essays and to be highly integrative in nature.

Students' access to material during written examinations may also vary.

closed-book. Traditionally, the material a student brings into the examination room is limited - in some cases a calculator, dictionary or ISI data book is permitted, but under such 'closed-book' examination conditions, no texts, reference books or course notes are allowed. The emphasis is on students working with only the knowledge and understanding that they bring to the examination.
open-book. The use of 'open-book' examinations is not new, but this approach appears to be gaining popularity in some courses. Students are permitted access to additional material in the examination - usually texts or notes that they provide themselves, and less commonly computer access to web-based resources. One academic cited the enthusiasm of students - particularly the more motivated and engaged students - for this style of examination because it 'challenged them to think', not simply 'remember'. Such examinations make it possible to set questions which involve some degree of research, are not contingent on information students can be expected to recall, and are thereby able to be more integrative in their design. An extension of this is use of 'take-home' examinations, which are unsupervised and, in effect, untimed [Example: Edwards].

Practical examinations

Practical examinations, or practical assessment more broadly, does not equate with 'assessment of practical classes'. The assessment of practical classes is often in the form of a paper-based short test or quiz, a written report, or a written examination - and each of these is described elsewhere. A distinct form of assessment is one in which students are directly assessed on their performance of a 'practical' task - for example: a dissection; observations and recording through drawings; manipulation and interpretation of biological material; or a biochemical assay. Such practical skills are sometimes assessed in an examination format [Example: Walker 1].

Clinical assessment is a common feature of professional degrees. The questions and tasks are typically authentic and applied, and the assessment takes place in a clinical setting.

The range of approaches taken in assessing practical skills and practical classes is described further under Practical assessment.

Oral examinations (the Viva)

Oral examinations are not commonplace in Australian undergraduate education, in large part due to student numbers and resource constraints. We were told, however, of an innovative, group-based approach to oral examinations. In this example, third year students in a class of 40 were assessed in groups of 2-3. The examination was worth around 40 per cent of the final mark for the course, and the graduates who described it to us were very positive about the experience.

You had your unit outline and, for about a month before the exam, you'd sit down and you'd discuss the implications and various things. I mean, you didn't know what questions you were going to be asked, you couldn't go and look at past exam papers, you just had to understand the application of what you were doing.

We went in there, and we were just bouncing off each other, just having a round the table discussion - bouncing ideas ... it was alright for someone else to come in and say, "Look, I think you're wrong, I think this is the way it was," and [the assessor] really loved that. I mean that's what he liked to see in class, as well, and you'd sort of bounce off ideas, and it was very, very interesting and very enjoyable. Although I was terrified when I went in there! It was worth a reasonable amount ... and that was why it was so scary, because it was this new idea. But I think everyone I've ever talked to seems to remember that exam, remembers that unit as being a really different and practical unit. [recent graduate describing a third year oral examination]

Feedback

As many examinations are held at the conclusion of a course or a course component, examinations are not typically designed to provide extensive, diagnostic feedback to students.

From exams you don't get your feedback - you do that exam, and it goes off, and it gets marked, and you get the mark back, and that's it. You're just relieved that you passed, or whatever, and that's it. So you don't actually find out, you know, what you did well in, what you didn't do so well in, unless you ask. [postgraduate student describing undergraduate examinations]

While it is often possible for students to seek additional information on their performance, the default for end-of-course examinations is usually the award of a single score for the examination, and a grade for the course overall (most courses include multiple assessment tasks in a variety of forms).

The most common approach to examination feedback is through practice examinations - feedback for students in preparation for the final examination [Example: Mrongovius 1; Gleeson]. One academic described an approach whereby students complete a written examination under conventional examination conditions - invigilated, timed, and based on previously unseen questions. Subsequently, their answers are entered on-line. Where an answer is incorrect, students receive specific feedback describing the correct response [Example: Peat 6].

Examinations held during semester are, in terms of feedback, often quite distinct from end-of-semester examinations. They are more likely to include specific feedback. Such in-semester tests are described further under Tests and quizzes.

Issues and strategies

Recognised as a 'traditional' form of assessment, the use of examinations comes under considerable challenge from both staff and students. Many of the measures taken to reduce the time needed for grading examinations, such as the extensive use of multiple-choice questions, sit uncomfortably alongside academics' desire to assess higher order skills and critical thinking. Students' views on examinations tend to be mixed. On the one hand, examinations are a 'known' and are perceived as an objective measure of student performance - in a way that, for example, the assessment of groupwork or oral presentations is not. On the other hand, many students find examination conditions stressful and the lack of feedback an issue. The following section presents some of the issues commonly associated with examinations, and some of the approaches taken to address these.

Issue 1: The time constraints of examinations, with no access to additional resources, prevent students from demonstrating their capabilities and knowledge

One means of addressing this issue is through the use of open book examinations. We were told of an interesting approach to this, in which students compiled their own set of 'key points' during semester, developing individual, tailored resources for use during the examination [Example: Brightwell]. Several of the students interviewed remarked that open book examinations allowed them to give 'better quality' examination responses.

An alternative approach is to provide the questions in advance of the examination. Students commented that questions provided before the examination allowed for greater reflection.

Issue 2: Some students find examinations extremely stressful

The stress associated with examinations can be quite debilitating for some students, while others find it an important motivating factor.

I know when I get into an exam situation, I just start, you know, I get a panic attack. It's better now than it used to be. [postgraduate student, recalling undergraduate examinations]

I prefer exams .... I like the stress that comes with exams. The need to study for the exam, whereas for an assignment, I will put it off until the deadline is coming. [postgraduate student]

Some academics argue that requiring students to work under pressure is developing a valuable 'generic skill'. Not all agree, arguing that the pressures of timed, closed-book examinations are unlike the pressures of the work place, where reference material and colleagues are generally available for consultation.

Many simple strategies were used to help students prepare for examinations and, thereby, reduce their levels of anxiety. These included practice examinations [Example: Peat 6], preparatory tutorials [Example: Miller 1] and provision of diagnostic revision materials online [Example: Rogers 2]. Some academics dedicate lecture time to the discussion of examination questions, while others distribute questions from previous papers.

Students also develop their own methods for preparing for examinations, sometimes collaboratively.

Well, we used to prepare for exams in groups. We'd each take a question and answer it and then we'd share the notes and if you thought that someone had missed out some interesting points on that, you might add them in yourself, and it was good. We sort of had a team of maybe four of us and we would do that, and that was helpful. [postgraduate student describing preparation for examinations as an undergraduate]

Despite such initiatives, students will at times experience anxiety that affects their performance. This is ameliorated by having a mix of assessment types for each course - students who perform less well on examinations have the opportunity to demonstrate their skills and knowledge in other forms of assessment.

Issue 3: Students tend to cram for examinations, leading to a superficial understanding of the subject

This was the experience of some students interviewed.

I prefer assignments to an exam - an exam you learn just what you need to know, whereas for assignments you go through researching and learning - better retention rate [postgraduate student, describing assessment as an undergraduate]

The nature of the examination questions is a significant factor in the approaches students take to their study. If the examination requires the recall of large amounts of specific information, particularly if it is list-like or categorised, many students will adopt a 'list-learning' strategy and, arguably therefore, a 'surface approach' to the task. This is less likely where questions are more integrative or interpretative [Examples: Mulder 2; Burke].

I try to make sure they sort of understand the content of the theoretical side of the unit. So I try to, where possible, avoid just covering a single lecture in a question. So I try and integrate a couple of lectures together and try to see if they actually really understand it, as opposed to just regurgitating the facts. So I try and word questions in a way that they have to actually have understood the information rather than just 'can you remember six characteristics of this...' [academic]

Issue 4: Multiple-choice questions are perceived by some students to be 'difficult, confusing and not very helpful'

Multiple-choice questions (MCQs) sometimes contain ambiguity due to the subtlety of language used. Students from language backgrounds other than English, in particular, may find the language structure differences between 'correct' and 'almost correct' options difficult to distinguish. For this reason, the need to review MCQs - indeed all examination questions - for clarity of language and logic is paramount. Increasingly, institutional assessment policies require that examination papers be 'proofed', specifically for issues around the use of language.

We have to submit our exam papers before they are approved. The head of school looks at them, he looks at the questions and then he gives us feedback saying, "You have too many questions," or, "This is not clear. Reformulate." ... From there on we modify the exam. Then he reviews it again. If it's okay, then we send it to the examination office who reviews it as well [academic]

Issue 5: Multiple-choice questions are often set to reduce strain on resources, yet may not be assessing appropriate skills

Staff with large classes quite commonly use multiple-choice questions (MCQs) in examinations due to the ease of scoring such questions. Automatic scanning or online assessment of MCQs, in particular, is clearly a 'time-saver'. Many of the staff interviewed expressed a level of discomfort associated with this decision - the assumption being that MCQs are limited in their capacity to assess critical thinking and other valued skills and knowledge. We were told of one university that is allocating addition funds to first year teaching, specifically to enable assessment by short-answer questions rather than 'resorting to' MCQs.

There are two assumptions here, and both warrant exploration:

The first is the assumption that MCQs save time. It is certainly true that scoring a set of MCQs is much quicker and easier than grading an essay. This overlooks, however, the often-considerable time required in the preparation of MCQs. Well-designed MCQs that target common misconceptions and so assess students' understanding of important concepts in the biological sciences are difficult to prepare. They take time. As do MCQs that require students to integrate or synthesise information. One option is to draw pre-prepared questions from available 'question-banks'. Typically, however, these are insufficient, inappropriate to the course, ill-designed, not sufficiently secure for examination purposes - or a combination of these various shortcomings.
The second assumption is that MCQs encourage undesirable approaches to learning because MCQs are only able to test students' ability to memorise detail. While it is true that MCQs can be used to assess the lower levels of cognitive skills described in Bloom's Taxonomy* - knowledge level, and the ability to 'list', 'label', 'name' or 'recall' - well-designed MCQs can do much more, but they take time to prepare. Also assessment of knowledge, as opposed to 'higher order'* skills of analysis or synthesis, can be appropriate and deliberate. It is when this is inappropriate for the intended learning outcomes, or otherwise unintentional, that problems arise with the use of MCQs.

Well-designed MCQs are an extremely powerful assessment tool. They can identify, for both staff and students, specific aspects of misunderstanding. MCQs are not, of themselves, deficient - but their effective use requires careful design and consideration.

[*see also: Outcomes and assessment for more detail regarding Bloom's taxonomy]

Issue 6: Determining the appropriate standard of writing for written examinations

Varying emphasis is placed upon the importance of correct spelling and grammar in assessing written responses to examination questions. In general, however, the emphasis is upon demonstrating understanding - spelling, grammar and sophistication of language are considered less critical. This is due in part to an acknowledgement that the examination setting is stressful and so somewhat clumsy writing is 'to be expected'.

For exams, it's [grammar and expression] not so important. For other things like assignments and that, yes it's much more important. Because if you can't communicate something, then it's not very useful knowing it. But for an exam you're more looking for the understanding and the content. So as long as you can actually understand to a point that you know what they're trying to tell you, then it's not such a big deal for exams [academic]

There are, however, typically two requisites. There is the requirement for the writing to be of a standard that 'unambiguously demonstrates understanding'. Incomprehensible writing is not rewarded. A second requirement is for care in the use of discipline-specific terms and conventions.

While skills of written communication are highly valued in the biological sciences, assessment of these skills is typically left to written assignments. In addition, higher standards of written language are expected in later year levels - there is usually considerable latitude in the assessment of examinations in the early undergraduate years.

Some academics seek to identify students in need of additional language learning support early in the semester, and so direct them to the appropriate services and resources [Example: Cavanagh 1].

Issue 7: Consistency in grading essay-style questions is a particular challenge for large classes

For large classes, the need for deliberate measures to ensure consistency of grading is widely recognised. A number of strategies are employed in response. For example:

Assessors are allocated a limited number of questions (rather than the entire paper), and the number of different assessors to each question is kept to a minimum [Example: Hancock 2]. This minimises the need to coordinate between assessors, and provides an 'internal control' on consistency. The 'downside' for assessors is that grading the same question, hundreds of times, can be monotonous and draining.
Development and application of explicit rubrics or grading schemes [Examples: Hargreaves 1; Mulder 3]. It is vital to make criteria and standards explicit when multiple assessors are involved. Explicit grading schemes are similarly beneficial when a single assessor is grading large numbers of papers.

Generic skills

Unlike most other forms of assessment, examinations are not typically associated with the development or assessment of generic skills. However, depending upon the style of the examination and the types of questions used, this form of assessment can promote a number of generic skills, including:

Critical thinking. Carefully designed questions on written assignments can stimulate and assess critical thinking [Example: Mulder 2], while oral examinations are well suited to problem-solving and the critique of ideas.
Teamwork skills. Students explained to us how they worked collaboratively in preparation for examinations - such voluntary group work is likely to have a positive influence on the development of teamwork skills.
Written communication. The need to communicate ideas in written form, under the pressure of time, places a high priority on concise and clear communication skills. Although elegant prose may not be a priority, the need to convey understanding, simply and clearly, is.
Time management. Clearly this is a priority during timed examinations - and effective time management is also a key to successful preparation for examinations.

Authentic assessment

Clinical examinations and practical examinations can provide an authentic experience for students particularly in professional degrees [Example: Noble]. Some academics make a point of using recent case studies or their own research data as a basis for examination questions [Example: Mulder 2].

To reference material from this site, please use:

Harris, K-L., Krause, K., Gleeson, D., Peat, M., Taylor, C. & Garnett, R. (2007). Enhancing Assessment in the Biological Sciences: Ideas and resources for university educators. Available at: www.bioassess.edu.au

Enhancing Assessment in the Biological Sciences

Relevant Examples

Attachments