Use of SOLO taxonomy to grade student written answers

Author: 
Charlotte Taylor
Institution: 
The University of Sydney
Year Level: 
First Year
Class Features: 

n/a

Key Assessment Issue Addressed: 

Characterising conceptual understanding

This work is part of an ongoing project looking at some implications to the changes in the Stage 6 Biology syllabus in the NSW secondary school system.  Although the initiative was part of an educational research project to investigate students' understanding of biology on entry to university, the model can be modified to work in a diagnostic process or as part of course assessment.

Investigating the conceptual biology knowledge of the students as they enter university requires us to choose one of the central themes of biology which has also been presented within the school syllabus within a variety of modules.  In this instance we chose to consider 'evolution' as our central theme and asked students what their understanding was of this area by posing the following question. 

"Much of Biology is about the way organisms have become adapted to their environment through the process of evolution. What do you know about adaptation?"

Students were given the opportunity to write a paragraph on this topic, which we could then analyse qualitatively.

Why did we choose the question?  According to Olsson (1999), this question is one that invites students to respond at a qualitatively high level.  The question is considered to be one that, in SOLO taxonomy terms, invites relational and extended abstract responses from students, as well as lower level responses.  SOLO (Structure of the Observed Learning Outcome) taxonomy is a model for qualitative evaluation of teaching and assessing (Biggs and Collis, 1982) and consists of five levels of increasing structural complexity.  These levels are called the pre-structural, uni-structural, multi-structural, relational and extended abstract levels.  This approach was used by Fraser (2005) to analyse the level of concept assimilation in ecology by biology students.    

We used a modified phenomenographical approach to categorise the answers since this provides a structure for examining the variation in understanding of the concept of evolution as experienced by the students (Marton 1986).  In this way a mass of qualitative data can be sorted into groupings appropriate to a quantitative analysis.  Such a process allows all perceptions of the phenomenon to be categorised into a list of critically different hierarchical categories (Trigwell 2000).  We modified this approach by using categories based on the SOLO taxonomy (Biggs & Collis 1982), such that the most sophisticated responses constituted a correct understanding of the concept (Webb 1996).  The creation of categories was done independently by two researchers reading and categorising student responses, then comparing their analyses prior to writing the level descriptors. 

The SOLO levels created were in alignment with the hierarchy as proposed by Biggs and Collis (1982) but were expanded to provide seven levels so that we could separate out two types of responses at Level 1 (= prestructural), and two at Level 2 (= unistructural).  The descriptions of the levels are:

 

SOLO score

SOLO Level Descriptor

 

1

No answer

Text Box: Increasing complexity of response

2

Misses the point; but there are written words, sometimes even biological but not relevant to question

3

Some comment on adaptations but not in an evolutionary context
Eg structural, physiological, behavioural

4

Description, key word, one idea; one relevant concept
Eg survival of the fittest; Darwin; idea of inheritance; natural selection

5

More comprehensive list, a group of ideas, often not clearly related; poorly explained

6

Relating change to environmental change; notion of time scale here; or bigger list
Eg. change + including an idea like survival of the fittest

7

Extended answer with a number of ideas, but no personal reflections.  Usually has relevant examples; explanations.

We scored the student answers and compared the 'performance' of incoming students in each of three years (2001, 2002 & 2005). 

Evidence of the Initiative's Effectiveness: 

n/a

Additional Material: 

References
Biggs, J.B. & Collis, K.F. (1982). Evaluating the Quality of Leaning. The SOLO Taxonomy, Academic Press, New York.

Fraser, A. (2005). The effect of using a woodland community analogy to teach about energy transfer in a cave community. Teaching Science, 51(1), 16-21.

Marton, F. (1994) Phenomenography. In Eds. Hussen, T. and Postlethwaite, T. N. The
International Encyclopedia of Education, 2nd edition, Volume 8, pp 4424-4429

Olsson, T. (1999). Qualitative Aspects of Teaching and Assessing in the Chemical Engineering curriculum: Applications of the SOLO Taxonomy. 7th International Improving Student Learning symposium, University of York, UK 6-8th September 1999.

Trigwell, K. (2000) A phenomenographic interview on phenomenography, in  J. Bowden and E. Walsh (Eds) Phenomenography, RMIT University Press: Melbourne, pp.  62-82.

Webb, G. (1996). Deconstructing deep and surface: Towards a critique of phenomenography for staff developers. Different Approaches: Theory and Practice in Higher Education. Proceedings HERDSA Conference 1996. Perth, Western Australia, 8-12 July. http://www.herdsa.org.au/confs/1996/webb.html

For Further Details
Contact Name: 
Dr Charlotte Taylor
Contact Institution: 
The University of Sydney
Contact Email: