Offered by distance education
Assessment using a rubric (or matrix)
The teaching context
The subject concerned is a final year subject of the three year Bachelor of Applied Science (Equine Studies) course called Farriery. Farriery involves a study of the anatomy and physiology of the normal hoof, hoof care and shoeing techniques, common hoof and foot pathologies and correction of conformation and hoof defects using corrective shoeing techniques. It does not attempt to make students competent farriers but practical sessions do involve students handling horses' feet and learning to use farriery tools. It is offered in the final semester of the degree, so students' equine knowledge and research skills should be quite high. However as the subject has no prerequisites and is offered by distance education mode, it can be taken at any time outside of the suggested study sequence, so the knowledge level of the students cannot be assumed. The fact that it is offered by distance education also presents challenges in access of resources for this cohort of students.
The reasons for change
There are a couple reasons for the change in assessment practice. Although it is a third year subject it was lacking in academic content being a largely practical subject. Secondly the author had embarked on formal study of education, completing a Graduate Certificate in University Teaching and Learning (GCUTL) and was introduced to concepts such as pedagogy, authentic assessment, constructivism, situated learning and rubrics. The watershed was the concept of using assessment as a learning tool, rather than simply a vehicle to grade students. One particular subject was presented using problem-based learning where the assessment was the curriculum and students were assessed using both an authentic assessment task and a marking rubric. The rubric defines the criteria of the assessment system and gives an explicit description of these criteria and the performance characteristics against which students will be assessed (Andrade 2000). This rubric becomes a learning tool and enables the teacher to direct student learning to ensure content but allows the student to be self-directed as well. The student is self -directed in that they choose the depth and breadth of knowledge they wish to attain. The depth they chose to attain is then rewarded by the appropriate grade. It makes the assessment process much more transparent, rather than the student having to guess what the teacher looking for.
As part of studies for the GCULT, a pedagogy was chosen and studied in detail. The pedagogy chosen was situated learning as it appealed to one's aversion of spoonfeeding students information via didactic teaching. Situated learning (SL) forms part of the social constructivist view of learning which draws heavily on the work done by Vygotsky (Slavin 1991). Social constructivism purports that the learning process is not a solitary exploration of the environment but is a process of appropriation by the learner of culturally relevant behaviour (Kozulin & Presseisen 1995 cited in McInerney & McInerney 1998). That is, the learner is influenced and assisted by the social environment in which one is learning, facilitated by practical sessions. Central to the theme of constructivism is the notion that teachers cannot simply feed students knowledge. The teacher facilitates while the students construct the knowledge in their own minds (Slavin 1991). Facilitation is given by allowing or providing students opportunities to discover or apply ideas themselves. The teacher provides foundation information required by the students but it must be meaningful and relevant to the situation confronting the students (Slavin 1991).
Situated learning emphasises the idea that students should be given complex, difficult, realistic tasks and then be given enough help to achieve these tasks - rather than be taught little bits of knowledge that are expected someday to build up to complex tasks. Learning takes place in real-life authentic tasks (Slavin 1991).
Three criteria are consistent with authentic instruction. These are: 1) students build meaning and generate knowledge; (2) they use ordered inquiry to build meaning and (3) students aim their endeavours toward creation of discourse, products and performances that have value and meaning beyond the classroom (Newmann & Wehlage 1993). In order to satisfy these criteria, authentic instruction is based on attaining five standards: higher-order thinking, depth of knowledge, connection to the real world, substantive conversation and social support for student achievement (Newmann & Wehlage 1993). As a means of evaluating learning it follows that assessment should consist of authentic tasks.
As mentioned above the predominant pedagogy of Farriery is situated learning. Students are given complex, difficult, realistic tasks and then be given enough help to achieve these tasks. Learning takes place in real-life authentic tasks (Slavin 1991). Therefore the following assessment task was set as a starting point for situated learning.
Farriery Assessment Task.
The assignment task was based on a complex realistic task as a situated learning pedagogy states (Slavin 1999). The instructions were as follows:
You are responsible for the care of a well-performed athletic horse that has bilateral laminitis in its front feet. It has a 3º rotation of the pedal bone. You are required to manage the horse to return it to official competition at a high level.
The assessment task was broken up into sub-tasks to direct the student in his/her learning. The task was designed to develop the student's knowledge of the structure of the normal equine foot, provide a deeper understanding of the disease process, treatment of laminitis and management factors that may influence the outcome for the afflicted horse. The assessment allowed the student to consider the implications that laminitis has on the hoof structure and how it affects the well-being of the horse. Finally the student explored how the already compromised hoof may be subjected to other hoof diseases. Intrinsic to the assessment was that the student researched the current literature, utilising primary reference material. This developed the student's skill at being able to access and read the scientific literature and process it into a form that could be understood by the general horse-owning public. The student's general writing skills were also developed.
Although the assessment focused on one disease (it is recognised as one of the most important equine diseases), it enabled the student to gain knowledge in hoof anatomy, correct trimming and shoeing techniques by applying this knowledge in a real-life situation of understanding and managing a problem that they will very likely encounter in their professional lives. In gaining this knowledge the students were afforded the opportunity to interact with other equine professionals such as veterinarians and farriers.
Rubric for Assessment Task 1
|Description of clinical signs of laminitis||Thorough description of signs including all proven causes||Adequate description of signs & most proven causes mentioned||Broad description of signs & some causes mentioned||Mentions only one or two signs &/or causes|
|Identification of two different models for pathophysiology of laminitis||Detailed explanation of 2 models at a cellular level||General description of both models at a macroscopic level||Both models mentioned but only one explained in any detail||Only one model discussed|
|Comparison & contrast of healthy & diseased hoof||Differences & similarities made at both a macroscopic & microscopic level||Differences & similarities made at a macroscopic level||Some attempt made to point out differences & similarities but very generalised||Differences & similarities not identified|
|Treatment & management options for laminitis to return horse to competition||Most up to date treatments used, novel, sound management strategies developed||Treatment & management options current but limited to only one or two options||Treatment & management options out dated & possibly contraindicated||No treatment &/or management options given|
|Identification of other hoof pathologies||All plausible pathologies outlined||Most plausible pathologies outlined||Some pathologies not plausible when considering the existing disease||Little reasoning evident in choice of pathologies|
|Discussion of long term prognosis of the horse||Long term prognosis discussed in detail based on treatment & management strategies||Long term prognosis based loosely on treatment & management strategies||Long-term prognosis unclear, not linked to treatment & management strategies||Little or no mention of long term prognosis|
|Review of current scientific literature & practice||Emphasis on current primary references & current research (last 5 years)||A mix of primary & secondary references used, some a little dated||Heavy reliance on textbooks||Reliance on textbooks & non-reviewed tertiary references|
|Forum Postings||Provides both postings in an interesting & informative way, provides quality feedback to other students||Provides basic information in both postings, responds to in a limited capacity to other students||Provides one posting but shows little interest in other students experiences||Nothing posted & no feedback on other students postings|
|Structure (Andrade 2000)||Work has a gripping introduction, enlightening body & fulfilling conclusion||Work has an introduction, middle & a conclusion||Structure is vague but workable, topic is sometimes strayed from||Work is aimless & disorganised|
|Word choice (Andrade 2000)||Words used are striking but natural, varied & vivid||Some fine & some routine word choices||Words used are often dull & uninspired or sound like author is trying too hard to impress||Same words used over & over again. Some words may be confusing|
|Sentence fluency (Andrade 2000)||Sentences clear, complete & of varying lengths making interesting reading||Well constructed sentences, work flows along but does not grip the reader||Sentences are often awkward, run-ons or fragmented||Many run-on sentences & fragments make the paper difficult to read|
|Conventions (Andrade 2000)||Correct grammar, punctuation & spelling used||A few errors present but generally correct conventions used||Enough errors present to detract the reader||Numerous errors make the paper difficult to read|
The effects of the change
The rubric was very well received by the students. Informal feedback indicated they thought the lecturer had 'gone soft' by giving the answers to the assignment. They were intrigued with the idea that this was a learning tool, and thus found the idea of doing the assessment less daunting. They did appreciate that they were given more direction with where they should spend their energies in completing the assessment task. Students were able to spend more time exploring the information they found rather than spending the time trying to second guess what the lecturer might be wanting them to include. Students varied in how closely they followed the rubric but overall it resulted in a better quality of assignment. Interestingly there was still a good spread of marks; diligent students milked the rubric for every piece of information, hint, tip etc, while others put in varying degrees of effort. As it turned out a sessional marker graded the assignments and this not only gave the marker a very good idea of what students needed to include but took away the angst felt by students when someone unknown to them marks their work. One drawback with the rubric was that it was a little too prescriptive for the marker. Some students would do some things but not all things in the 'excellent' box and other things in the 'good' box so they were in between the two. This was overcome by assigning a number value for each category and giving half marks. For example Excellent was worth 4 marks, Good 3 marks, Average 2 marks and Poor attracted no marks; borderline between Excellent and Good scored 3 ½ .
The rubric was handed back with the student's assignment and provided feedback for the student. It limited the amount of written feedback that needed to be provided throughout the assignments, cutting down on marking time. As the students could clearly see how they were marked there were no disputes on the marks granted.
I can only say anecdotally that students' work had improved. Really there are too many variables to make a valid comparison. I have used rubrics subsequently and have found assignments are much better written and easier to follow. The rubric allows the students to get ideas and concepts organised in their heads so when it comes to writing there is a structure and a logical progression, rather than ideas all over the place. I encourage students to use subheadings based on the content criteria to ensure they cover all requirements.
Andrade, HG 2000, Using rubrics to promote thinking and learning, Educational Leadership, vol. 57, no. 5, pp13-18, [available online]
McInerney, DM and McInerney, V 1998, Developmental perspectives on cognition and effective learning. In Educational Psychology: constructing learning, 2nd edn, Prentice Hall, Frenchs Forest, NSW.
Newmann, FM and Wehlage, GG 1993, Five standards of authentic instruction, Educational Leadership, vol. 50, no. 7, pp8-12, [available online]
Slavin, RE 1991, Student-centred and constructivism approaches to instruction. In Educational Psychology - theory and practice, 5th edn, Allyn & Bacon, Needham Heights, MA