From a reflective audit of assessment feedback practices emerges a framework for ‘paving the way for feedback’

Abstract

Faced with my own inability to think of a full response to a question put to a panel at a recent conference about the nature of effective feedback for students whose first language is not English, I was motivated, following a scholarship of teaching (henceforth SoTL) approach (see e.g. Prosser, 2008), to conduct a reflective audit of feedback practices, on a group of seven undergraduate academic English modules. The aim was to investigate the feedback practices and explore the currently somewhat tacit rationale for these. Then to evaluate the extent to which they were integrated into course design to effectively target the specific needs of English as an additional language novice undergraduates (henceforth EAL students). The audit did reveal that the feedback practices, which had grown somewhat organically over a number of years, are generally fit for purpose, but more interestingly, that a significant number of the practices provide a scaffolding function to cater for novice HE students unfamiliar with assessment and feedback processes in the UK. The feedback practices therefore, aim to prepare students for their future assessment experiences. The reflective audit resulted in the development of a framework for paving the way for feedback for the students in question. This serves as a conceptual framework for planning, monitoring, targeting and developing assessment feedback practices. It is hoped that the paving the way for feedback framework may inform the practice of others.

Fiona Gilbert and Garry Maguire, Oxford Brookes International, Oxford Brookes University

 

Introduction

At a recent event on feedback organised by the Assessment Standards Knowledge Exchange (http://www.brookes.ac.uk/aske), the panel were asked for advice about feedback practices when dealing with a large proportion of international students in an institution. My role at Oxford Brookes University is to support undergraduate EAL students in their studies so the question was of particular interest to me. The answer offered by the panel was to be mindful of lecturers’ use of language and to follow feedback good practice, for example, legibility, timeliness and dialogue, as you would for all students.

Although at the time I felt the panel’s response reflected my feedback practice on the academic English modules that I am responsible for, afterwards I began to ask myself – are those elements of good practice the only things done in terms of assessment feedback for EAL students on the modules? Especially given that the core function of all feedback on these modules is to feed forward to future assessments? To fully answer that question prompted me to undertake an audit of feedback practices. The audit was focused exclusively on the pedagogic practices in operation themselves and did not include either an evaluation of related staff induction activity or an analysis of the feedback provided to students.

The aim of conducting the audit of feedback practices was to uncover the implicit feedback practices, to evaluate whether they were fit for purpose, were following recommended practice and to what extent feedback was effectively integrated into course design to meet the target student needs.

Method  

The method followed a scholarship of teaching approach (O’Brian 2008; Prosser, 2008; Trigwell et al, 2000). The main aim was to improve student learning and the methodology encompassed Gibbs’ (2008) full spectrum of SoTL activity to include a reflection on personal philosophy and on practice, both of which were informed by an exploration of the relevant literature and driven by pedagogic research on feedback practices. This began with reflecting on my learning and teaching context, my knowledge of the EAL students’ previous experiences of feedback and the type of feedback practices they may experience during their studies at Oxford Brookes. I then looked at each of the seven modules I am responsible for and listed the assignments on those modules. The feedback practices are often implicit, therefore I systematically went through each individual assignment using the student handbooks, the assignment briefs, the tutor’s guides and the pedagogic resources to uncover and itemise the feedback practices. Each of the individual lecturers in the team were interviewed to check their exploitation of the practices available within the curriculum to verify implementation and pedagogic focus of each to ensure they were ‘intentional, coherent and recurrent’ (Price et al. 2012, p. 144). From the list of active feedback practices it became possible to tally and group these depending on what stage of the assessment process they were practised and the pedagogic purpose they served.  This provided a sound basis upon which to reflect on their effectiveness.

Reflection on context

The assignments undertaken on the academic English modules are, for the majority of the undergraduate EAL students and regardless of stage of entry, their first UK HE assignments and their first experiences of feedback. Each student brings different expectations to the assessment experience derived from their culture and past educational experiences and other factors (see for example, Scudamore, 2013, p. 6; Ryan, 2000, p. 44). Novice EAL students, indeed all novice HE students, may have expectations about feedback which are at odds with actual practice (HEA n.d.; McLean and Ransom 2005, p. 58 cited in Carroll and Ryan, 2005).

Novice HE students may not only be unaccustomed to UK HE feedback practices, but the practices they will encounter may vary. Feedback practices not only vary between individual institutions, they can even vary between different tutors, who for example, can have different feedback styles and feedback practices (see Handley et al., 2008. p. 34). Therefore, it is important on the academic English modules, not only to ensure students are aware of what they can expect in terms of feedback practices for each assignment on these modules, but also to expose them to as many means and modes of feedback as possible to prepare them for potential future feedback encounters.

Outcomes of the reflective audit of feedback practices

The feedback practices described in my particular context were grouped, itemised, described and illustrated. The general framework resulting from this reflective process is shown in Figure 1. This shows the four phases of the assignment process in terms of feedback practices. The terms awareness raising, familiarising, training and engaging refer to the type of cognitive processes which may more commonly be employed during each of the four phases, but are also likely to be targeted at any phase. Although listed in a one directional, linear manner, the process, as the diagram helps to show, is recursive and iterative in nature.

Paving the Way for Feedback Framework image

Figure 1: the paving the way for feedback framework

A clarification of context and terms should first be made. The aim of feedback in my particular context is primarily to play a formative role. All the feedback, whether at first or final draft stage, aims to feed forward to students’ assignments in their discipline. Therefore the practices often apply to feedback feeding forward (see Price et al., 2012, p. 109). Mode of feedback refers to written or oral feedback. The source of feedback can be the teacher, peer or in fact, self. Feedback can be given using a variety of means, for example, written comments on a text, overall comments, use of a feedback sheet, audio feedback, face to face dialogue, screencasts, or links made to lecture material (see Price et al., 2012, p. 116). Feedback can be individual or generic.

Pre-task feedback practices

This refers to general awareness-raising to the nature of feedback practices that occur out with an individual assignment. Examples of these practices included:

  • Raise awareness of assessment feed forward feedback processes. For example, discuss past feedback experiences and what students understand by the terms feed forward and feedback, discuss what students can expect (where, when, how they will get feedback?) See example sources of feedback table mode (written / oral), who they will get it from, what they should do with feedback. See Feedback: Make it work for you! Leaflet
  • Raise awareness of the first and final draft assignment process, the rationale and how it works in practice. Explain the source, breadth, extent, modes and means of feedback students can expect at each stage. Discuss and compare with previous experience.
  • Raise awareness of the processes used to assess assignments. For example, standardisation, moderation, the external examiner’s role, and, anonymous marking, if used. This process can be stated in the student handbook as well as in lectures.
  • Raise awareness of the marking grade scheme used at the institution. This includes the terms used, the grade range, the average marks for different stages of study and compare this to grade scheme students are familiar with. See Managing Student Expectations Case Study.
  • Raise awareness of Turnitin and Grademark (if used). Show sample assignments and modes of feedback within Grademark. Explain how Turnitin is used formatively and what a Turnitin originality report means (see How to Interpret Turnitin Reports, Resource )
  • Raise awareness of the language of feedback by providing a glossary of assessment and feedback vocabulary.  For example: feedback, feed forward, learning outcome, first draft, final draft, assessment criteria, weighting and so forth. A glossary can be used from your own institution or prepare your own to suit your context. 
  • Raise awareness of the need for a sense of enquiry. Encourage students to ask questions about feedback. Compare to other educational cultural norms. Explain who to ask, whether it be, for example, the module leader, the tutor, a discussion forum on VLE, short Q & A slots at the end of a lecture, questions on slips of paper for groups of students and/or the lecturer to deal with. Manage student expectations by explaining what it is appropriate to ask for and what students cannot expect, for reasons of equity and staff workload, for example. See Face to Face with Freya

Task setting feedback practices

This refers to feedback practices that refer to an individual assignment when it is set. Examples of these practices included:

  • Familiarise students with assessment criteria and feedback sheet band descriptors. This includes why they are used, what they mean, the terms used, how they connect to the module learning outcomes. Unpick terms such as ‘discuss’, ‘critically evaluate’ and warn students of such words having multiple meaning depending on the subject/ module/tutor. Provide a glossary of assessment and feedback terms.
  • Familiarise with type of texts required and associated feedback sheets. Exploit exemplar texts. Ask students to a) assess the exemplar and b) give feedback on the exemplar text. This helps familiarise students with the assessment criteria, the feedback sheets, the grade scheme and also fosters self-review and self evaluation (see Price et. al., 2012).
  • Familiarise with peer review and peer feedback. Useful phrases for peer feedback could be fed into the peer review process, for example, ‘You might consider……’, ‘In this section do you mean …. ?’, ‘this section could be improved by ….’. These are particularly useful for EAL students who may be unfamiliar with the language as well as the practice of peer review. See ‘Making Peer Feedback Work in Three Easy Steps’ leaflet and Peer Review Checklist
  • Familiarise with staff expectations and manage students’ expectations in terms of feedback for each stage of each assignment. Students need to be aware of decisions staff has taken in terms of giving feedback. For example, regarding the breadth of feedback with large groups of students, feedback may solely focus on areas to improve on at first draft stage and not include areas performed competently (for workload reasons) so students need to be aware of this. Price et al. (2012, p. 115) refers to this as aligning expectations of staff and students. See First draft feedback good practice guidelines

On-task feedback practices

This refers to feedback practices undertaken on returning a first or final draft assignment. Examples of these practices included:

  • Train students to navigate the sources, modes and means of assignment feedback to maximise take up. Feedback can consist of a range of different unfamiliar elements (as referred to above) so it is important students are helped to navigate these by, for example, using an exemplar parallel text with feedback and show students how to access and interpret the feedback.
  • Train students to use self study resources on aspects of feedback on the VLE. For example, short clips of students discussing their feedback and how they used it. For example Spotting Feedback.
  • Train students to exploit all sources of feedback. For example, provide generic feedback on the assignment – this can be given orally and written. If the same assignment type is used each semester, generic feedback on an assignment for one cohort can often become generic feed forward for the next cohort. The generic feedback from the last cohort can even be distributed and discussed at the setting an assignment stage. The more generic feedback that is given, the more individual feedback can focus on individual needs. See ‘Using Generic Feedback Effectively’ leaflet.
  • Train students to self review. For example, when giving generic feedback, ask students to identify which feedback points apply to them. Dictate the list of feedback points and ask students to write down only those points they think apply to their work. This also helps to personalise generic feedback, which is rated as the least valued by students (Handley et.al., 2008, p. 21).

Post-task feed forward practices

This refers to fostering a cognitive environment for feedback to feed forward into the next assignment. Examples of these practices included:

  • Engage students by encouraging to pay attention to and interact with the feedback they receive. For example, students write down/discuss which feedback comments they are prioritising working on for the next assignment or make a ‘feedback action list’ in order of priority. This could be done on the return of an assignment, at a short period after that to monitor student engagement with feedback, or at the point when the next assignment is set. See Feedback Action Plan.
  • Engage students by encouraging discussion of feedback. For example, build into lectures assignment feedback swop slots where students discuss their feedback from their last assignment and pair up with a student with similar feedback priorities. ‘Feedback partners’ discuss their plan of action and present this to peers.
  • Engage students in reflecting on feedback. Build in to the next assignment a ‘response to past feedback’ section. For example, include a reflective statement, which requires students to outline areas of feedback they received previously and have worked on improving for this assignment. See Assignment Brief.

Discussion

A large amount of guidance has been provided by the Higher Education Academy and the Assessment Standards Knowledge Exchange for staff and students on the ‘how’, the ‘why’, the ‘when’ and ‘who’ of feedback; that is, how it is given, why it is given, when it is given and by whom, and how students can best use it (HEA 2012; Price et al., 2012; Handley et al., 2008). There is less on inducting students to the ‘what’ of the processes of assessment and feedback, that is, inducting students less familiar with UK HE practices to assessment and feedback processes, and in turn, paving the way for all future assignments.  

This reflective audit on practice, inspired by my inability to respond fully to the question put to the panel, has enabled me to uncover and evaluate this function of feedback practices in my context. The audit, which indicated that the feedback practices are generally fit for purpose, follow recommended practice and are ‘intentional, coherent and recurrent’ and not ‘lone experiences’ (Price et al., 2012, p. 144), also resulted in an explicit pedagogic framework to inform feedback practices in my context.

The conclusion the panel came to was that good practice for all students represents good practice for international students. This is also Ryan and Carroll’s conclusion (2005, p. 10) and is echoed in turn by this reflective audit. However, it has also in the process made the value of scaffolding feedback practices and integrating these in a progressive manner more salient to me. The resulting framework has served to fine-tune my planning and targeting of feedback practices to better suit the socio-linguistic academic literacy competency needs (Paul et al., 2012) of the EAL students in my particular context, permitting more effective planning of pedagogic interventions to target awareness-raising, particularly with regard to linguistic needs and cultural differences in feedback practices.

The notion that inducting to the assessment and feedback processes is an essential requirement for all, particularly novice, HE students is widely accepted (see Scudamore, 2013; Handley et al., 2008, p. 33). The first experience a student has of assessment and feedback creates a baseline expectation for all future assessments. If effectively initiated into the assessment and feedback processes, students will then have an opportunity to become better equipped to actively engage with the assessment and feedback process and make full advantage of it as opposed to being a passive consumer and this I feel, constitutes recommended practice for all HE students. As practitioners our task is to pave the way for effective engagement with assessment feedback as students’ academic careers progress and to do so by raising awareness to, familiarising, training and engaging students in feedback practices in a manner that best suits their specific needs.

Conclusion

The reflective process following a SoTL approach has in itself been personally valuable. Although the feedback practices were in place before I undertook the audit and appeared to be functioning effectively, they were on the whole intuitive and tacit. In making the practices explicit, it enabled them to be grouped into phases, which, in turn made the associated cognitive processes salient. This process has ultimately provided terms of reference within a perceptual framework for personal and collaborative professional pedagogic development purposes aiming to pave the way for students to engage more effectively with assessment feedback. It may add to the conceptual frameworks already available to practitioners (e.g. Handley et al., 2007; Price et al., 2012) to inform in some measure their assessment literacy and feedback practices.

Resources

Resources from the Assessment Standards Knowledge Exchange (ASKe)

1. Oxford Brookes Assessment Compact

2. Leaflets for students

3. Multimedia Resources

Resources from Oxford Brookes University YouTube Chanel

4. How to interpret Turnitin reports

5. Face to Face with Freya

Resources from the Academic Writing for Business Module at Oxford Brookes University at Google Drive folder address

6. Peer review checklist

7. Sources of feedback table

8. First draft feedback good practice guidelines

9. Feedback Action Plan

10. Assignment Brief

Resources from the Higher Education Academy

11. Gilbert, F. (2009) Guidance on managing student expectations when giving feedback. Case Study for the Higher Education Academy’s Teaching International Students Project at

Author Biographies

Fiona Gilbert is a Senior Lecturer and Oxford Brookes Teaching Fellow responsible for the Oxford Brookes Undergraduate Academic English Service. She has recently been awarded an HEA Senior Fellowship.  She is currently involved in the  Assignment Brief Consultancy which is an HEA funded pedagogic research project.

Garry Maguire is a Senior Lecturer and Oxford Brookes Teaching Fellow responsible for the Oxford Brookes Postgraduate Academic English Service  and is the BALEAP Teacher Education Officer. He is currently involved in several research and development projects: the BALEAP practitioner accreditation scheme, the HEA funded  Assignment Brief Consultancy  pedagogic research project and a parallel staff CPD in assessment literacy resource Brookes Fellowship project.

References

Carroll, J. and Ryan, J. (2005). Teaching International Students: Improving learning for all.  (Eds.) London: Routledge.

Gibbs, G., (2008).  Designing Teaching Award Schemes. York: Higher Education Academy

Handley, K., Szwelnik, A., Ujma, D., Lawrence, L., Millar, J. and Price, M. (2007). When less is more: Students’ experiences of assessment feedback. Paper presented at Higher Education Academy, July 2007. Retrieved August 2012 from: www.heacademy.ac.uk/resources/detail/when_less_is_more

Handley, K., Price, M. and Millar, J. (2008). Engaging students with assessment feedback. Oxford Brookes University: Business school. Final report for FDTL5 Project 144/03.

Higher Education Academy (2012). A Marked Improvement. Transforming assessment in higher education. Accessed December 2012 from: www.heacademy.ac.uk/resources/detail/assessment/a-marked-improvement

Higher Education Academy (n.d.). Assessment and Feedback resources. Retrieved March 2013 from: http://www.heacademy.ac.uk/assessment

McLean, P., and Ransom, L. (2005). Building intercultural competencies: Implications for academic skills development. In Carroll, J, & Ryan, J. (eds) Teaching international students: Improving learning for all. London: Routledge. pp. 45–62

O’Brien, M. (2008). Navigating the SoTL landscape: a compass, map and some tools for getting started, International Journal for the Scholarship of Teaching and Learning, 2 (2). Retrieved March 2013 from: www.georgiasouthern.edu/ijsotl

Paul, A., Gilbert, K. and Remedios, L. (2012). Socio-cultural considerations in feedback. In D. Boud & E. Molly (eds) Feedback in Higher and Professional Education. Abingdon: Routledge. pp. 72–89.

Price, P., Rust, C., O’Donovan, B., Handley, K. and Bryant, R. (2012). Assessment Literacy: The foundation for improving student learning. Oxford: The Oxford Centre for Staff and Learning Development. 

Prosser, M. (2008). The scholarship of teaching and learning: what is it? A personal view, International Journal for the Scholarship of Teaching and Learning, 2 (2).

Ryan, J. (2000). A Guide to Teaching International Students. Final report for FDTL5 project 144/03. Oxford: The Oxford Centre for Staff and Learning Development.

Ryan, J. and Carroll, J. (2005). Canaries in the coalmine: International students in western universities. In J. Carroll & J. Ryan (eds) Teaching international students: Improving learning for all. London: Routledge. pp. 3–10.

Scudamore, R. (2013). Engaging home and international students: A guide for new lecturers. Higher Education Academy. Retrieved March 2013 from: www.heacademy.ac.uk/news/detail/2013/engaging_home_international_students_new_lecturers

Trigwell, K., Martin, E., Benjamin, J. and Prosser, M. (2000). Scholarship of teaching: a model. Higher Education Research and Development, 19 (2), pp. 155–68.

Contact Details

Fiona Gilbert, Directorate of Corporate Affairs, Headington Campus, Gipsy Lane, Oxford OX33 1HX

Phone: +44 (0)1865 484358. Email: .(JavaScript must be enabled to view this email address)// ‘;l[1]=’a’;l[2]=’/’;l[3]=”;l[28]=’\”‘;l[29]=’ 107′;l[30]=’ 117′;l[31]=’ 46′;l[32]=’ 99′;l[33]=’ 97′;l[34]=’ 46′;l[35]=’ 115′;l[36]=’ 101′;l[37]=’ 107′;l[38]=’ 111′;l[39]=’ 111′;l[40]=’ 114′;l[41]=’ 98′;l[42]=’ 64′;l[43]=’ 116′;l[44]=’ 114′;l[45]=’ 101′;l[46]=’ 98′;l[47]=’ 108′;l[48]=’ 105′;l[49]=’ 103′;l[50]=’ 46′;l[51]=’ 102′;l[52]=’:’;l[53]=’o’;l[54]=’t’;l[55]=’l’;l[56]=’i’;l[57]=’a’;l[58]=’m’;l[59]=’\”‘;l[60]=’=’;l[61]=’f’;l[62]=’e’;l[63]=’r’;l[64]=’h’;l[65]=’a ‘;l[66]=’= 0; i=i-1){
if (l[i].substring(0, 1) == ‘ ‘) output += “&#”+unescape(l[i].substring(1))+”;”;
else output += unescape(l[i]);
}
document.getElementById(‘eeEncEmail_sQYHcGkKpq’).innerHTML = output;
// ]]>

Garry Maguire, Directorate of Corporate Affairs, Headington Campus, Gipsy Lane, Oxford OX33 1HX

Phone: +44 (0)1865 484358. Email: .(JavaScript must be enabled to view this email address)// ‘;l[1]=’a’;l[2]=’/’;l[3]=”;l[27]=’\”‘;l[28]=’ 107′;l[29]=’ 117′;l[30]=’ 46′;l[31]=’ 99′;l[32]=’ 97′;l[33]=’ 46′;l[34]=’ 115′;l[35]=’ 101′;l[36]=’ 107′;l[37]=’ 111′;l[38]=’ 111′;l[39]=’ 114′;l[40]=’ 98′;l[41]=’ 64′;l[42]=’ 101′;l[43]=’ 114′;l[44]=’ 105′;l[45]=’ 117′;l[46]=’ 103′;l[47]=’ 97′;l[48]=’ 109′;l[49]=’ 103′;l[50]=’:’;l[51]=’o’;l[52]=’t’;l[53]=’l’;l[54]=’i’;l[55]=’a’;l[56]=’m’;l[57]=’\”‘;l[58]=’=’;l[59]=’f’;l[60]=’e’;l[61]=’r’;l[62]=’h’;l[63]=’a ‘;l[64]=’= 0; i=i-1){
if (l[i].substring(0, 1) == ‘ ‘) output += “&#”+unescape(l[i].substring(1))+”;”;
else output += unescape(l[i]);
}
document.getElementById(‘eeEncEmail_JIRYRCe2Iy’).innerHTML = output;
// ]]>

Garry Maguire

Garry Maguire is a Senior Lecturer and Oxford Brookes Teaching Fellow responsible for the Oxford Brookes Postgraduate Academic English Service and is the BALEAP Teacher Education Officer. He is currently involved in several research and development projects: the BALEAP practitioner accreditation scheme, the HEA funded Assignment Brief Consultancy pedagogic research project and a parallel staff CPD in assessment literacy resource Brookes Fellowship project

Fiona Gilbert

Fiona Gilbert is responsible for the academic English provision for undergraduate students with English as a second language in the International Centre at Oxford Brookes. In the field of teaching English language, Fiona is a fully qualified language trainer, holds an MA in applied linguistics and is an accredited Cambridge ESOL teacher trainer. She became an Oxford Brookes teaching fellow in 2008. Her pedagogic research interests include academic literacy, discipline specific academic English provision, teacher education, elearning and assessment. Her main current focus being written assignment tasks and written assignment briefs.

Creative Commons License
This paper is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Double Blind Review
This paper has been subject to a double blind peer review by at least two reviewers. For more information about our double blind review process please visit: http://bejlt.brookes.ac.uk/about/double-blind-review/

How to cite this paper.
Posted in Research Paper

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Subscribe to BeJLT

Get email alerts when there is a new issue.
* = required field

Send this page to Kindle

your kindle user name:
(you@kindle.com, without @kindle.com)
Approved E-mail:
(Approved E-mail that kindle will accept)
Kindle base email kindle.com | free.kindle.com
(Use kindle.com to download on wispernet or wifi, use free.kindle.com for wifi only.)
using kindle.com may incur charges)