Engaging Feedback?

Introduction

How can we promote student engagement with feedback? These research notes discuss the trial of three feedback methods which aimed to disseminate good feedback practice and promote student engagement with that feedback. The trials came towards the end of a four year Higher Education Academy funded research project entitled ‘Engaging students with assessment feedback’ (ESWAF), which, as the name suggests, investigated student engagement with assessment feedback in eight pre- and post- 1992 English universities. The project was iterative in design, involving the collection of data through case studies, quantitative questionnaires and semi structured interviews, with the data being investigated further through the implementation of small scale practice initiatives. The choice and organisation of the methods trialled were structured by findings developed as a result of this research process. These findings will be discussed briefly before describing the methods trialled and some of the outcomes.

Research background: a problem of engagement

An important premise of the ESWAF project was that assessment feedback is integral to learning. This understanding reflects the growing recognition of the value and effectiveness of formative assessment in improving learning, originally highlighted in the work of Black and Wiliam (1998) and developed by them and by others including Hayward, Priestley and Young (2004), Rust, O’Donovan and Price (2005) and James and Pedder (2006). Feedback is a crucial aspect of the formative assessment process, providing students with information about their learning and giving them an indication of how they can improve, or, as Sadler (1998) might put it, helping students to close the gap between a learner’s actual and required performance. This understanding of the role of feedback and assessment in learning draws upon a socio-constructivist analysis which treats knowledge as co-constructed and learning as a collaborative activity (James, 2006). Within this understanding engagement is key: students are not seen as passive recipients of feedback information, rather, they are seen to act upon or engage with the information given (Black & Wiliam, 1998; Nicol & Macfarlane–Dick, 2006), responding to a variety of influences including motivation (Higgins, Hartley & Skelton, 2002).

There is, however, a body of research that suggests students do not engage with their feedback (Gibbs & Simpson, 2004). As a result, the ESWAF project aimed to develop a better understanding of what is getting in the way of students responding appropriately to their feedback and sought to identify strategies for facilitating engagement among learners in Higher Education.

Drawing on educational and organisational literature on engagement (Handley, Price and Millar, n.d.), the ESWAF project took the position that engagement with feedback involves two levels: ‘readiness to engage’ and ‘active engagement’. Readiness to engage includes a willingness and ability to pay attention to, value, and act upon feedback. It also includes as positive affect, such as a student expressing an interest in and enthusiasm for feedback. Active engagement involves the behavioural component of engagement, ranging from an individual reflection on feedback to interaction with peers and tutors aimed at making sense of the feedback given.

Findings from the ESWAF project suggested that barriers to such engagement included:

  • Uncertainty among students about the purpose and value of feedback, which could have an adverse effect on readiness to engage and the nature of engagement itself. For instance, if feedback was seen as simply justifying the grade, then feedback that did not appear to relate directly to the grade might be ignored, creating a form of ‘grade fixation’ (Prowse, Duncan, Hughes & Burke, 2007).
  • A feeling that feedback was not relevant to future work. Some students interviewed suggested that they would only re-read feedback if they were doing a similar piece of work in the future or were likely to be marked by the same tutor.
  • The calibre of staff/student relationships. Students (and staff) argued that good relationships support feedback engagement and bad relationships can act as barriers. This was particularly relevant in connection with discussing feedback. The ESWAF research suggested that many students wanted to reflect upon their feedback through discussions with staff (a finding which confirmed the analysis of Nicol and Macfarlane- Dick (2006) among others), but were likely to do so only where they felt comfortable and that staff were approachable.

Addressing the problem: identifying feedback methods

Building upon these findings, the ESWAF project suggested strategies for improving readiness for and active engagement with feedback at the modular, programme and institutional level (ESWAF Project Report, 2007). At the modular level it was suggested that it would be helpful to integrate formative feedback practices into the module timetable, creating space for activities that clarify the purpose of feedback within the module and that give students the opportunity to receive and then act upon relevant feedback.

Such an integrated approach was common to the three feedback methods discussed in these research notes, as illustrated by the generic feedback method (see Figure 3). As the diagram suggests, the feedback process takes place over the course of the teaching module, with activities scheduled to take place both in and out of class time. A readiness to engage is encouraged by devoting class time to introducing the feedback method before students write the draft assignment. Active engagement is encouraged by giving formative feedback at an early stage in the module – allowing time to apply it – and by asking students to articulate how they applied feedback in reflective commentaries submitted with their completed assignments.

The ESWAF project recommendations stressed the ‘relational dimension’ of the feedback process, acknowledging that the feedback process operates in a social context and is influenced by socio-cultural norms of behaviour and the expectations and experiences of the participants. The recommendations also took into account the complexity of supporting interactions that facilitate engagement with feedback in a higher education context in which cohorts are large and in which students may encounter a series of different tutors and associate staff. The literature on teaching and learning does provide indicators for facilitating good classroom interaction, such as the suggestion that active engagement and dialogue can be encouraged by providing a supportive environment or ‘safe’ space in which students feel able to discuss their work without feeling uncomfortable or misunderstood (see for example Cornelius–White’s meta-analysis (2007), which draws on the work of Carl Rogers).

The feedback methods trialled incorporated this approach into their planning. The resubmission method (see Figure 2) allocated class time for the discussion of generic feedback and for a series of individual discussions with tutors. The intention in requiring students to discuss feedback with staff was to give all students, particularly those who might feel uncomfortable in so doing, ‘permission’ to approach staff.

The peer review method (see Figure 1) also deliberately created opportunities for students to discuss their feedback in class, this time with other students. The ESWAF project research suggested that, although peer review can be difficult for students, they can experience a peer review discussion as a safe space to experiment with ideas and to develop their work in response to peer feedback. In order to ensure that peer review was a positive experience, it was suggested that staff should allocate students to discussion groups (encouraging constructive debate), that sanctions should be considered where students failed to prepare for peer review and that the peer review discussion should follow a set proforma (to avoid negative or irrelevant comments).

Such a concern with the affective aspects of learning is currently under attack. Ecclestone and Hayes (2009), for instance, argue that education has taken a ‘therapeutic turn’, emphasising ‘the emotional’ over ‘the intellectual’ (p. 380) at the expense of the subject content of learning. Whatever the merits of promoting emotional development as an outcome of education, that was not the object of the feedback methods outlined above. Recognising the affective element in feedback interactions was not to downgrade ‘the intellectual’. Rather, the intention was to develop feedback methods that harness affective responses in order to encourage feedback engagement with positive consequences for intellectual development.

Trialling the feedback methods

The three feedback methods discussed in these research notes were trialled in different Schools at one post- 1992 University. As suggested above, the trials aimed to create a debate about good feedback practice among the staff involved, as well as to conduct an evaluation of the effectiveness of the different methods in supporting feedback engagement. In order to encourage staff participation, the evaluation process was limited and involved:

  1. a statement of staff perceptions of the feedback method;
  2. a comparison of module results with results from two previous years (if available);
  3. an indication of student participation in the trial (through attendance lists) and
  4. a short analysis of student responses to the feedback method obtained through survey or interview.

Peer review

Engaging Feedback 1

The peer review method was trialled in an MSc module run in the School of Health and Social Care with a cohort of 24 students. The aim of the trial was to promote active engagement with formative feedback by enabling students to review each other’s work constructively. Students were introduced to the peer review process in class discussions in weeks one and three (Figure 1, step 1). In addition, in week three they were given a marking exercise in which they applied grading criteria to essays from previous years. The intention was to complement the later peer review session by helping students understand the assessment process, giving them an insight into how their work would later be marked and the standards for which they should be aiming.

In week seven (Figure 1, step 3) students were asked to bring a draft or outline of their essay to the peer review session. They were divided into small groups and given a proforma that asked the group to consider how each draft addressed the assessment criteria and how each draft should be developed further.

It was suggested that students complete a proforma for each of their classmates and return it to them. Students could then refer their proforma when completing their assignments. Students were asked to submit the proforma they had received with their completed assignment, but only four students did so. The work was subsequently marked by the tutor (Figure 1, step 5) and feedback was provided in writing (rather than being discussed in class as in Figure 1, step 6).

Following the return of the coursework, 13 students attended an evaluation session during which they discussed the feedback process and completed an evaluation form.

Evaluation

The students who attended the evaluation session were overwhelmingly positive about the method. It was felt to improve communication with peers (n.11), to provide meaningful feedback (n.12) and to improve criticality (n.12). As one student said, ‘I found feedback from the other students very helpful, and I took on board their comments’.

This perspective coincided with the staff perception that students demonstrated an active engagement with their feedback in the class discussion sessions. It was noted that there was constructive debate within the student groups, and that this experience enhanced student ability to reflect critically upon their work. More negatively, it is not clear whether all students took on board the comments of their peers; only four students submitted proformas with their final piece of work, and student results did not improve in comparison with previous cohorts.

Resubmission method

Engaging Feedback 2

The resubmission method was trialled in a second-year Modern Languages module with a cohort of eight students.

Students had the opportunity to reflect on feedback and to resubmit work for additional marks (an approach agreed with the external examiner). The trial explored the impact that this approach had on understanding of the task and also on student motivation and readiness to engage with feedback.

At the beginning of the module the students were introduced to the method illustrated in Figure 2. They were advised that they would merit an increase in marks of up to five per cent if their re-submitted work demonstrated that they had applied the feedback received. Both generic feedback (written and oral) and individual feedback were given in class. Individual oral feedback sessions were offered in tutorials and (on a voluntary basis) in office hours. The method was evaluated through a combination of staff and student perceptions of the approach, with student responses captured during informal discussions and through the end of module evaluation form.

Evaluation

The tutor who ran the trial felt that students ‘rejoiced in the opportunity to improve their marks’. All but one of the cohort took this opportunity, and their resubmitted work showed evidence of reflection on, and application of, feedback. This led to improvements in performance. Students appeared to be making a connection between learning and assessment, seeing assessment as a further way of learning, ‘not just a threat’. In class students showed a willingness to discuss their work and reveal weaknesses. The tutor felt that the opportunity to resubmit work and discuss feedback individually contributed to an ‘enhanced perception that staff were to be seen as a resource to facilitate learning, and less as judges of performance.’ Despite the fact that the approach was time-consuming and resulted in increased paper-work, the tutor felt that it was worthwhile and recommended it to colleagues.

From the student evaluation forms it was clear that the cohort valued the resubmission method, to the extent that they suggested that it should be used in subsequent Modern Languages modules.

Generic feedback method

Engaging Feedback 3

The final trial explored student response to generic feedback at a formative stage in the first module of a MA in Education. Half of the cohort of 14 students were studying online and half were studying on campus.

At the outset students were informed about the intention to deliver generic rather than individual feedback. A formative assignment (based on individual reflection in response to online group discussion) was submitted in week six (Figure 3, step 1). Generic feedback based on the whole cohort’s coursework was subsequently posted on the shared website (Figure 3, step 2). This generic feedback formed a framework for peer evaluation during an on-campus discussion session (Figure 3, step 3).

The generic feedback prepared by the tutor was detailed and reflected all the issues raised by the individual formative work. The tutor also allocated a ‘ballpark’ grade (which was not disclosed) to each piece of formative work in order to be able monitor any changes in grade between the formative and summative work.

The summative assignment was not simply a re-drafting of the formative work. Rather, for their summative assignment (Figure 3 step 4) students were given three options, only one of which was closely aligned with the formative work. The generic feedback was directly relevant to all three options, in that it addressed core aspects of academic discourse.

Students were invited to evaluate their experience of the generic feedback, both immediately after receiving it and on submission of the summative assignment.

Evaluation

Student responses revealed that students liked the collaborative element of receiving generic feedback, the fact that it supported the sharing of a diverse range of problems, and the possibility of using the generic feedback to improve their own work. One student commented, ‘from others’ problems I got to know how to make my paper better’. Some students who were positive about the process also recorded the desire to have individualised feedback, perhaps because they felt unclear as to the specific ways the feedback related to their own work. Yet, even those students who were negative about the process appear to have benefited from it.

A comparison of the formative ‘ballpark’ grades with the summative grades revealed that 71 per cent (n.10) had improved by a full grading band. In addition, 21 per cent (n. 3) achieved final results in the ‘A’ category. In the two previous cohorts, who did not experience a formative stage, only 17 per cent (n.4) and 9.5 per cent (n. 2) of students achieved an ‘A’ grade.

Generic formative feedback appeared to be equally transferable to all three options for summative assignments, despite the fact that one task was more closely aligned to the formative work. The feedback addressing core aspects of academic discourse was identified as most helpful by the students and proved to be transferable across academic tasks and genres.

Finally, from a staff perspective it was possible to recognise students developing into more independent learners through feedback. This was displayed during a one-hour, on-campus seminar in which students gave peers detailed and constructive feedback derived from the generic checklist.

Conclusions

All three staff involved in the feedback trials discussed here felt that the methods used contributed to an environment in which students were prepared to discuss, reflect upon and apply their feedback. All three anticipated that they would use the method trialled again, despite the additional work that could be involved (particularly in the case of the resubmission method).

From the comments made in all three trials it would seem that good classroom relationships were promoted through adoption of the feedback methods. This is illustrated by the MA students in the generic feedback trial who suggested that the process ‘helped us to feel like a group and work collaboratively’. At the same time, the tutor in the resubmission trial suggested that there had been a normative shift from an ‘us and them’ approach to one where engagement and partnership with staff was seen by students as acceptable rather than sycophantic.

There are indicators that all three methods had a positive impact on students’ ‘readiness to engage’ with feedback. It may be argued that this readiness is driven by extrinsic motivation. This charge is especially compelling against the resubmission method, which could be seen as simply ‘pandering to a form of student grade fixation’ (Prowse et al., 2007). Findings from one trial did raise this concern (as mentioned in the introduction, the methods were trialled at several universities), with better-performing students less likely to resubmit. The students involved in the resubmission trial discussed in these research notes, however, expressed an intrinsic motivation in their readiness to engage. One student commented, ‘it is so good to hand in a piece of work and feel that it is good, and that you have learnt from it. It makes you feel proud’. This response suggests that while students may have been ‘hooked in’ by the chance to increase their marks, some at least also valued the chance to learn, as might be predicted by the notion of students as ‘conscientious consumers’ put forward by Higgins et al., 2002; see also Prowse et al., 2007). That students were conscientious and displaying an intrinsic motivation was also suggested by the fact that even those students in the generic feedback trial who disliked the method were still prepared to engage with and to apply the feedback.

Active engagement with feedback was evident in the constructive classroom discussions noted in all three trials. Active engagement in terms of applying feedback was particularly evident in the resubmission and generic methods where tutors were able to chart the development of student work from the formative to the summative assignments.

Readiness to engage and active engagement with feedback would have strong potential to develop in a learning environment where students experienced the formative feedback methods discussed here on a regular basis, thus becoming socialised into positive rather than defective feedback practices (Sadler, 1998). At present, however, it is uncertain whether positive levels of engagement would be maintained. Are students actually learning the value of feedback and finding ways to use it, or are they simply taking the opportunity of a novel feedback method to improve their grade within a particular module? Further research involving sustained use of formative feedback methods, such as those described here, and a more rigorous evaluation of those methods is needed before firm conclusions can be reached.

Biographies

Dr Jill Millar is an ASKe Research Associate based at Oxford Brookes Business School. Her background is in socio-legal research with a PHD from Warwick University. Her current research focus is on assessment and feedback with a particular interest in feedback as a social practice.

Contact: ASKe Research Associate Oxford Brookes University, Wheatley Campus, Wheatley, OX33 1HX, ph: 01865 485418, Email: .(JavaScript must be enabled to view this email address)
//’;l[1]=’a’;l[2]=’/’;l[3]=”;l[26]='”‘;l[27]=’ 107′;l[28]=’ 117′;l[29]=’ 46′;l[30]=’ 99′;l[31]=’ 97′;l[32]=’ 46′;l[33]=’ 115′;l[34]=’ 101′;l[35]=’ 107′;l[36]=’ 111′;l[37]=’ 111′;l[38]=’ 114′;l[39]=’ 98′;l[40]=’ 64′;l[41]=’ 114′;l[42]=’ 97′;l[43]=’ 108′;l[44]=’ 108′;l[45]=’ 105′;l[46]=’ 109′;l[47]=’ 106′;l[48]=’:’;l[49]=’o’;l[50]=’t’;l[51]=’l’;l[52]=’i’;l[53]=’a’;l[54]=’m’;l[55]='”‘;l[56]=’=’;l[57]=’f’;l[58]=’e’;l[59]=’r’;l[60]=’h’;l[61]=’a ‘;l[62]=’= 0; i=i-1){
if (l[i].substring(0, 1) == ‘ ‘) output += “&#”+unescape(l[i].substring(1))+”;”;
else output += unescape(l[i]);
}
document.getElementById(‘eeEncEmail_x1JDrqZ4eq’).innerHTML = output;
//]]>

Sally Davis is a programme lead for the Global Health and Wellbeing cluster in CPD Directorate in the School of Health and Social Care. Sally leads a multi-professional MSc on Rehabilitation. In the School, Sally takes a lead on inclusion and diversity and is also on the Learning, Teaching and Assessment Committee. 

Hilary Rollin was Field Chair of Spanish Studies and the Minor in Spanish at Oxford Brookes University. As a Brookes Fellow in Teaching and Learning, she did her fellowship project on interculturality. Other areas on which she has published include: gender equality in the workplace, aspects of contemporary Spain, entrepreneurship and pedagogy.  

Dr Jane Spiro is programme leader of the international and TESOL (Teaching English to Speakers of Other Languages) pathways through the MA in Education at Westminster Institute of Education, a Learning and Teaching Development co-leader and a University Teaching Fellow. She completed a PhD in language education in 2008, and has written two books with Oxford University Press on creative writing skills. She was formerly Head of Applied Linguistics, teaching and co-ordinating programmes in academic literacies for international students.

We warrant that this paper is our own work, and is not currently under consideration by any other publication, nor has it been published elsewhere. The paper is based on work on the transferability phase of the FDTL 5 Project Engaging Students with Assessment Feedback and relates to a joint paper given at the 2009 Brookes Student Learning Experience Conference

References

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice 5(1), 7-74.

Cornelius-White, J. (2007). Learner-centered teacher student relationships are effective: a meta-analysis. Review of Educational Research 77(1), 113-43.

Ecclestone, K., & Hayes, D. (2009). Changing the subject: the educational implications of developing emotional well-being. Oxford Review of Education 35(3), 371-89.

ESWAF FDTL. (2007). Final Report. Retrieved on 25 September 2009 from: https://mw.brookes.ac.uk/display/eswaf/Home

Gibbs, G., & Simpson, C. (2004). Conditions under which assessment supports students’ learning. Learning and Teaching in Higher Education 1(3), 1-31.

Handley, K. J., Price, M. & Miller, J. (unpublished draft) Student engagement with assessment feedback: Framing a research agenda.

Hayward, L., Priestley, M., & Young, M. (2004). Ruffling the calm of the ocean floor: Merging practice, policy and research in assessment in Scotland. Oxford Review of Education 30(3), 397-415.

Higgins, R., Hartley P., & Skelton, A. (2002). The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies in Higher Education 27(1), 53-64.

James, M. (2006). Assessment, teaching and theories of learning. In J. Gardner (Ed.) Assessment and learning (pp. 47-60). London: Sage.

James, M., & Pedder, D. (2006). Beyond method: assessment and learning practices and values. The Curriculum Journal 17(2), 109-38.

Nicol, D., & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education 31(2), 199-218.

Prowse, S., Duncan, N., Hughes, J., & Burke, D. (2007). ‘…do that and I’ll raise your grade’: Innovative module design and recursive feedback. Teaching in Higher Education 12(4), 437-45.

Rust, C., O’Donovan, B., & Price, M. (2005). A social constructivist assessment process model: How the research literature shows us this could be best practice. Assessment and Evaluation in Higher Education 30(3), 231-40.

Sadler, D. R. (1998). Formative assessment: revisiting the territory. Assessment in Education: Principles, Policy and Practice 5(1), 77-85.

Posted in Research Report

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Subscribe to BeJLT

Get email alerts when there is a new issue.
* = required field

Send this page to Kindle

your kindle user name:
(you@kindle.com, without @kindle.com)
Approved E-mail:
(Approved E-mail that kindle will accept)
Kindle base email kindle.com | free.kindle.com
(Use kindle.com to download on wispernet or wifi, use free.kindle.com for wifi only.)
using kindle.com may incur charges)