This article addresses the challenges of designing the assessment of students’ collaborative writing. It explores the shift in focus from individualistic notions of authorship and ownership to shared practices, values and goals, which are developed through dialogue and discussion. Existing notions of connoisseurship refer to the capacity to evaluate and enhance one’s writing. Drawing on our research (O’Shea and Fawns, 2014), we extend this to include an understanding of academic quality that is co-constructed and interdependent across the members of a group – a concept we term ‘group connoisseurship’. This is challenging and it requires the development of a range of complex knowledge and skills within a supportive environment to reach a standard that is shared between students as well as tutors. In this article, we explore how this concept can inform course design to improve groups’ academic alignment, functional roles and writing practices in collaborative assessment.
Collaborative work can involve powerful learning processes, through which understanding is not only individually and socially constructed, but is also promoted through the generation of shared artefacts (Papert and Harel, 1991). Although each learner’s understanding must, essentially, be individually constructed, in sharing both the artefact and its creation, the likelihood that these constructions align and converge is increased. Further, in learning to work together, members of a group must reflect upon and discuss metacognitive processes of creation and collaboration, and both give and act upon peer feedback around these (McCune and Hounsell, 2005; Sadler, 2010). Opening up working processes to others provides opportunities for vicarious learning through modelling (Mayes et al., 2002) and the development of the skills necessary to appraise and feedback on the work of others (Carless, 2007; Nicol, 2014). Collaborative work is also strongly related to transferable skills, authentic learning and educational outcomes around lifelong learning and employability (Boud et al. 2001; Gibbs, 2006).
In higher education, challenges arising from a tradition and culture of individual assessment often undermine the very processes collaborative work is intended to promote (Boud et al., 2001; Forte and Bruckman, 2007; Paulus, 2005). Most students have learned to prioritise their own individual improvement (often in competition with others) rather than learning to progress as an interdependent group. If, as Gibbs (2006, p. 3) argues, ‘Assessment frames learning, creates learning activity and orients all aspects of learning behaviour,’ then collaborative assessments need to be structured in a way that requires and scaffolds authentic engagement not only with tasks but with others engaged in that task.
‘True’ collaboration – where all individuals make significant, integrated contributions – can be supported by appropriately designed assessments that promote shared goals and learning processes. Even with a seemingly shared goal (the assessed artefact), groups still require the development of forms of social connectedness between members, where there is fair and effective negotiation of the various challenges of the task and a set of tools that support the group’s competence. This is part of developing a level of intrinsic motivation that moves participation away from the ‘ritual engagement’ of seeking external reward (like a grade) or the ‘passive compliance’ of avoiding negative consequences (such as failing) (Zyngier and May, 2004). The goal, as Roschelle and Teasley (1995, p. 70) describe it, is to create forms of activity that are ‘the result of a continued attempt to construct and maintain a shared concept of a problem’ and that promotes positive interdependence (Johnson et al., 2007).
From individual to group connoisseurship
Collaborative assessment also requires a change in how students understand quality. Advanced individual understanding of quality has been termed ‘connoisseurship’, encompassing notions of evaluative acumen, judgment and self-regulation. Eisner, for instance, conceptualises connoisseurship as an act of appreciation, where the connoisseur has ‘an awareness and an understanding of what one has experienced. Such an awareness provides the basis for judgment’ (Eisner, 1976, p. 140). However, for Eisner, connoisseurship is only one part of the conceptual combination needed for thinking about educational evaluation. For him, criticism – entailing the communication of that connoisseurship and the explication of the qualities of the matter under discussion – is also essential, as part of a kind of connoisseurship in action.
For us, connoisseurship as a term encompasses both those aspects of criticism and evaluation, and is then extended by Sadler’s concept of evaluative judgement (1989, p. 121). This is where ‘the student comes to hold a concept of quality roughly similar to that held by the teacher’ and develops a range of strategies they can draw on to evaluate and regulate the quality of their work. This evaluative judgment is developed through experience of appraising a range of the work of others (Carless, 2007; Hounsell et al., 2007).
In collaborative writing, we suggest that the goal is actually ‘group connoisseurship’, where ‘individual understandings of quality align with a co-constructed group understanding of what constitutes “good” work for a shared task’ (O’Shea and Fawns, 2014, p. 267). In this sense, it is not just the individuals, but also the group that understands the quality of the work and how it might be improved upon in practice. The development of group connoisseurship can be problematic because it requires unlearning ways of working individually (Naismith et al., 2011) and destabilises individual authorship (O’Shea and Fawns, 2014). In addition, to function effectively as a group, individual notions of self-regulation must be left behind (Karasavvidis, 2010) in favour of attitudes and practices that move the group forward as a whole. The focus on a group’s ability to evaluate the quality of their work and their working practices is not necessarily something that needs to be assessed directly. Rather, this group connoisseurship provides a lens for thinking about assessment design with the aim of improving the collaborative experience, the learning of group members, and the written product.
A case study of ‘online assessment’
For this research we explored our students’ collaborative practices in a class-wide, wiki-based assignment which formed part of the twelve-week ‘Online Assessment’ course, a component of the online, distance learning MSc in Digital Education at the University of Edinburgh. All nineteen students on the course participated in the research. Data was gathered and analysed from three qualitative sources: forum discussion threads, wiki comments and email interviews. Two rounds of email interviews were held with students individually at the midpoint of the activity and after the assignment was assessed. These were open ended discussions of the process and experience intended to ‘to provide an environment conducive to the production of the range and complexity of meanings that might occur to all interview participants’ (Holstein and Gubrium 2004, 152). This approach enabled ongoing reflective discussions between researcher-tutors and individual participants to explore the subject matter (Berger and Paul 2011) which supported our thematic analysis of the data (Boyatzis 1998) alongside our interpretation of concepts from the literature and our own tutor experiences.
Students self-selected into groups of two to four members that collaboratively authored responses to provocative, highly-interpretable and interrelated topic statements about online assessment (such as ‘In the age of online assessment, plagiarism is an outdated concept’). This co-authoring took place in a shared wiki space where students could work together on web pages that included text, images, video and hyperlinks both between each groups’ writing and out to the wider web. Each group was encouraged to make links to other groups’ work, both in their writing and in acting as ‘critical friends’ by engaging in ongoing dialogue with other groups around collaborative processes and the arguments being constructed. This expectation of inter-group coherence and connection was underlined by a class-wide – rather than a group or individual – mark, worth 25% of the overall grade.
Engagement in this activity was highly-scaffolded, for, as Boud et al. (2001) note, there needs to be sufficient opportunity for group processes, planning and responsibility to develop before effective group assessment can occur. There were several weeks of non-assessed, cumulative activities around both the technological and social practices for wiki writing (e.g. playful creations of text, image and comments, moving to interdependent group synthesis of ideas). The course was designed as an unfolding dialogue with students so that just-in-time tutor support was available throughout via forums, wiki comments, emails and Skype. In addition, tutors offered feedforward on the assignment itself by discussing the progress of each group two weeks prior to submission. Inevitably, the assignment was still challenging (as we will see below) but we saw this as an opportunity for our students to experience and reflect on the challenges (and benefits) of online collaborative assessments.
Pieterse and Thompson (2010, p. 357) posit the concept of academic alignment, in which ‘members are homogenous in terms of academic abilities, skills and goals’, as important to successful collaboration. For us, this alignment must also take into account the expectations individuals hold regarding the purpose of group work, the nature of group working and their relationship to the group. Students engage with collaborative projects for multiple, and sometimes conflicting, purposes (Davies, 2009). Among our students, for instance, reasons for engagement included: an opportunity to gain experience in online collaboration; to learn about the topic; to meet the course requirements; to achieve a high grade; and to experience the student perspective of a collaborative activity. Thus, for some, group work was considered beneficial for their learning, while others saw it as a challenge to individual performance and grades.
While Pieterse and Thompson’s concept of academic alignment seems to be reliant on homogenous groups in terms of academic skills and abilities, we are not convinced that academic alignment is essential at the beginning of a collaborative exercise. Part of the value of group work is in the discrepancies between members (in terms of ability, background and perspective) that create opportunities for peer learning and support of the development of both skills and understandings (Boud et al., 2001; Nicol, 2014; Johnson et al., 2007). By taking up these opportunities, groups can achieve alignment through the process of collaboration. Across groups, in both preparatory and assessed activities, we saw many examples of sharing technical support, resources and perspectives on literature and concepts. In particular, students referred to the help they received in developing their critical thinking skills, with one student noting:
I was amazed at how much we had done early on in the first week and how much my team’s insights are helping me think differently about certain points (Student 1).
Another student explained:
I found that some of [another student’s] questions/remarks on things I had written, helped me to polish my argument and think more on the related issues. This wouldn’t have happened, if I was working on my own (Student 2).
In our view, alignment did not require groups to be homogenous in terms of skills, since these could be distributed effectively across group members.
The extent to which goals and expectations were aligned seemed to influence the success of the project in terms of equity, satisfaction and the generation of collective understanding. This varied across our student groups. For some, ‘Group cohesion – this was great from day 1, stayed great and ended great’ (Student 3). For others, cohesion only developed once synchronous discussion had taken place:
It wasn’t until I had spoken to everybody that I felt part of the group. It may have been synchronicity or the fact we had the same objectives to maintain the same goals and commitment (Student 4).
Here, it seems that even when processes were aligned, they needed to be made explicit through synchronous discussion before they were recognised by the group. For members to be satisfied with progress, the discrepancies within a group (which can provide a powerful stimulus for learning) needed to feel aligned.
This was not an easy process: alignment around goals and expectations was never achieved for some groups, with one or more members expressing tensions between their expectations and the perceived expectations of others. For instance, one student experienced a tension between waiting for group consensus and feeling left behind:
What also flummoxed me was not making sure I had a clear idea of what was expected of me as an individual and as a group member. This resulted in me waiting too long and not prioritising the activities I should have done in tandem with other members, and therefore left me way behind, consequently being unable to contribute much of interest (Student 5).
Thus, designing collaborative assessment requires not only creating opportunities for alignment issues to be articulated, but also multiple points at which dialogue can move that articulation towards negotiated consensus.
Roles and functions
Educators often assume that students are natural collaborative learners. After all:
Humans are small group beings. We always have been and we always will be. The ubiquitousness of groups and the inevitability of being in them makes groups one of the most important factors in our lives (Johnson and Johnson, 2003, p. 579).
However, as Johnson and Johnson (2003) argue, there is an artificiality to educational groups. Different roles, for example, are not dictated by job title but are allocated by a tutor or negotiated within the group. Indeed, even the use of ‘roles’ as a concept in educational group work can be problematic as it suggests not only the tasks that a student is undertaking (e.g. writing, editing) but also ways that a student’s identity might be understood in group interactions (e.g. leader, organiser) (Hargie, 2010). In group assessment, tensions around roles are often unresolved as there is unlikely to be an established collaborative culture and, for short-lived projects, this culture may never emerge. In online collaboration, in particular, roles may need to be negotiated asynchronously – a slow process in a time-pressured situation.
A less rigid collaborative culture can have benefits for group work. Identity-related roles such as ‘leader’ can be explicitly negotiated, resisted or emerge through the collaborative process. In our groups, we saw ‘leadership’ achieved through a series of functions – one person leading in relation to one aspect of a project or during one period of time and being led at other times. Likewise, all group members did some writing of content and most, if not all, edited content written by themselves or others. They did not seem to take on roles of ‘writer’ or ‘editor’ but instead performed temporary functions of ‘writing’ or ‘editing’. These were akin to Mudrack and Farrell’s task roles (1995), which are aimed at helping a group achieve its goals. Similarly, sharing responsibilities for group organisation, processes and social cohesion were in keeping with relationship building and maintenance roles (ibid).
Sharing functions can encourage group productivity (Hargie, 2010) in two key ways. First, all members need to participate to some extent in a range of relationship-building and maintenance roles to develop and maintain group cohesion and responsibility (Strijbos et al., 2004). One student noted ‘There is a whole layer of project management on top of the assignment’ and another explained ‘the ‘sideline’ communications are important energising, reassuring glue for a working group’. Secondly, sharing functions can help each student to understand their contribution to the wider group work as integrated rather than individual:
What’s is really taking a lot of time is that I’ve been looking for and reading stuff about the topic that wouldn’t fit in ‘my part’ of the Wiki work, but I feel I need it to get a better picture (Student 6).
Group connoisseurship is most likely to emerge where each individual provides a discernible but not necessarily distinguishable contribution to the group’s approach. This contribution need not be fundamental or particularly skilful, but should be enough to create a mutual sense of ownership of both the product and of the group itself.
This holistic approach, adopted naturally by most of our student groups, did provide challenges. As one student explains about an earlier, non-assessed writing exercise:
The fact that in our group we didn’t divide the work and assign roles had both positive and negative results. Negative: we didn’t manage to finish/polish the critique. Positive: the whole experience was less stressful, mainly because of the freedom we enjoyed in working on the assignment (Student 2).
Another student described this kind of approach as ‘democratic chaos’, that was both demanding and rewarding (Student 8). Transient functional roles opened up opportunities for exploring ideas and ways of working. However, this was situated within the sometimes challenging context and practices of digital environments.
Technologies and writing practices
Collaborative authoring processes do not unfold in a neutral environment. A community is not independent of the tools that mediate its activities, and, indeed, those tools often sustain and transform the community’s practices (Engeström, 1999). In this respect, technologies provoke us to consider:
The practices and activities that surround them, the meanings that people attach to them, and the social relations and structures that these technologies are linked to (Selwyn, 2011, p. 2).
The same technologies can both open up and close down possibilities for interaction, depending on their use and the development of group practices. For example, wikis, in particular, provide sustained access to the changes made by different group members, which can enable a more transparent writing process and support understanding that members can contribute in different ways (Carr et al., 2007; Lundin, 2008; Vassell et al., 2008). Work can be open to tutors or peers at multiple points during the generation of a product, facilitating formal and informal formative feedback that can be received and acted upon during a project. However this very feature can also create barriers to collaborative authoring and reinforce individuality over group consensus.
For instance, in one group where members all contributed to specific pages and sections, these contributions were demarcated with coloured fonts denoting authorial origins:
I’ve also added a few arguments in the product section that relate to stakeholders and purposes and seemed critical to me. Some I’ve copied from the previous version. I’m in green 😉 And of course feel free to edit as you see fit (Student 1).
While this facilitated initial dialogue around perspectives on the work, it also seemed to inhibit the integration of those contributions. Likewise, marking a space as initially written by one author could add a layer of difficulty to creating an integrated product:
In our project some of our pages have initial or sole authors who researched/wrote particular angles. Each author now feels an ‘ownership of/loyalty’ = emotional investment (?) to that page. Some were quite collaborative…Others were very collaborative with everyone adding refs and comments (Student 7).
This example illustrates the wariness of authoring as a group and the use of strategies like resource sharing and commenting as a useful way into negotiating direction and synthesising understanding without undermining the original author’s position. Much editing was done in a way that both permitted and facilitated the reversal of changes. The features of the wiki, such as page history, created a further safety net in this regard.
These practices illustrate how contributions to collaborative authorship are made not simply to specific content, but to a situated work. Making that situated work a space for continuous and developing understandings requires a shift in perspective on the nature of ownership: once a contribution has been made to a wiki, it belongs to the group (Wheeler et al., 2008) and thus criticism and challenges are made about the work rather than an author. This requires trust between group members that they each have something useful to bring to the work and that such contributions are not a ‘breach of trust but an act of responsibility and mutuality’ (Hemmi et al., 2009, p. 28).
For tutors to facilitate this process effectively, they need to understand how challenging this can be. We suggest that collaborative writing for assessment is a threshold practice (Meyer and Land, 2003), one that most educators have already crossed and so may find it hard to understand how troublesome it can be for students (Perkins, 2006). As such, group connoisseurship is not likely to be achieved by a group in isolation but mediated through dialogue with the tutor, the course design and environments.
Collaborative assessments are both highly challenging and potentially fruitful learning opportunities. However, learning to collaborate meaningfully requires not only learning new approaches, it requires letting go of old, individualistic ways of thinking and working. As one of our students explained:
That’s a difficult group of skills to acquire; it involves leaving behind our personal intentions, prioritizing group interaction, seeing the assignment evolve to something perhaps far from what we personally had in mind to do, a general challenge on many levels (Student 1).
Designing both the formal and informal aspects of assessment to enable this shift from individual to shared purposes and practice is essential in helping students understand and make the most of educational group work. The design must be cumulative in nature, allowing for scaffolded and iterative opportunities to develop understandings not only of content, but critical thinking skills and group processes. Multiple opportunities for feedforward and feedback can help surface the goals, processes and support needs of the group.
Our purpose here is not to argue for a change in the process of assessing collaborative writing, although there may be a place for such arguments. Instead, we want to encourage a focus on supporting effective collaboration through the development of a collective understanding of quality. Making this group connoisseurship an explicit part of the assessment design can facilitate a more supported and effective collaborative experience.
Berger, R., and Paul, M.S(2011) ‘Using e-mail for family research.’ Journal of Technology in Human Services 29: 197–211.
Boud, D., Cohen, R., and Sampson, J. (2001) ‘Peer learning and assessment.’ In Boud, D., Cohen, R.,and Sampson, J. (Eds). Peer Learning in Higher Education: Learning from and with Each Other. London: Routledge: 67-84.
Boyatzis, R. E. (1998) Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks: Sage.
Carr, T., Morrison, A., Cox, G., and Deacon, A. (2007) ‘Weathering wikis: net-based learning meets political science in a South African university.’ Computers and Composition 24 (3): 266-284.
Carless, D. (2007) ‘Learning‐oriented assessment: conceptual bases and practical implications.’ Innovations in Education and Teaching International 44 (1): 57-66.
Davies, M. (2009) ‘Group work as a form of assessment: common problems and recommended solutions’. Higher Education 58: 563-584
Eisner, E. W. (1976) ‘Educational connoisseurship and criticism: their form and functions in educational evaluation.’ Journal of Aesthetic Education 10 (3/4): 135-150.
Engeström, Y. (1999) ‘Innovative learning in work teams: analysing cycles of knowledge creation in practice.’ In Perspectives on Activity Theory, Y. Engeström et al. (Eds). Cambridge: Cambridge University Press: 377-406.
Forte, A., and Bruckman, A. (2007) ‘Constructing text: wiki as a toolkit for (collaborative?) Learning.’ Proceedings of the 2007 International Symposium on Wikis, Montreal, October 21-25, 2007. Available from: http://www.andreaforte.net/ForteBruckmanConstructingText.pdf
Gibbs, G. (2006) ‘Why assessment is changing.’ In Bryan, C., and Clegg, K. (Eds). Innovative assessment in higher education. London: Routledge: 11-22.
Hargie, O. (2010) Skilled Interpersonal Communication Research, Theory and Practice. Hove: Routledge.
Hemmi, A., Bayne, S., and Land, R. (2009) ‘The appropriation and repurposing of social technologies in higher education.’ Journal of Computer Assisted Learning 25 (1): 19-30.
Holstein, J., and J. Gubrium (2004) The active interview. In Qualitative Research: Theory, Method and Practice, D. Silverman (Ed). London: Sage: 140–61.
Hounsell, D., R. Xu, and Tai, C. M. (2007) Balancing Assessment of and Assessment for Learning (Scottish Enhancement Themes: Guides to Integrative Assessment, no. 2). Gloucester: QAA. Available from http://www.enhancementthemes.ac.uk/docs/publications/guide-no-2—balancing-assessment-of-and-assessment-for-learning.pdf
Johnson, D. W. and Johnson, F. P. (2003) Joining Together. Group theory and group skills. Boston: Allyn and Bacon.
Johnson, D. W., Johnson, R. T., and Smith, K. (2007) ‘The state of cooperative learning in postsecondary and professional settings.’ Educational Psychology Review 19 (1): 15-29.
Karasavvidis, I. (2010) ‘Wiki use in higher education: exploring barriers to successful implementation.’ Interactive Learning Environments 18 (3): 219–31.
Lundin, R. W. (2008) ‘Teaching with wikis: toward a networked pedagogy.’ Computers and Composition 25 (4): 432-448.
Mayes, T., F. Dineen, J. McKendree, and Lee, J. (2002) ‘Learning from watching others learn. In Networked Learning: Perspectives and Issues, Steeple, C. and Jones, C. (Eds). London: Springer: 213–27.
McCune, V.,and Hounsell, D. (2005) ‘The development of students’ ways of thinking and practising in three final year biology courses.’ Higher Education 49 (3): 255–89.
Meyer, J.,and Land, R. (2003) Threshold Concepts and Troublesome Knowledge: Linkages to ways of thinking and practising within the disciplines. Edinburgh: University of Edinburgh.
Mudrack, P. E.,and Farrell, G. M. (1995) ‘An examination of functional role behaviour and its consequences for individuals in group settings.’ Small Group Research 26 (4): 542-571.
Naismith, L. B., Lee, H., and Pilkington, R. M. (2011) ‘Collaborative learning with a wiki: differences in perceived usefulness in two contexts of use. Journal of Computer Assisted Learning 27: 228–42.
Nicol, D. (2014) ‘Guiding principles for peer review: unlocking learners’ evaluative skills.’ In Advances and innovations in university assessment and feedback. Kreber, C., Anderson, C., Entwistle, N. and McArthur, J. (Eds). Edinburgh: Edinburgh University Press: 197-224.
O’Shea, C. and Fawns, T. (2014) ‘Disruptions and dialogues: supporting collaborative connoisseurship in digital environments.’ In Advances and innovations in university assessment and feedback. Kreber, C., Anderson, C., Entwistle, N. and McArthur, J. (Eds). Edinburgh: Edinburgh University Press: 259-273.
Papert, S. and Harel, I. (1991) ‘Situating constructionism. In Constructionism. Papert, S. and Harel, I. (Eds.) Ablex, Westport: 1-11.
Paulus, T. M. (2005) ‘Collaborative and cooperative approaches to online group work: the impact of task type.’ Distance Education 26 (1): 111-125.
Perkins, D. (2006) ‘Constructivism and troublesome knowledge.’ Overcoming barriers to human understanding: threshold concepts and troublesome knowledge. Meyer, J. H. F. and Land, R. (Eds). London: Routledge: 33-47.
Pieterse, V., and Thompson, L. (2010) ‘Academic alignment to reduce the presence of ‘social loafers’ and ‘diligent isolates’ in student teams.’ Teaching in Higher Education 15 (4): 355-367.
Roschelle, J., and Teasley, S. D. (1995) ‘The construction of shared knowledge in collaborative problem solving.’ In Computer Supported Collaborative Learning. O’Malley, C. (Ed.) Springer Berlin Heidelberg: 69-97.
Sadler, D. R. (1989) ‘Formative assessment and the design of instructional systems.’ Instructional Science 18 (2): 119–44.
Sadler, D. R. (2010) ‘Beyond feedback: developing student capability in complex appraisal.’ Assessment and Evaluation in Higher Education 35 (5): 535–50.
Selwyn, N. (2011) Education and Technology: Key issues and debates. London: Continuum International Publishing.
Strijbos, J. W., Martens, R. L., Jochems, W. M., and Broers, N. J. (2004) ‘The effect of functional roles on group efficiency using multilevel modeling and content analysis to investigate computer-supported collaboration in small groups.’ Small Group Research,-35 (2): 195-229.
Wheeler, S., P. Yeomans, and Wheeler, D. (2008) ‘The good the bad and the wiki: evaluating student generated content for collaborative learning’ British Journal of Educational Technology 39 (1): 987–95.
Vassell, C., Amin, N., and Winch, S. (2008) ‘Evaluating the use of wikis in student group work within Blackboard.’ 9th annual conference of the Subject Centre for Information and Computer Sciences. Liverpool Hope University, Liverpool. Available via: http://www.ics.heacademy.ac.uk/events/9th-annual-conf/Papers/Proceedings/Proceedings%20Full.pdf#page=128.