This study comes from an attempt to respond to the Brookes student feedback surveys on the Extended Writing Project module of the pre-Master’s diploma for international students. In the 2005 module feedback, students stated that if they were to take the module again, they would want to try to improve their writing more between drafts and learn more about using sources. From 2006-07, the study of the experiential use of Turnitin with first drafts before assessment aimed to examine the learning and ‘unlearning’ opportunities for academic literacy and plagiarism education.
This study focuses on an attempt to respond to student feedback at the end of the 2005 run of a pre-postgraduate academic writing module entitled the Extended Writing Project on a pre-Master’s diploma for international students. The key objectives of the module are to improve use of academic writing conventions, develop greater academic literacy skills and apply these skills to a research project leading to an extended research paper in a subject-specific field. This module is considered the most important on the course, as the extended research paper is the longest assignment, a 3,000-word mini-dissertation, which both motivates and challenges students. While completing the module, students are required to learn UK academic conventions such as citations of author and year within the text, and ‘unlearn’ old conventions, such as using other author’s words in preference to one’s own. Students stated in the module feedback that what they would like to do differently next time would be to improve their writing more between drafts and learn more about sources. It was considered that the introduction of Turnitin in the module with tutorial feedback in first drafts might be a useful way of answering these requests, while also providing key learning opportunities.
Challenges for international students
International students in the UK face considerable challenges in relation to the issue of academic literacy. While plagiarism is a growing global concern, many studies (e.g., Barrett and Malcolm, 2006; Hayes and Introna, 2005; Handa and Power, 2005) focus on international students in English-speaking countries where understanding specific academic conventions and avoiding plagiarism are seen as essential skills. There could also be more focus on international students with a non-native command of English, because changes in discourse style in their assignments may be more obvious: as Bull et al (2001:5) found, ‘the most common trigger that arouses academics’ suspicions of plagiarism in assignments is a change of writing style within text’.
A first step to understanding the issue would be to examine why international students plagiarise, but it should be remembered that there is no easy answer to that question; as Hayes and Introna (2005:213) suggest, ‘plagiarist practices are often the outcome of many complex and culturally situated influences’. One particular factor relevant to East Asian students is the importance given to copying from other authors in their own academic culture; as East (2005:3) explains, many international students ‘come from educational cultures where copying is an expected learning practice’. Another problem for international students is assimilation of new academic conventions, which can take time to comprehend; as Handa and Power (2005:77) found, ‘many students end up plagiarising because they do not fully understand the academic requirements of their new academic culture’. Thus the issues of cultural influence, the practice of copying and the difficulty adapting to a new academic culture are likely to influence international students’ academic literacy. The complex and painful process of adaptation to other academic conventions and ‘unlearning’ former ones has been illustrated by Carroll and Appleton (2001) as ‘setting aside previously successful strategies and learning new ones’. Dealing with plagiarism needs to be done in a culturally sensitive way (Carroll, 2005). Therefore, providing support to international students by creating learning and unlearning opportunities with the use of plagiarism detection software, in this case Turnitin, in academic writing forms the rationale for this study.
Turnitin is a text-matching tool first developed by Dr. John Barrie at the University of Berkeley in 1994, now used in 85 countries (iThenticate, 2007), principally as a method of plagiarism detection. As an online resource, it does not require downloading of software: student assignments are simply uploaded to the Turnitin website, which generates an originality report showing up all sections of the assignment that match text on its database of web sources, books and periodicals, and student work. It uses colour coding of a 5-point scale from blue (<20 words) through green (0–24%), yellow (25–49%), orange (50–74%) and red (75–100%) to indicate overall similarity, then provides colour-coded web-based links to each source it identifies to allow for direct comparison. These functions allow for in-depth analysis of the use of sources in student work; features of this analysis include the amount of textual borrowing, the degree of reliance on sources, the accuracy of citation, the effectiveness of paraphrasing and the competence of the student in avoiding plagiarism.
Turnitin was incorporated for the first time in the 2006 run of the academic writing module. It was not intended as an isolated measure but rather formed part of a plagiarism education programme, which included at least 6 hours’ instruction, a focus in online discussions, handbook self-study exercises, library guidelines and institutional warnings. The written task in itself was carefully designed to deter plagiarism in that it was based on an individual research question guided by the tutor to be evaluative rather than descriptive (to avoid information simply being copied and pasted). Thus the programme and task followed the suggested principles for deterring plagiarism from Carroll and Appleton (2001). Meanwhile, it was also borne in mind that students do not seem to absorb instruction about plagiarism until it refers to their own work (Barrett and Malcolm, 2006). Therefore, after students’ first drafts were submitted to Turnitin in week 9 of the module, the originality reports generated were used in individual tutorial feedback with ample opportunities to discuss the students’ own results using the visual evidence on the screen. The one-on-one discussion was entirely open, without any threat of assessment; no penalties were imposed, unlike in the study by Barrett and Malcolm (2006) in which a heavily plagiarised first draft could only achieve a bare pass of 40% in the final results. Following the feedback, the students had a period of 4 weeks (including the Easter break) to work on improving their drafts
. In order to assess the perceived improvements in drafts, the final assignments were also uploaded to Turnitin and the drafts were compared. The data was gathered from the originality reports in the form of the percentages of overall similarity and individual sources, and the highlighting of paraphrasing and evidence or lack of citation. Further quantitative feedback was gathered from both 2006 and 2007 in the form of questionnaires of eight positive and negative statements to answer on a Likert scale and two open questions, and qualitative feedback from focus groups using seven closed questions and three open questions. The study in 2007 developed a more in-depth approach to specific areas of improvement, such as paraphrasing and reducing over-reliance on some sources.
Findings and analysis
Interpreting percentages of matching text
Significant improvements were found through the intervention of Turnitin with first drafts. Overall, there was a tendency to reduce the use of other authors’ words (i.e., 59% of students had a lower percentage of similarity in the originality reports for final drafts compared to first drafts). However, this can partly be explained by incomplete first drafts in which the first sections including the literature review were more heavily quoted, and furthermore, is not the most significant development of Turnitin. A student asked ‘Which percentage means good or bad?’ The software website itself does not answer this question, and some researchers have attempted to, such as Barrett and Malcolm (2006), who used a threshold of 15%. In this study, establishing an ideal percentage for the overall similarity index did not seem to be beneficial. The colour coding of Turnitin is useful, but a wide range of percentage is covered with each colour. The general perception of green may be safe, but it covers up to 24% similarity, and low percentages may still contain plagiarism. Meanwhile yellow results of 25–49% may not indicate incorrect referencing but are likely to indicate over-reliance on some sources. Peacock et al (2006) in a study that has some parallels with this one, considers that the approach to interpretation is crucial and necessitates clear guidelines and training for tutors to give good feedback to students. As considered above, percentages cannot be considered at their face value: for example, in this study, a student’s final draft was found to have a 72% similarity to another source, which in fact was the similarity with the student’s own first draft submitted to Turnitin!
Helping with citation
As previously mentioned, student feedback indicated an interest in learning more about how to use sources. Turnitin originality reports demonstrate uses of other authors’ words from their huge database, colour code them and indicate their sources through a web-based link on the right of the screen. Thus, when students have not cited correctly, it can be demonstrated clearly with Turnitin. Over 70% of students in both 2006 and 2007 stated that they thought Turnitin had helped them to write citations more accurately. Some of their comments were ‘We don’t know about citation before we come here. In my country, we need to use professional language in our writing, otherwise it does not sound right. We don’t need to put the author, just at the end’; ‘The software helped me to be more cautious about citation’; ‘For me, I will never ever forget about efficient citations according to my first experience of Turnitin’; ‘sometimes we make a mistake and don’t use citation or not correctly… this way gives us a second chance’. Thus the perceptions of students highlight the potential of Turnitin as a tool to know more and be more cautious about citations, to commit the learning experience to memory and to have a second chance to get it right. It would seem that Turnitin is well received in relation to this learning point.
Avoiding over-reliance on some sources
Over-reliance on some sources could lead to what Howard (2000) refers to as ‘patch writing’, i.e., correctly cited sources that are used so extensively that individual thought is not shown. While text matches found by Turnitin of 1% are widely considered insignificant, many students’ first drafts included matches to a single source of 10% of more, which is likely to indicate over-reliance. The highest results in the student feedback questionnaires of 2006–07 were achieved in answer to whether Turnitin was useful for avoiding over-reliance on one or certain sources, with more than 80% affirming. Comments included, ‘it helps my work lead to the right track. I would like to use it for to find out over-relianced (sic) sources because I tend to cite many points from one source’; ‘Turnitin is useful for avoiding over-reliance…because there is a sort of pressure, from the software to make aware of use of sources’; ‘it helps a lot to avoid over-reliance…because it shows you the percentage, like this source is 5%, in colour’. The visual impact from the originality reports seems very helpful in demonstrating over-reliance and in educating students to take more care in the amount they are using sources.
Thinking about appropriate paraphrasing
International students who are not native speakers of English tend to have difficulty in paraphrasing, and a focus is given to the practice of effective paraphrasing in academic writing modules such as this one. However, Turnitin may also be able to help by showing other authors’ words in colour next to the grey paraphrased words and the appropriateness of the re-wording may be discussed. Over 60% of students in both 2006 and 2007 found Turnitin useful for thinking about appropriate paraphrasing. Their comments were: ‘International students are not good at paraphrasing. So Turnitin is useful for students thinking about it’; ‘it highlights some part of bad paraphrasing, then we can correct it’; ‘it doesn’t tell us which part of paraphrasing is good – we get that from the teacher’. This last comment is important in seeing the role of Turnitin not as an educator in itself, but as a support to the educator, who needs to examine the results of originality reports carefully to give effective feedback to students.
Overall, the purpose of developing the above-mentioned learning opportunities abo
ut sources from Turnitin must be to fulfil the wider aim of educating students to avoid plagiarism. Over 70% of students in both 2006 and 2007 felt that Turnitin was useful for understanding how to avoid plagiarism. They said ‘the software shows your mistakes. Then we can correct them. At the end, when we submit the final draft we don’t want to fail’; ‘I would like to use it again because sometimes we do plagiarism without noticing it’; ‘I think checking plagiarism by computer is good because all students are trying to avoid plagiarism’; ‘in my country, I can copy a whole assignment, without writing a word, and get an A. Here it is different, and at the beginning I didn’t know anything. Turnitin helped correct a lot of my mistakes’. The last, rather worrying comment indicates the huge difference in practice and the extent of the challenge of learning and ‘unlearning’ for some international students. The students also identified how they may plagiarise without noticing and how they do not want to fail. Turnitin would seem to assist in the process of educating about what plagiarism is, and avoiding failure, although its success in these areas does seem to rely on the careful exploitation and interpretation of its reports by the tutor.
Tutor guidance or self-access?
Turnitin actively promotes self-access and offers peer review among other functions for those students registered with the site. Self-access for this assignment has been discussed and student comments such as ‘I would like my own access to Turnitin’ have been considered. However, at present, it is recommended that Turnitin be used through tutor guidance as recommended by Peacock et al (2006), because students may not have developed sufficient understanding of conventions to interpret originality reports. Student comments on this topic include ‘It would be good if we could submit 3 times then I think I would get it right’; ‘if I had my own access, I think I would spend a long time submitting again and again to get it right’; ‘I think it would be very helpful if students could use it. However, it may help students to do well-organised plagiarism’. Clearly, students submitting their work to Turnitin again and again without learning, or ‘well-organised plagiarism’, presumably by attempting ways to avoid plagiarism being detected, should not be encouraged. It was seen that students benefited a great deal from the guidance given in the tutorial feedback. For example, one first draft contained a useful definition of a key term, but without an acknowledgement of the source. In the discussion about the originality report, the source was shown as a reputable web-based dictionary and the tutor guided the student toward the realisation that the correct citation of the source was necessary to make it acceptable academic practice. If the student had been interpreting the results on his own, he may have decided that he needed to cut the highlighted section or make other undesirable changes. While it is acknowledged that the feasibility of individual guidance depends on the smaller size of the group, this study found tutor guidance to be preferable to self-access.
Drawbacks of Turnitin
While many of the benefits of using Turnitin in this way have been highlighted, three main drawbacks need to be mentioned. Some problems with sources were experienced in this study, such as sources being different to the ones students used, not being accessible, books not being available without an electronic format or database link, or not being on the database because of being a less popular discipline, as found by Barrett and Malcolm (2006). Another issue is that all other authors’ words are immediately identified by Turnitin, including those correctly cited and the bibliography, which may give inaccurate results. They may be omitted by using an extra function key, but the initial results may confuse students. The third drawback is that Turnitin undoubtedly creates more work for tutors in the administering, uploading, checking and tutorial discussions, as found in the study of tutor reactions to Turnitin by Sutherland-Smith and Carr (2005). However, this time appears to be well spent in achieving significant learning experiences for students and responding to their needs.
The student feedback from 2005 indicated the need to improve writing more between drafts and learn more about using sources. These needs appear to have been addressed in this study through the students’ improvement in their use of accurate citation, their reference to a more appropriate number of sources, their use of more appropriate paraphrasing and their greater understanding of how to avoid plagiarism. Turnitin provided effective support for these key improvements through its use in tutorial feedback on first drafts, without assessment. This conclusion coincides with that of Peacock et al (2006) that ‘when used in non-policing mode, (Turnitin) acted as a form of goal-oriented learning for good referencing practice’. Therefore, this approach to using Turnitin is shown to be assisting students in achieving the course objectives of improving use of academic writing conventions, gaining more competence in academic literacy and applying this competence successfully to a research paper. Students also evaluated Turnitin with first drafts as useful for this assignment and helpful in understanding more about plagiarism. Thus, this study examines a practice which goes some way toward ‘enhancing the students’ opportunities for successful learning’ as suggested by Wend (2006) in the Brookes Student Learning Experience Strategy.
The strongest recommendation from this study is to use Turnitin with first drafts before assessment, to provide beneficial learning opportunities. Its use through tutor guidance is suggested in order to ensure correct interpretation of originality reports and to avoid students misunderstanding, or misusing, Turnitin. Evidently, Turnitin should not be seen as a single solution but rather as part of an integrated approach to plagiarism education, as recommended by the JISC-PAS guidelines for good practice (Carroll and Appleton, 2001). While considerable more research into specific areas of improvement with Turnitin were made in the 2007 run of the module, further exploitation is still recommended, including its role in the understanding of ownership of words and ideas. While it seems many Brookes staff are aware of the potential for learning provided by Turnitin, further evaluation from tutors in a university-wide study would be very useful.
Barrett, R. and Malcolm, J. (2006), ‘Embedding plagiarism education in the assessment process’, International Journal for Educational Integrity, 2 (2), 38–45.
Bull, J., Coughlin, E., Collins, C. and Sharp, D. (2001), ‘Technical review of plagiarism detection software report’, (online) Joint Information Systems Committee, pp.1–36, accessed on 9/1/2007 from http://www.jisc.ac.uk/uploaded_documents/luton.pdf.
Carroll, J. (2005), ‘Deterring, detecting and dealing with plagiarism’, a brief paper for Brookes staff for Academic Integrity week, accessed 4/1/07 from http://www.brookes.ac.uk/services/ocsd/5_research/jude.html.
Carroll, J. and Appleton, J. (2001), ‘Plagiarism: A Good Practice Guide’, Oxford Brookes University Guide Series (online) Joint Information Systems Committee. Accessed on 10/1/2007 from http://www.jiscpas.ac.uk/apppage.cgi?USERPAGE=6202.
East, J. (2005), ‘Proper acknowledgement?’, Journal of University Teaching and Learning Practice, 2 (3a), 1–11, accessed 14/9/2006 from http://jutlp.uow.edu.au/2005_v02_i03a/east005.html.
Handa, N. and Power, C. (2005), ‘Land and discover! A case study investigating the cultural context of plagiarism’, Journal of University Teaching and Learning Practice 2 (3b), 64–86.
Hayes, N. and Introna, L.D. (2005), ‘Cultural values, plagiarism, and fairness: when plagiarism gets in the way of learning’, Ethics and Behaviour 15 (3), 213–231.
Howard, R.M. (2000), ‘Sexuality, textuality: the cultural work of plagiarism’, College English, Urbana. 62 (4), 473–491.
Peacock, S., Sharp, J. and Anderson, S. (2006), ‘Pickup a plagiarism practice? A holistic approach to deploying Turnitin UK as a sustainable driver in altering institutional plagiarism practice’, Centre for Academic Practice, Queen Margaret University College, Edinburgh. Paper presented at JISC International Plagiarism Conference, Newcastle, 19–21 June 2006.
Sutherland-Smith, W. and Carr, R. (2005), ‘Turnitin.com: teacher’s perspectives of anti-plagiarism software in raising issues of educational integrity’, Journal of University Teaching and Learning Practice, 2 (3b), 94–101.
Wend, P. (2006), ‘Towards a joined-up student learning experience at Oxford Brookes University’, Brookes e-Journal of Learning and Teaching, November 2(1). Accessed on 21/5/07 from http://www.brookes.ac.uk/publications/bejlt/index.html/.