Investigating student experiences of e-learning using the Diary Interview Approach

Introduction

With the rapid growth in technology, and its expanding use within an educational context, we feel that more research into students’ experiences of using technology for learning is necessary. Putting the student voice at the heart of the agenda, our research investigates the student experience of e-learning in order to identify areas of good practice and potential areas for improvement.

Consistent with previous research conducted at Sheffield Hallam University (Aspden et al., 2003), our research was designed to consider the holistic student experience whilst placing an emphasis on engagement with the virtual learning environment and other technologies used to support student learning (Thorpe and Lyons, 2008). The aim was to gain a greater understanding of how our students are engaging with technology to support their learning and for this to influence policy and practice at the University.

Method

The research employed the diary interview approach (Zimmerman and Wieder, 1977). This method was felt to be most appropriate in gathering rich, participant-focused data that would provide insights into the role of technology in the students’ daily learning experience (Lyons and Thorpe, 2008).

An invitation to participate in the study was advertised via the University’s student portal and a pre-study questionnaire was used to establish the suitability of the candidates to participate in the study. The questionnaire established basic demographic data and determined whether students fulfilled the necessary criteria to participate in the research. All participants were required to own a mobile telephone and be enrolled on at least two Blackboard module sites. This was a small-scale, qualitative study and whilst the sample was not intended to be representative of the entire student population, participants from each faculty and level of study were selected to continue to the next phase of the research. Eleven students participated in the research; eight undergraduate and three postgraduate. The participants were aged between 18 and 46. Participants were predominantly based on campus, but two were distance-learning students.

Selected participants were invited to attend a briefing session where they were informed of the purpose of the study, provided with a notepad in which to keep a written diary and a disposable camera (where no digital camera or camera phone was owned). Distance-learning students were given the option of whether to attend the face-to-face briefing session or a telephone briefing, but chose to attend a face-to-face session as they were from the local area. They were also given access to a Blackboard organisation site containing relevant information related to the study, contact details of the researcher and access to a private space where they could keep an electronic diary in the form of a blog.

Participants were required to keep a diary over a two-week period towards the end of the first semester, making daily entries with three elements

  1. description of learning activities
  2. reflections upon those learning activities
  3. responses to specific questions set by the researcher.

Diaries could be paper or electronic, text or image, or mixed media, whichever participants felt most comfortable with and most appropriately met their needs.

In the event all participants chose to use electronic, text-based diaries, suggesting that this was due to convenience, accessibility and assurance that their electronic diary would not be lost in comparison to a physical diary which they would have to carry around and could easily be misplaced (Lyons and Thorpe, forthcoming). The questions that informed the third element of the diary entry were sent via daily SMS text message (See Appendix 1 for list of questions). This aspect of the study received positive feedback from all participants who noted that as well as serving as a reminder to complete their diary, it also allowed them time to reflect on the question and provided some structure (Lyons and Thorpe, forthcoming). It was hoped that the diary would provide an accurate account of the student experience of e-learning, and how they felt about the use of technology within their studies.

Following the diary phase, participants were required to attend a short interview during which they were provided with an opportunity to elaborate on points raised within their diaries and some issues of interest were explored in greater depth. Again, the distance-learning students were given the option of a telephone interview but they attended in person.

Findings

In using the diary interview approach, and providing the participants with a question on a daily basis to form the focus for their diary, it was possible to draw cross-diary comparisons, highlighting similarities and differences in participants’ experiences of e-learning. In obtaining 11 unique accounts of ‘the student experience of e-learning’, the research highlighted the differences in learners’ experiences of e-learning. The insights into the individual differences in learners’ experiences highlighted by this study pose a significant challenge in how to move forward the institutional e-learning agenda to meet the ever-changing needs and expectations of an increasingly diverse student population.

Initial analysis of the diaries has identified a number of similarities in participants’ experiences. Whilst the study considered e-learning within the context of the holistic learning experience, there was a greater focus on the use of the institutional virtual learning environment, Blackboard. Participants made numerous references throughout their diaries, partly in response to a question about how their learning is currently supported by technology but also in other aspects of their diaries, as to what they considered to make a good module site within Blackboard, i.e., how technology can be used to enhance their experience of learning that module. Regardless of the discipline or level of study, participants’ views were largely similar. Participants felt that a good module site should:

  • be full of resources (including lecture notes and links to external resources),
  • be updated regularly,
  • have spaces to accommodate online group work, specifically highlighting the benefits of wikis to support this
  • facilitate online self-testing, online submission of assignments and online feedback.

Though participants were on the whole in agreement about what would make a good module site, each individual’s experiences of module sites differed greatly from site to site, and again from one student to the next. This was often due to variations in tutor engagement with Blackboard. Where Blackboard was used well and was regularly updated with relevant resources, students appreciated the benefits that this approach offered, and often compared this experience to less-well-used sites, suggesting that their learning was suffering as a result. There is an issue here of very good use of technology making weaker use look disproportionately worse. Work at the University is currently underway to enhance consistency across modules so that learners know what to expect from the virtual learning environment. The idea is not to standardise use of Blackboard, but to ensure that, at subject level, tutors work collaboratively (often with the help of a student e-learning assistant) to deliver module sites that follow a consistent structure so that students can easily navigate each of their module sites.

Students also appreciated the use of technology to support both formative and summative assessment, the online submission of assignments, and online grades and feedback. All participants were positive about the benefits that technology had to offer in terms of assessment, although some were not provided with the opportunity to use technology in this manner. However, others were able to provide tangible examples of how technology had enhanced their experiences of assessment.

‘The best learning activity I’ve done is the practice multiple-choice exams. You could retake each test and the questions would come in different orders to ensure you had actually learnt the answer and at the end of the test it showed your percentage and then all the right and wrong answers. It also explained the correct answers to aid learning. I found these tests invaluable as they really tested my knowledge and prepared me for the exam and as a result I achieved my highest grade in that module.’

Using technology in this way allows learners to take control of their own learning. Online formative testing requires an initial investment from the tutor in terms of setting up the tests but once set up, students can use the tests as often as they like in order to enhance their learning with little ongoing support from their tutors.

In addition to promoting the use of online formative testing, the University is currently introducing a custom-built Assignment Handler in order to facilitate the online submission of assignments and subsequent feedback on assignments to the students.

Overall, participants in the study recognised that e-learning was a crucial part of their learning experience. As part of the study, participants were required to ask three other students how important e-learning was to their overall learning experience. This element of the study served to highlight some disparities in students’ experiences of e-learning. While the majority of those asked felt that e-learning was beneficial, each reflected on very different elements and a minority felt that e-learning was unimportant. Of those who felt that e-learning was unimportant, little data was gathered regarding their reasons, but one stated that their tutors did not use Blackboard. Of those who felt that e-learning was beneficial, some themes emerged as common factors including employability and digital fluency (students talked about how e-learning was helping them to develop the skills to enable them to work in the digital workplace, including IT skills and critical thinking skills), and how level of competence with IT can affect the overall experience: ‘It really affects your ability to participate on the course if your IT skills are limited.’

The University is currently promoting a ‘Digital Fluency’ initiative which aims to raise awareness of the skills required to live learn and work in the digital age. The initiative is aimed at both students and academics.

Tutor engagement with e-learning is an important contributing factor to the overall experience of e-learning. This study found that students assigned the same amount of value to learning activities and tools as their tutors. Thus, where tutors assigned little importance to e-learning and did not engage with the virtual learning environment, students became disillusioned by e-learning and did not engage. An example of this came from a distance-learning student who felt that the potential benefits of e-learning were enormous, but when the Blackboard sites used to support her modules were not updated after the first week she lost interest and stopped logging into Blackboard. In addition, tutors must state clearly the purpose for using e-learning within each specific context. Where e-learning was seen as an additional task with no clear purpose, students lacked commitment to the tasks.

‘There are e-learning objectives tacked onto the course wherever people can fit them. For example, today I have had to make use of the discussion boards for a module that really does not require this. If the objectives are so important they should be focussed on separately or at least somewhere they make sense.’

The University strongly recommends that all Blackboard module sites contain a rationale section that defines the purpose for the sites and sets out what is expected from students in terms of engagement with the site.

Conclusion

The study highlighted a number of common experiences with technology and its impact upon learning. These served to reiterate findings from earlier research (Aspden et al., 2003) and anecdotal evidence from academics. The University has used this broad set of evidence to inform policy and practice including implementing several action-based initiatives. There is the need to explore further the complexity of meeting diverse needs and expectations and the associated impact of change initiatives upon the evolving student experience.

Biography

Helen Lyons is a researcher in e-learning at Sheffield Hallam University. Her current role is to investigate student expectations and experiences of e-learning, and the application of technology to support assessment and feedback.

Louise Thorpe is the Head of Academic Innovation and her responsibilities include the development of e-learning across the University and its evolution into a personalised online learning environment, digital fluency, evolution of innovative learning spaces, and the application of new and emerging technologies to learning.

Contact details

Helen Lyons

Sheffield Hallam University, Learning and Teaching Institute, Level 7 Adsetts Centre, City Campus, Howard Street, Sheffield, S1 1WB,
Phone: 0114 2254744
Fax: 0114 2254755

References

Aspden, L. Helm, P. and Thorpe, L. (2003), ‘Capturing learners’ experience with e-learning: preliminary findings’, in J. Cook and D. McConnell (Eds.) Communities of practice. Research Proceedings of the 10th Association for Learning Technology Conference (ALT-C 2003), 8–10 September 2003, The University of Sheffield and Sheffield Hallam University, UK.

Lyons, H. and Thorpe, L. (2008), ‘The Diary-Interview Approach: Exploring student experiences of e-learning’, Paper presented at Improving Student Learning—Through the Curriculum, 1–3 September, University of Durham, Durham, UK.

Lyons, H. and Thorpe, L. (Forthcoming), ‘The Diary-Interview Approach: Exploring student experiences of e-learning’, in Proceedings of ISL 2008—Through the Curriculum, 1–3 September, University of Durham, Durham, UK.

Thorpe, L. and Lyons, H. (2008), ‘Diaries and Decisions: Placing the student voice at the heart of the developmental agenda’, Paper presented at BbWorld Europe ’08: Engaging Student, Engaging Communities, 12–14 May, Manchester, UK.

Zimmerman, D. H. and Wieder, D. L. (1977), ‚The Diary-Interview Method’, Urban Life, vol. 5, no. 4, pp. 479–98.

Appendix 1: Prompt Questions

  • How is your learning currently supported by technology? Think about the module that is best supported by Blackboard and describe why.
  • Think about your learning experience without technology. How would it be different and why?
  • Think of a learning activity that you did today (or recently). If you were to design this activity how might you do it differently?
  • How confident are you in using technology and where do you go to for support?
  • Describe the best learning activity that you have done since you arrived at the university and explain why it was effective
  • Describe what kind of learning activities you did over the weekend, how you did them and why. Did you use technology, and did this help or hinder?
  • Has anything that you have experienced today made you think differently about how you would approach your learning tomorrow?
  • How important is e-learning to your overall learning experience? Ask three students the same question and write about their thoughts.
  • Describe your last assessment and how technology was or could have been used.
  • If there was one message that you would want to give about your learning experience at Hallam, what would it be and why?
Posted in Research Notes

Leave a Reply