Understanding Learners’ Needs Through Evaluation

Understanding our learners’ needs is a vital part of my work. I ensure there are moments to gather student feedback during consultations, testing, and evaluation phases of the projects I undertake. A key example of this is my work with the Creative Unions unit. 

Creative Unions is a cross-course unit that emphasises collaboration. It is a large-scale initiative; this year, over 1,000 students will participate in the unit simultaneously over five weeks. Delivered through a blended learning model, I partner with the Unit Leader to plan and develop the digital components of the unit. My responsibilities included structuring the Moodle environment, setting up Miro boards, and facilitating all live online events. 

The goal is to integrate the digital elements seamlessly with face-to-face teaching, ensuring all students could access the resources and information needed to complete the unit successfully. Given the scale of the unit, the digital content is crutial; students missing live events or recordings could lose significant learning opportunities. Additionally, with 27 Associate Lecturers facilitating the unit, Moodle and Miro became essential tools for delivering a consistent learning experience. This consistency is especially important as the unit has been to growing year on year.

Task

At the outset of my work with Creative Unions, I recognised the importance of understanding how students experienced the digital elements of the unit. This understanding would allow us to make necessary adjustments and ensure our systems were robust enough to support the growing numbers. 

Action

To address this need, I implemented an annual evaluation process following the completion of the unit. This process included: 

1. Focus Groups: I recruited students who had recently completed the unit to participate in focus groups. These students were incentivised with National Book Tokens to encourage participation. 

2. Data Analysis: I examined analytical data from Moodle to track engagement and identify potential issues with access or navigation. 
3. Surveys: Teaching staff, including the 27 Associate Lecturers, were asked to complete surveys to provide feedback on their experiences with digital tools and resources. 
4. Reporting: After collecting and analysing data, I identified patterns, produced a report, and made recommendations for improving the unit’s digital delivery in its next iteration. 

Results

In 2023, the evaluation process enabled us to identify and address the following key issues, as well as validate the elements that were working effectively. Some significant outcomes included: 

• Moodle: Feedback from students indicated that they appreciated having all necessary resources gathered in one place and made available in advance, which allowed them to manage their time more effectively. However, some students still expressed feeling overwhelmed and struggled to locate information quickly, particularly regarding lecture recordings and the reflective journal template. To address this, we implemented color-coding for topics and sections to help students find information more easily, and we added recordings to the online sessions topic.

• Communication: Students expressed their dissatisfaction with receiving multiple Teams links in their calendars, as it made it difficult for them to locate the correct one. In response to this feedback, the CU lecture Teams room set-up was changed. Instead of multiple links, there is just one room with a single link. This change has resulted in increased attendance at optional sessions, including those offered for library and academic support.
• Miro: Responses from students showed a significant divide in their opinions about Miro. Many acknowledged the usefulness of the Miro board, especially when they missed a class and needed to catch up. However, some students felt that the extensive use of Miro activities during in-person sessions negatively affected team interactions. As a result of this feedback, there has been a decision to reduce the number of Miro activities and an effort to encourage students to discuss topics face-to-face and upload images and work once the discussion has been completed. 

Reflections

Reflecting on this evaluation process, I see both advantages and limitations. Holding a focus group with students immediately after the unit allows me to identify key trends and issues they have experienced. By providing prompts for discussion, the format encourages students to express their concerns in their own words, offering valuable qualitative insights.

However, one noticeable challenge is the relatively small number of students participating in the focus group, especially given the unit’s large scale. For example, only six students attended the previous session. While focus groups are intentionally smaller due to their discursive nature, participation remains limited despite incentives and online sessions.

This raises the question of whether we should reconsider the timing of the focus group, currently scheduled after the unit ends. Or explore alternative forms of incentivisation to encourage greater participation. The simplest way to increase participation might be to start recruitment earlier and ask the facilitators to encourage participation.

Another strategy would be to combine the student focus group data with the student survey text responses. This could help to reassure that the themes were representative of a broader group.