The findings discussed below emerged from a thematic analysis of five open-response items from the large-scale survey of staff and students at Deakin University and Monash University (click here for more information on method and participants). These items addressed respondents’ general understandings of feedback, as well as their experience of a specific, self-selected instance of feedback. The five items are as follows:

• What is the purpose of feedback? (answered by staff and students)
• How do you know whether your feedback has been effective? (answered by staff)
• Why was [the specific feedback example] effective? (answered by staff and students)
• How do you know [the specific feedback example] worked? (answered by staff)
• Describe a time when you received feedback that was not effective. (answered by students)

Key takeaways

• Students typically had more simplistic understandings than staff regarding the purpose of feedback and features of effective feedback. Without a deep understanding of feedback, students may be limited in their ability to fully utilise it to improve. Academic staff may consider addressing this issue by assisting students to develop feedback literacy.
• Between 6-9% of staff and students were not able to judge whether a specific example of feedback had been effective. This is concerning, given that the principles of effective feedback design emphasise the need for students to have opportunities to demonstrate improvement.
• Students consider feedback ineffective when the comments are unhelpful, unusable, general, lacking in detail, or too critical. Nearly one in five students also felt that not receiving any comments at all was a form of ineffective feedback.

Method

Data presented below are based on a representative sample of 400 students, which includes 200 students from each institution. Sampling was conducted based on the characteristics of each institution’s overall student populations, in terms of gender, international/domestic enrollment, on-campus/online enrollment, and faculty. Sampling for staff was not conducted, as the entire data set of interest (i.e. staff with assessment responsibilities) was a comparable size to the students subsample (n = 323 staff). Percentages reported below are based on these sample sizes; however it should be noted that individual responses were assigned multiple codes, thus totals may exceed 100%.

For the thematic analysis, data were coded using a framework developed through an iterative process of reading subsets of data and testing preliminary codes. Four members of the project team were involved in this process, and five major iterations of the framework were developed. Once a preliminary framework was finalised, a fifth member of the project team then coded a sample of the data, and minor amendments were made to the framework. The final coding framework was then applied to the entire dataset.

Findings

Purpose of feedback

Staff and students were asked to indicate what they considered to be the purpose of feedback. Three staff and fourteen students did not respond to this question, and were excluded from analysis.

As shown in the graph below, the majority of both staff and students were clear that the purpose of feedback should be to improve. Interestingly, a much higher proportion of staff were likely to suggest that feedback should serve an affective purpose, such as offering students praise or encouragement, although as data presented below reveals (see Features of Ineffective Feedback), students consider negative comments (such as those that are too harsh or critical) to be a marker of ineffective feedback. A higher proportion of staff than students also mentioned that the purpose of feedback is grade justification. These results suggest that the majority of staff and students understand that improvement is a critical aspect of feedback. However, it is interesting to note that students were less likely than staff to hold the opinion that the purpose of feedback is to provide affective comments to students and to justify grades.

There were many subthemes within the  large proportion of staff and student responses that mentioned that the role of feedback is to improve learners’ work. As shown in the graph below, the most common responses from both staff and students were that feedback should lead to unspecified improvement or improvement in student work. In both cases, student percentages for both capacities were higher than staff percentages. On the other hand, staff responses were more likely to reference sophisticated pedagogical understandings and language when identifying capacities that feedback should improve. For example, a higher proportion of staff than students reported that feedback should improve students’ learning or study strategies, and students’ reflexive, self-evaluative and critical thinking.

Overall, while staff and students largely agree that the purpose of feedback should be to improve, staff typically recognise a broader range of areas in which feedback might support students’ improvement. This suggests that students have less complex understandings of the purpose of feedback than staff. Additionally, this disparity could hint at mismatches between implicit feedback cues that staff may consider obvious, but which students may fail to recognise and implement. Teaching staff might consider broadening students’ feedback literacy by explaining the role of feedback and outlining ways in which it might be used.

Features of effective feedback

Staff and students were asked why they thought a specific, self-selected example of feedback had been effective. Eighty-eight staff and 39 students did not respond to this question, and were excluded from the analysis.

The most common reasons provided by respondents related to content of the feedback comments and aspects of feedback design. However, the strength of these themes differed across staff and students. A vast majority of students (83.8%) reported that effective feedback is most likely to be determined by the content of comments. Of these, 49% mentioned that feedback is effective when comments indicate knowledge and skills that students should focus on in future, while 20% felt that feedback is effective when it is detailed, specific and thorough. This theme also emerged among staff respondents; however, it was less prevalent than with students. As with students, staff responses in this theme primarily related to the usability of comments on knowledge and skills (12%) and detailed, specific and thorough comments (10%).

The strongest theme among staff (52.6%) was that feedback design is a key factor in effective feedback. Within these responses, staff referred to feedback that was iterative (12%), returned quickly or promptly (10%), delivered comments face-to-face (9%), involved students playing an active role in the feedback process (9%), or incorporated peer feedback (7%). In contrast, only a small number of students overall (17.3%) reported that feedback design is a factor in effective feedback, with face-to-face feedback comments being the most commonly cited feature (7%).

Hence, staff were more likely to focus on feedback design, while students were more likely to focus on the content of comments. This split likely reflects a tendency for students to primarily engage with feedback through comments. Even where staff may have invested significant thought, time and effort in designing feedback, students are less likely to be aware (or to have been made aware) of how and why feedback has been designed.

Evidence of effective feedback

Staff were asked how they know if their feedback has been effective, and students and staff were also asked how they knew that a specific, self-selected example of feedback had been effective. Sixty-one staff and 51 students did not respond to this question, and were excluded from analysis.

Nearly all students (92%) reported that some form of change is the clearest indicator that feedback has worked. Of these students, 51% told us that they know feedback has been effective if it results in a change to their grades or work. Similarly, just over half of staff overall (55.3%) noted that some form of change is the indicator that they use to determine whether the comments they provide to students have been effective. A further 34.1% indicated that they use comments or information from students – including unsolicited comments, student evaluations, and reduced queries and requests to re-mark – to evaluate whether feedback has worked. Of the staff who indicated that they use some form of change as evidence of effective feedback, nearly two thirds (59.7%) mentioned that they know feedback has been effective if it results in a change to a student’s work.

Given that principles of effective feedback design emphasise the need for students to have opportunities to demonstrate change or improvement, it is concerning that 8.9% of staff and 6% of students reported that they are unable to judge if feedback has been effective. Teaching staff may need to consider designing opportunities for students to demonstrate the effect of feedback, for instance in a subsequent, related task.

Features of ineffective feedback

To gauge the features of ineffective feedback, students were asked to describe a time when they had received feedback comments that were not effective. Eighty-four students provided invalid or no response, and were excluded from analysis.

As with features of effective feedback, a strong majority of students (81%) reported that ineffective feedback is likely to be determined by the content of comments. Of these, 24.8% reported that feedback is ineffective when comments are unhelpful or unable to be used. Similarly, students reported that feedback comments are ineffective if they are generic (12%) or lack detail (15.5%). These results suggest that, while students may wish to make use of feedback, they can be thwarted by vague or poorly-expressed feedback comments.

Unsurprisingly, 18.25% of students reported that feedback is ineffective when no comments, or only a mark, is provided. It is concerning that nearly one in five students has experienced an instance of ‘feedback’ in which comments were not provided, particularly given staff and students overwhelmingly recognise the role of feedback in effecting improvement. Interestingly, 9.25% of students reported that feedback comments can be ineffective if they are perceived to be too critical or harsh. This result reminds us that feedback can be an emotional business, and that potential affective impacts of comments should always be a consideration for teaching staff when providing comments to students.