Sustainable Metrics in Higher Education

From assessment of learning to assessment for learning within datafied systems

Yes, mostly done, this year. The Webinar Series on Data Cultures in Higher Education is reaching its end, and this piece could not be missed: the needed talk about metrics, quantification and assessment in Higher Education. And for this complex endeavor I called my colleague and friend Valentina Grion, from the University of Padua

My dialogue with Valentina Grion started early in 2013, when she invited me to take part, together with Patrizia Ghislandi (University of Trento), to her book on Students’ Voice (the Italian perspective). Those years, she was the only one talking about the need to listen the students not only as consumers of the schooling and the Higher Education system, but mainly as stakeholders and builders of such system. Our reflection with Patrizia retraced the need of engaging the students within quality cultures, in connection with a number of experiences we had been conducting in understanding how the students perceived and evaluated quality teaching and learning.

Valentina then devoted her work to Assessment. She started looking at such area of pedagogical practices inspired by her findings in the Students’ Voice movement: little or no engagement of the youngest in the system, connected to little or no willing to share spaces of leadership within the educational process from the side of teachers and institutions. I followed her in several papers dealing with the problem of peer-assessment and self-assessment, as well as self-assessmentboth at the school level and in Higher Education.

Here are just examples of her productive work (not to mention the numerous contributions in Italian):

• Peer assessment practices in an online context: does the group size matter?
• Self-Assessment in the university context: a systematic review
• IMPROVe: Six research-based principles to realise peer assessment in educational contexts
• Introduction to the Issue: Assessment and Feedback in the 21st Century: Lessons Learnt from the Past and Future Directions.
• The Power of Giving Feedback and Receiving Feedback in Peer Assessment.
• Peer feedback and technology-enhanced assessment as critical issues to foster student learning

Overall, two relevant citations can describe Valentina’s endeavor:

Assessment is probably the most important thing we can do to help our students learn.  We may not like it, but students can and do ignore our teaching; however, if they want to get a qualification, they have to participate in the assessment processes we design and implement (Brown, 2005, p. 1)

Assessment methods and requirements probably have a greater influence on how and what students learn than any other factor. This influence may well be of greater importance than the impact of teaching materials (Boud, 1988, pp.39-40).

During the last year, we had an informal correspondence where we shared our research topics. We suddenly become aware that there were many research problems in our respective topics at the crossover: data literacy within pedagogical practices might be embedded within an overall assessment literacy. Valentina was hence incredibly open to spread my research comparing data practices in higher education. We then collaborated in analysing data practices connected to data in quality evaluation and data in assessment (we are now writing two articles!).

Therefore, the Webinar Series Data Cultures in Higher Education worked well as a space to open such reflection. We aimed at exploring the need to move towards sustainable data/metrics which support a perspective of assessment for learning. Here are some of the questions we explored together:

• Why do you think assessment is such a crucial element of pedagogical practices
• How can we empower students through assessment, and give them a voice?
• Which is the problem with a focus on quantitative results, namely, assessment of learning, and the lack of focus on a broader/holistic way of seeing assessment, namely, assessment for learning?
• Which are your concerns (both technical and ethical) connected to implementing learning analytics and AI within assessment practices?
• How can we, as educators, be prepared to face such a challenge?

Here is Valentina’s presentation.

You might have noticed that whereas the initial questions were more based on Valentina’s experience and current research on sustainable assessment, the last questions were open doors to future research.

The Webinar triggered the debate over teachers’ competence to put sustainable assessment into practice. But it also called for action relating to the relevance given to numeric indicators, scores and grades that very much reflect assessment of learning, namely, the traditional idea of assessment.

As we move further in developing AI systems and datafied learning environments, it’s clear that the pedagogical (mis)conceptions on assessment are pervasive and just reproduce a system of toxic metrics. The debate about proctoring software (example here); the scandal of automatised grading in the UK; the huge concern about exam cheating with absurd strategies like using double webcams; or the learning analytics debate on the lack of pedagogical support to the systems: these are but warnings about how the data culture in assessment sticks to the idea of assessing surface knowledge, more than deep engagement and competence development.

Along our debate, another concern emerged: that of the assembled system of metrics preventing university educators to step aside the assessment of learning practices. If teaching is given less importance; HEIs have become a sort of educational clockwork were more graduates are expected by the system; and such productivity is hence traced and measured to produce university rankings, moving towards a sustainable assessment system appears difficult. Indeed, sustainable assessment is time consuming and requires considerable teacher’s workload and students’ engagement and responsibility.

This is what Valentina pointed out: through her Students’ Voice perspective, she saw clearly the other side effects, namely, the little co-responsabilisation by the students as stakeholders in the system, which follows to surveillance, proctoring, and other top-down approaches in the educational relationship.

Doubtless, there is a long way to go from here: no technological system will change human values and institutional strategies in assessment, but rather the opposite.

She is also known as the national advocate of the Student Voice approach. She founded and currently leads the PAFIR (Peer Assessment and Feedback International Research group) promoting scientific initiatives on the topic of assessment and feedback. She has published numerous articles and book chapters.