How to redesign teaching and assessment to respond to the abrupt appearance of AI has become a heated question for Higher Education. For some institutions, how to reliably validate learning has become an existential challenge. This is less the case for Arts Education which has always focused on process and the demonstration of learning, avoiding over reliance on a final ‘product’ in assessment. I believe that many institutions and subjects could now benefit from applying Arts Education thinking.
My role at UAL includes developing strategies to respond to the challenges and opportunities of AI, this post is my thoughts on the recently published QAA draft principles on academic standards in the age of AI rather than a UAL position. It outlines where Arts Education maps well to the draft principles, and where the territory is still problematic for all.
Arts Education is (part of) the answer
Many institutions have developed AI policies, positions and principles. How successful these are is not yet clear, but I have seen a recent trend towards Principles (including our approach at UAL) as a response which can weather the noise around AI’s potential impact.
MIT recently published an interesting report on ‘AI Use in Teaching, Learning, and Research Training’ which leads with Principles. The UK Quality Assurance Agency for Higher Education also recently published ‘Assuring Academic Standards in the Age of AI: Draft sector principles for assessment validity and standards’.
What both have in common is that an Arts Education approach to teaching and assessment hiding in plain sight. Neither expresses this view directly, however, the educational practices best placed to respond to the reports look distinctly Arts Education.
Responding to the QAA
To illustrate my point, I’m going to run through some of the headlines in the QAA draft report and describe how a well-informed Arts Education approach is a valid response to much of what has been drafted.
This is not to say that we don’t have a significant amount of work to do at UAL, or that our approach to teaching and assessment is always as robust or confident as it could be. I’m drawing from the many practice-based discussions I’m party to, rather than an exhaustive knowledge of everything Teaching and Learning that happens across the institution.
The underpinning approach in the QAA draft principles
It’s reasonable to design a chunk of assessed learning banning the use of AI as an engine in the production of outputs (as distinct from AI use in ‘finessing’ work, such as grammar checks or for access purposes). It’s also reasonable to specify the use of a particular genre of AI tool where this use is the focus of the learning.
However, the QAA draft principles are attempting to outline practices which maintain learning as a process on the assumption that students are using AI on a day-to-day basis and probably in ways which will damage their learning if this risk has not been considered in the design of their learning. The draft principles take the position that detection of AI use or policing of use ‘after the fact’ (i.e. after work has been submitted) is not an effective strategy to maintain academic integrity.
The emphasis then is on the design of learning and assessment which is inherently both inclusive and discourages damaging (in educational terms) AI use. The aim here being to respond to the cause through robust design, rather than the symptom through policy and regulations which are likely to be unenforceable.
I agree with this approach, but it does raise some difficult questions.
- Not relying on the academic integrity of students is an erosion of trust. However, this is less the case when students are not clear on where AI use starts to damage the learning process. The design of learning and assessment should, in some senses, manage this on the student’s behalf. i.e. It’s not fair to ‘trust’ students to make pedagogic judgements but critical literacy should be taught in some form to support students in identifying how/where to stay intentional. (See below for my video of my recent talk on this at MTU)
- It’s genuinely difficult to imagine a way of managing ‘some but not all AI is allowed’ learning and assessment that responds to these draft principles. Even defining the edges of what genres of AI and what practices are allowed is tricky. Designing the pedagogy that can then identify those edges and keep things on track is even harder.
The ‘some but not all’ approach probably works ok where specific AI skills are being taught directly but where an academic practice is much broader, such as essay writing, this middle way tends to put the burden on the student to interpret a complex and rapidly changing territory (or worse they have to second guess the interpretation of the academic who might be marking their work).
Again, this is where the teaching of critical literacy is crucial. As is equipping students to develop strategies which maintain their agency and intentionality. This is no small ask within curriculum which is already likely to be full to the brim with subject specific content and assessment. - Most effective or legitimate responses to the draft principles are likely to put more pressure on teaching staff. Adding ‘meaningful moments’ of accountability which build towards a final submission requires time and focus. Similarly, shifting towards more dialogue-based forms of assessment is difficult to do within workload models where time allotted per student has been trimmed down over the years to balance the books.
At UAL we have spent years developing systems and practices which allow us to undertake this form of teaching and learning at scale but it’s not always easy and it is not cheap.
The Draft Principles where an Arts Education approach works well
I’m going to group some AI specific extracts from the draft principles rather than going through them one by one in full.
From draft principle 1: Assessment provides valid evidence of learning at individual assessment and programme level
“ Assessment is designed to reasonably verify the student’s own learning and capability, rather than the output of an AI tool, against the learning outcomes, including, where AI use is permitted, the student’s direction and critical evaluation of that use.“
From draft principle 3: Summative assessment includes proportionate points of assured demonstration across the programme
“Providers identify meaningful points across the programme where independent capability must be demonstrated… That demonstration can take different forms, such as oral, practical, observed, placement-based, simulation, presentation and questioning, or structured defence of an artefact.”
From draft principle 4: Assessment drives learning, building foundational knowledge, skills and competencies over time, valuing process alongside outcome
“Students are assessed on how they reach, justify and refine their work as well as on the final product. Feedback is timely, effective and designed to be applied to improve later tasks.”
From draft principle 7: Assessment security is supported by realistic, meaningful and transparent academic regulations and accessible policy and regulation, not by academic integrity alone
“Assessment security consists of meaningful actions to prevent and detect attempts by students to avoid accountability for their learning. Detection here means gathering evidence of learning, for example through questioning or evidence of process, rather than reliance on AI-detection software.”
How UAL education maps to the points above
Almost all assessment at UAL is the wholistic assessment of a creative portfolio with other elements such as written reflections and/or creative artefacts. Most of our teaching is geared around project-based learning where a creative brief is set and then project work is assessed against Learning Outcomes which are geared to our assessment criteria.
This approach embeds several elements which respond well to the substance of the QAA draft principles noted above:
- Project-based learning inherently builds incrementally toward summative assessment.
- In keeping with the project-based approach, teaching incorporates ‘meaningful points’ where the work-in-progress is discussed and tested. These commonly act as moments of accountability where students will demonstrate ‘independent capability’.
Often these moments are dialogue based, incorporating formative feedback which builds understanding and knowledge towards the final submission. This is rooted in studio-based pedagogy where work-in-progress is developed visibly and shared within trusted groups for reflection and discussion.
This is not to say that all UAL education is centred around a physical studio, it’s more that the philosophy of ‘studio’ is woven through much of our offer. - One of our five assessment criteria is ‘Process’. The portfolio constructed by the student is used not only to defend the validity of a final output but also to tell the story of how that output was arrived at. In effect, the portfolio documents the decision making of the student. It is a record of critical thinking and therefore strong evidence of learning-through-process. Other elements such as written self-reflections add more opportunity to describe ‘what was learned’ to go beyond describing ‘what was produced’.
If AI has been used as process then the creative/project portfolio should contain an account of how this use influenced learning/understanding/identity. A list of AI technology used with no critical reflection is of very little value and can easily become a non-enforceable form of policing ‘after the fact’. Additional constructive approaches to acknowledging AI use are outlined by Borowik, Smith, and Birkett on the UAL AI Network blog. - Another UAL assessment criteria is Realisation. This is focused on ‘standards of production’ and as such is weakened by AI as a proxy for learning, especially in the light of the draft QAA principles. For many of our students Realisation shakes out as ‘how good is the thing I’ve created’ or ‘to what extent would I want to include this in my professional portfolio when seeking work’.
Ian Truelove and I have been discussing how AI pushes many aspects of Realisation outside of ‘evidence of learning’ and reemphasises the importance of Process. One response to this would be to design learning wherein Process is awarded credit and Realisation is validated by industry professionals (which includes many of our tutors). The same people might be assessing both criteria but within distinct framings which helps to reduce the temptation for students to over focus on product over process.
Problematic aspects of the draft principles
From draft principle 2: Assessment is inclusive and accessible by design
“Assessment is designed to be inclusive and accessible from the outset, rather than adjusted after the fact.”
Clearly this is a good idea. My question though is about AI resistance or refusal. If the course of study is not optional or not obviously signalled as incorporating AI is it acceptable to mandate AI use as evidence of meeting the learning outcomes?
In other words, does inclusive assessment demand learning outcomes that can be met without the use of AI? I think this should be the case as resistance and refusal are defendable positions within the framework of most universities stated values.
From draft principle 5: All assessments have a clear and justifiable design rationale for their position on AI use.
“Because students may have the opportunity to use AI regardless of whether it is permitted, where the validity of an assessment depends on AI not being used, this is secured through the conditions of assessment rather than by instruction alone.”
From draft principle 6: Prohibition and detection are not reliable or effective approaches
“Given this evolving and contested evidence base, providers do not rely on detection software as a primary safeguard; the more durable response is robust, valid assessment design, combined with meaningful assessment security actions, rather than after-the-fact policing.”
From draft principle 7: Assessment security is supported by realistic, meaningful and transparent academic regulations and accessible policy and regulation, not by academic integrity alone
“It is insufficient, and unfair, to set rules that providers have no practical means of verifying – for instance, distinctions between permitted and prohibited AI use that cannot be evidenced, checked, or distinguished after the fact – and then rely solely on students’ individual academic integrity to ensure compliance. This approach transfers all responsibility for assessment security onto students, and risks penalising honest students who gain no advantage from AI use while others who breach the rules go undetected.”
I agree that AI detection software is not effective. I also agree that our response to AI should be to improve the quality of our teaching and assessment rather than find ever more detailed, and yet vague, methods of policing ‘after the fact’. This is an approach we have taken at UAL from day one and our guidance has been developed on that basis.
The problem here is if there is no way of detecting AI use then what is a valid ‘assessment security action’, surely it can’t be ‘make everyone sit a paper-based exam’? Also, if we shouldn’t be policing ‘after the fact’ then how are we avoiding permitted or prohibited AI use that ‘cannot be evidenced, checked, or distinguished after the fact’? This latter point somewhat invalidates the traffic light approach to permitted AI use in assessment as anything in ‘amber’ is practically impossible to verify/check (This is an area we are currently in discussion about at UAL).
Additionally, it’s indicated that we shouldn’t only rely on students’ academic integrity as this is unfair on the students that have academic integrity. This nets out as ‘don’t trust the students because this is unfair on the students you can trust’, which leaves us at: ‘the fairest approach is to not trust students or technology’. It’s difficult to know how to respond to this in day-to-day teaching and assessment practice.
A way out of this knot
Rather than get tangled up in the self-defeating nature of this I turn to the excellent ‘Trained to stop learning’ report form WonkHE. The report highlights the effectiveness of ‘points of accountability’, a pragmatic description of what the QAA principles refer to as ‘meaningful points’ . These are exactly what they sound like, points where students are required to explain their work. The WonkHE report suggests this can work well peer-to-peer as well as peer-to-tutor. A few moments of accountability (or what I’d describe as ‘Designed Friction’) reduce the temptation to overuse AI as it quickly becomes clear where any cognitive offsetting has eroded an individual’s ability to ‘speak to the work’.
However, asking every student to account for their learning through dialogue will disadvantage some and, as ever, offering a choice of how to make that account is good practice. There is a long history of accounting-through-dialogue at UAL, the arts crit being the classic form. There is also an acknowledgement that the practice can be distressing if not undertaken carefully and plenty of examples of practice which responds to this. For example, work on compassionate crits and other supportive forms of communication.
A difficult principle
From draft principle 8: Staff are supported in valid assessment design through ongoing professional development
“Teaching staff, and staff who support assessment design, are supported to develop and maintain competence in developing and delivering discipline-specific approaches to assessment that secure academic standards…”
This principle goes on to acknowledge that this is an ‘evolving area’ so not to rely on ‘one-off or definitive training’. We have a healthy staff AI Network at UAL which is full of lively discussion and practice sharing. I’ve also been running an UAL AI Principles workshop which has had around 300 attendees over six sessions. However, it’s often the case that our most anxious and/or busy staff won’t connect with this kind of support and it’s hard to sell the idea of staff training which might go out-of-date almost immediately.
Having said this, even over the last six months discussion at the Principles workshops has shifted from broad concerns to pragmatic responses. The overall response being cautious engagement combined with an acknowledgement of resistance and refusal as legitimate positions.
The enhancement principles
There are three of these in the QAA draft. For now, I’ll just highlight an extract from the third one as this is suggesting something I’ve been advocating for:
“Overloaded portfolios and an imbalance toward summative assessment drive up both student and marker burden while narrowing what is actually being tested…This is a matter of purpose rather than volume…”
What we know only too well at UAL is that our assessment practices are time-consuming. We also know that any student who feels like they ‘have to get through the work’ is more likely to lean on AI. Add to this that the more flexible the offer is the more assessment points are required to award small chunks of credit. This can rapidly spirals up to an enormous and ever-changing amount of work for everyone involved.
‘Less but better’
I worked with Ian Truelove and colleagues in UAL Online to developed an assessment framework for the group which asked for a much smaller volume of work from students. The focus was ‘less but better’, and it appears to be working well.
Most discussion around AI and academic standards points to the downsides of our habit of upping the wordcount as students go up through the levels. Academic rigour is not proportional to the volume of output. I would argue that the emergence of AI strengthens the opposite view.
Learning as becoming
When we consider the teaching and assessment practices which respond positively to the risk AI poses to the learning process, it all tends towards a learning as becoming model of education. Assessment becomes about evidencing a change in the person; evidence that the learning has become part of who they are rather than an object located in a nearby computer/phone. This has always been central to Arts Education and it’s fascinating to see institutions with a predominantly STEM focus lean into it. For example, this from the MIT report which could easily have been published by UAL:
“If learning is (mis)understood primarily as content delivery, then highly responsive AI systems may seem capable of replacing teachers, mentors, peers, collaborators, and the interpersonal dimensions of learning altogether. But while this attitude may be increasingly common, it’s shortsighted. Education is a cultural practice through which students learn to make meaning, exercise judgment, form identities, and participate responsibly in community.”
Now there is a machine which can analyse and produce most forms of content we are challenged to reassert an education which goes beyond that. Ironically, universities have always made claims around meaning making and identity formation (becoming) but narrow skills agendas, massification and academic career pathways which favour research over teaching have enticed many an institution to shift the day-to-day experience of learning towards content consumption and reproduction. We are in no way immune to those pressures at UAL but our focus on practice-based education and subjectivity means our approach has retained a focus on becoming and experience which AI has made increasingly pertinent beyond Arts Education.
Perhaps the single most positive thing about AI is the way it is forcing institutions to develop a form of education they claimed to be offering all along.