Teaching and testing in the era of text-generative AI: exploring the needs of students and teachers
| Date | 02 July 2024 |
| Pages | 149-169 |
| DOI | https://doi.org/10.1108/ILS-10-2023-0165 |
| Published date | 02 July 2024 |
| Author | Julia Jochim,Vera Kristina Lenz-Kesekamp |
Teaching and testing in the era of
text-generative AI: exploring the
needs of students and teachers
Julia Jochim
Department of Digital Media, Europäische Fernhochschule Hamburg,
Hamburg, Germany, and
Vera Kristina Lenz-Kesekamp
Department of Business, Digitalisation and Management,
Europäische Fernhochschule Hamburg, Hamburg, Germany
Abstract
Purpose –Large language models such as ChatGPT are a challenge to academic principles, calling into
question well-established practices, teaching and exam formats. This study aims to explore the adaptation
process regarding text-generativeartificial intelligence (AI) of students andteachers in higher education and
to identifyneeds for change.
Design/methodology/approach –The issue is explored in a mixed-methods approach based on
Domestication Theory (Silverstoneet al., 1992; Silverstone, 1994), incorporating views of both teachingstaff
and students.Both statistical and content analyses were carried out.
Findings –The results show that both students and teachers are conflicted about generative AI and its
usage. Trepidationand fear stand against a general feeling that AI is an integral part of the future and needs
to be embraced. Both groupsshow marked needs for training and rules and offer a varietyof ideas for new
exam formats.
Originality/value –This study provides a unique insightby exploring the attitudes and usage intentions
regarding generativeAI of two stakeholder groups: studentsand teachers. Its results can be of significant use
to institutions decidingon their strategy regarding AI. It illustrates attitudes and usage intentions as wellas
needs of bothgroups. In addition, ideas for new assessmentand teaching formats were generated.
Keywords AI, Generative artificial intelligence, GenAI, ChatGPT, Large language model,
Higher education, Exams, Domestication Theory
Paper type Research paper
Introduction and motivation
Text-generative artificial intelligence (GenAI) has the potential to bring about a major
transformation in higher education,especially as it calls into question established academic
practice and unsupervised exam formats such as papers and theses, because students can
delegate at least parts of the writingwork to chatbots. Innovative solutions are required that
take advantage of artificial GenAI while ensuring a fair and transparent examination
system (Buck and Limburg, 2023;Gimpelet al.,2023;Grassini, 2023;Hodges and Kirschner,
2024;Michel-Villarrealet al., 2023;Sullivan et al.,2023;Yu,2023). A variety of AI-robust and
AI-integrative examinationformats are currently being discussed (Buck and Limburg, 2023;
Cotton et al.,2024;Gimpel et al., 2023;Hodges and Kirschner, 2024;Huang, 2023;Lo, 2023;
Rasul et al.,2023;Sullivan et al.,2023). Furthermore, the question arises as to what
competencies future graduates will require, as the use of AI can be expected to be an
AI, teaching
and testing
149
Received31 October 2023
Revised22 April 2024
Accepted30 April 2024
Informationand Learning
Sciences
Vol.126 No. 1/2, 2025
pp. 149-169
© Emerald Publishing Limited
2398-5348
DOI 10.1108/ILS-10-2023-0165
The current issue and full text archive of this journal is available on Emerald Insight at:
https://www.emerald.com/insight/2398-5348.htm
important skill in most vocations in the future (Braun,2023;Buck and Limburg, 2023;Ehlers
et al.,2024;Eloundou et al.,2023;Evans et al.,2023;Rasul et al., 2023;Zarifhonarvar, 2023).
In addition, the question of how GenAI can and should feature in teaching and learning
needs to be explored (Southworth et al., 2023), as well as the question how students can be
motivated in future to learn skills that GenAI can do for them (Baron, 2023;Gimpel et al.,
2023;Ng et al.,2023;Reinmann, 2023).
Institutions of higher educationare under pressure to adapt to the new technology. This
study aims to contribute to the evolution of strategies and concepts in this context by
examining the main stakeholdergroups together: teachers and students.
Previous research
The advent of GenAI has instigateda broad range of research activities.In higher education,
a significant number of the research projects undertaken focus on the possibility of
academic fraud by students. The ability of ChatGPT to pass exams has been investigated
for several academic fields, such as IT (Finnie-Ansley et al., 2022;Malinka et al.,2023),
medicine (Kung et al.,2023) and law (Choi et al.,2023), showing that a passing grade is
possible using ChatGPT. Susnjak (2022), in a paper boldly entitled “ChatGPT: The End of
Online Exam Integrity?”, described research over a broad range of academic disciplines,
finding that the AI displays remarkable levels of critical thinking when asked to answer
exam questions, and concludingthat students could potentially use the chatbot for cheating.
The author suggests supervised exams as well as detection tools, but also developing
further strategiesto combat fraud.
Other authors, such as Gimpel et al. (2023) and Baume et al. (2024) have addressed the
issue from another angle, arguing for changing exam formats by incorporating GenAI and
developing new ways of assessing and testing. This approach ties in with another part of
the academic discourse, which has been focusing on the question of how teaching and
didactics should evolve in the face of the disruption. The question of required future skill
sets in the job market as well as in the academicfield is being discussed, which in turn leads
to the question of how GenAI needs to be integrated into teaching and learning (e.g.
Friedrich, 2023;Gimpel et al., 2023;Grassini, 2023;Rasul et al., 2023;Reinmann, 2023;
Southworth et al.,2023;Weßels et al.,2022). Several authors have been discussing the ways
and concepts of adapting teachingand assessments to the new technology by embracing it,
instead of trying to fend itoff, rethinking teaching practices and established roles(e.g. Buck
and Limburg, 2023;Hodges and Kirschner, 2024;Rasul et al., 2023;Reinmann, 2023). The
question of ethics and academic integrity constitutes another field of discussion within the
current climate (e.g. Bin-Nashwan et al.,2023;Crawford et al.,2023;Meyer et al.,2023;
Reinmann, 2023;Sullivanet al., 2023).
Theoretical approach and research question
This study is based on Domestication Theory, developed by Silverstone et al. (1992) as a
basis for studying interactions between people and new media technologies, especially
regarding the process of their adoptionand usage (Harwood, 2011, p. 84). The theory breaks
down the integration and usage of new technologies into four phases, with the aim of
making the transformation process more easily interpretable and to analyse its effects on
behaviour, socialinteractions and social structures (Harwood, 2011,p.87).
The first stage, a phase of familiarisation, is called Appropriation: people gather
information, experiment with the new tool(s) and develop personal preferences. This is
followed by Objectification: the new technology is integrated into everyday life and
becomes normal instead of something extraordinary. The next stage, Incorporation,
ILS
126,1/2
150
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting