Human intelligence versus artificial intelligence in classifying economics research articles: exploratory evidence
| Date | 16 December 2024 |
| Pages | 18-30 |
| DOI | https://doi.org/10.1108/JD-05-2024-0104 |
| Published date | 16 December 2024 |
| Author | Jussi T.S. Heikkilä |
Human intelligence versus artificial
intelligence in classifying economics
research articles: exploratory evidence
Jussi T.S. Heikkil€
a
LUT University, Lahti Campus, Lahti, Finland and
Jyv€
askyl€
aUniversity School of Business and Economics, Jyv€
askyl€
a, Finland
Abstract
Purpose –Wecompare human intelligenceto artificial intelligence (AI) in the choice of appropriate Journal of
Economic Literature (JEL) codes for research papers in economics.
Design/methodology/approach –We compare the JEL code choices related to articles published in the recent
issues of the Journal of Economic Literature and the American Economic Review and compare these to the original
JEL code choices of the authors in earlier workingpaper versionsandJEL codes recommended by various generative
AI systems (OpenAI’s ChatGPT,Microsoft’s Copilot, Google’s Gemini) based on the abstracts of the articles.
Findings –There are significant discrepancies and often limited overlap between authors’ choices of JEL codes,
editors’ choices as well as the choices by contemporary widely used AI systems. However, the observations
suggest that generative AI can augment human intelligence in the micro-task of choosing the JEL codes and,
thus, save researchers time.
Research limitations/implications –Rapid development of AI systems makes the findings quickly obsolete.
Practical implications –AI systems may economize on classification costs and (semi-)automate the choice of
JEL codes by recommending the most appropriate ones. Future studies may apply the presented approach to
analyze whether the JEL code choices between authors, editors and AI systems converge and become more
consistent as humans increasingly interact with AI systems.
Originality/value –We assume that the choice of JEL codes is a micro-task in which boundedly rational
decision-makers rather satisfice than optimize. This exploratory experiment is among the first to compare
human intelligence and generative AI in choosing and justifying the choice of optimal JEL codes.
Keywords JEL codes, Artificial intelligence, Large language models, Search costs, Bounded rationality
Paper type Research paper
Introduction
The Journal of Economic Literature (JEL) classification codes system maintained and
published by the American Economic Association is the de facto standard for classifying
research papers in economics (Cherrier, 2017;Heikkil€
a, 2021,2022;Bornmann and
Wohlrabe,2024). Data on JEL classification codes has been utilized to analyze theevolution of
published economics papers by fields and styles (e.g. Card and DellaVigna, 2013;Angrist
et al., 2017;Bornmann and Wohlrabe,2024). Kosnik (2018) has documented that there canbe
differences in the author-assigned and editor-assigned JEL codes (in the American Economic
Review journal) and Heikkil€
a(2022) further illustrated that JEL codes of economics working
papers can differ from those of the final peer-reviewed and published articles. Concurrently,
the micro task of classifying economics research papers has become increasingly complex. For
instance, Bornmann and Wohlrabe (2024) report that the average number of JEL codes per
paper has increased steadily from about 1.9 in 1991 to 4.3 in 2021.
Using “human intelligence” may lead to subjective and boundedly rational choices
(Artinger et al., 2022) and different researchers choose different JEL codes that in their
JD
81,7
18
© Jussi T.S. Heikkil€
a. Published by Emerald Publishing Limited. This article is published under the
Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and
create derivative works of this article (for both commercial and non-commercial purposes), subject to full
attribution to the original publication and authors. The full terms of this licence may be seen at http://
creativecommons.org/licences/by/4.0/legalcode
I thank two anonymous reviewers for their helpful comments. Financial support from the P€
aij€
at-H€
ame
Regional Fund of the Finnish Cultural Foundation is gratefully acknowledged.
The current issue and full text archive of this journal is available on Emerald Insight at:
https://www.emerald.com/insight/0022-0418.htm
Received 8 May 2024
Revised 20 October 2024
Accepted 24 October 2024
Journalof Documentation
Vol.81 No. 7, 2025
pp.18-30
EmeraldPublishing Limited
e-ISSN:1758-7379
p-ISSN:0022-0418
DOI10.1108/JD-05-2024-0104
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeUnlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations