Assessing changes in thinking about troubleshooting in physical computing: a clinical interview protocol with failure artifacts scenarios
| Date | 06 January 2025 |
| Pages | 286-312 |
| DOI | https://doi.org/10.1108/ILS-06-2024-0075 |
| Published date | 06 January 2025 |
| Author | Luis Morales-Navarro,Deborah Fields,Yasmin B. Kafai,Deepali Barapatre |
Assessing changes in thinking about
troubleshooting in physical computing:
a clinical interview protocol with
failure artifacts scenarios
Luis Morales-Navarro
Learning Sciences and Technologies Program,University of Pennsylvania,
Philadelphia, Pennsylvania, USA
Deborah Fields
Department of Instructional Technology and Learning Sciences,
Utah State University, Logan, Utah, USA, and
Yasmin B. Kafai and Deepali Barapatre
Learning Sciences and Technologies Program,University of Pennsylvania,
Philadelphia, Pennsylvania, USA
Abstract
Purpose –The purpose of this paper is to examine how a clinical interview protocol with failure artifact
scenarios can capture changes in highschool students’explanations of troubleshooting processesin physical
computing activities. The authors focusonphysical computing, as finding and fixing hardwareand software
bugs is a highlycontextual practice that involves multiple interconnecteddomains and skills.
Design/methodology/approach –This paper developed and piloted a “failure artifact scenarios”clinical
interview protocol. Youthwere presented with buggy physical computing projects over video calls and asked for
suggestions on how to fix them without having accessto the actual project or its code. Authors applied this clinical
interview protocol before and after an eight-week-long physicalcomputing (more specifically, electronic textiles)
unit. They analyzed matching pre- and post-interviewsfrom 18 students at four different schools.
Findings –The findings demonstrate how the protocol can capture change in students’thinking about
troubleshooting by eliciting students’explanations of specificityof domain knowledge of problems, multimodality
of physical computing, iterative testing of failure artifact scenarios and concreteness of troubleshooting and
problem-solving processes.
Originality/value –Beyond tests and surveys used to assess debugging, which traditionally focuson
correctness or student beliefs, the “failure artifact scenarios”clinical interview protocol reveals student
troubleshooting-relatedthinking processes when encountering buggy projects. As an assessment tool, it may
be useful to evaluatethe change and development of students’abilities over time.
Keywords Debugging, Troubleshooting, Computer science education, Clinical interview,
Assessment, Electronic textiles, Physical computing
Paper type Research paper
With regards to Katherine Gregory for support in data analysis. This work was supported by a grant
from the National Science Foundation to Yasmin Kafai and Mike Eisenberg (#1742140/#1742081).
Any opinions, findings, conclusions or recommendations expressed in this paper are those of the
authors and do not necessarily reflect the views of NSF, the University of Pennsylvania or Utah State
University.
ILS
126,3/4
286
Received30 June2024
Revised2 S eptember2024
2 December 2024
Accepted4December2024
Informationand Learning
Sciences
Vol.126 No. 3/4, 2025
pp. 286-312
© Emerald Publishing Limited
2398-5348
DOI 10.1108/ILS-06-2024-0075
The current issue and full text archive of this journal is available on Emerald Insight at:
https://www.emerald.com/insight/2398-5348.htm
Introduction
Debugging, a key computingpractice (Grover and Pea, 2013;Lodi and Martini, 2021;Shute
et al.,2017), is the process of finding errors and fixing them in computing systems
(McCauley et al., 2008;Michaeliand Romeike, 2021). Much research has demonstrated that
students, especially programming novices, face multiple challenges when debugging code,
ranging from identifyingsimple syntax problems to more complex semantic problems when
programs run but do not function as intended (McCauley et al., 2008). These challenges are
even more evident in physical computing applications such as robots or electronic textiles
(e-textiles), where debugging needs to focus on both code and the physical hardware. This
means that beyond commonsyntactic and semantic issues in their programs, studentshave to
attend to electronic andother physical aspects of their designs (Desportes and Disalvo, 2019;
Lui et al., 2024). For this reason, debugging in physical computing is a highly contextual
practice, often also called by the broader term troubleshooting (Jonassen and Hung, 2006),a
term which we will use in the remainder of this paper when referring to identifying and
solving problems in physical computing. To better understand novices’learning to
troubleshoot physical computing applications, we need instruments to capture and measure
changes in how they engage with troubleshooting.
Yet there are few instruments available to assess changes in novice students’thinking
about troubleshooting in K-12 contexts.Assessing students’understanding is a challenge, as
troubleshooting involves specific domain knowledge (Vessey, 1985), systemic processes or
“global strategies”(Schaafstaland Schraagen, 2000) and emotional factors (DeLiema et al.,
2023). Some approaches to assessing troubleshooting focus on very limited, context-specific
domains, such as a designed bug embedded in code (Denny et al.,2020) or Parsons’
problems where learners rearrange pieces of code to fix a bug (Ericson et al., 2022).
Analytics and automated evaluation can often be applied to these types of assessments
because they have singular, “correct”solutions. Similar to this approach but with an eye
toward physical computing contexts, some surveys query students about domain-specific
components and processes needed in common machinery or makerspace projects (Blikstein
et al., 2017). In contrast, some have developed “common sense”scenarios that get at
everyday troubleshooting processes, for instance, how to help someone fix a light that does
not turn on (Simon et al., 2008) or escape rooms with carefully designedpuzzles (Michaeli
and Romeike, 2020). Complementing this list of more cognitive-focused assessments are
surveys that capture more social-emotional components such as grit, growth mindset and
learners’self-reported comfortlevels with various aspects of physical computing (Scott and
Ghinea, 2013;Morales-Navarro et al., 2023). Each of these assessments has affordances
related to time, thinking processes and practices within specific domains. In their review of
computational thinking assessments, Tang et al. (2020) outline several gaps, particularly in
assessing students’computational thinking in high school (upper secondary) or higher
grades, moving beyond just domain-specific programming skills for learning and surveys
about social-emotional dispositions, developing assessments that could be used in informal
settings and using interviews (such as think-aloud interviews) to capture more “complex
mental operations”thantraditional tests and surveys generally allow (p. 10).
In this paper, we report on the pilot of an exploratory instrument to appraise students’
troubleshooting thoughtprocesses in the form of a clinical interview protocol for high school
introductory computingstudents that focused on changes in students’thinking about “failure
artifact scenarios”specific to electronic textiles (a type of physical computing projects).
Electronic textiles (e-textiles hereafter) involve sewable, programmable circuits using
conductive thread to connect sensors and actuators to a microcontroller (Buechley et al.,
2013). We designed context-specific scenarios with problems in e-textile artifacts that were
Information and
Learning Sciences
287
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeUnlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations
Unlock full access with a free 7-day trial
Transform your legal research with vLex
-
Complete access to the largest collection of common law case law on one platform
-
Generate AI case summaries that instantly highlight key legal issues
-
Advanced search capabilities with precise filtering and sorting options
-
Comprehensive legal content with documents across 100+ jurisdictions
-
Trusted by 2 million professionals including top global firms
-
Access AI-Powered Research with Vincent AI: Natural language queries with verified citations