Interviewer-respondent interaction and rapport in PIAAC
| DOI | https://doi.org/10.1108/QAE-05-2017-0022 |
| Published date | 03 April 2018 |
| Date | 03 April 2018 |
| Pages | 182-195 |
| Author | Bryan Maddox |
| Subject Matter | Education,Curriculum, instruction & assessment,Educational evaluation/assessment |
Interviewer-respondent
interaction and rapport in PIAAC
Bryan Maddox
School of International Development, University of East Anglia,
Norwich, United Kingdom
Abstract
Purpose –This paper aims to investigate small-scale, qualitative observations of interviewer–respondent
interaction in the Organisation for Economic Co-operation and Development’s Programme for the
InternationalAssessment of Adult Competencies (PIAAC).
Design/methodology/approach –The paper uses video-ethnographic methods to document talk and
gesture in assessment in Slovenianhousehold settings. It presents an in-depth case study of interactionin a
single testingsituation.
Findings –Observing interaction in assessment captures data on assessment performance that is not
available in quantitativeanalysis of assessment responseprocesses. The character of interviewer–respondent
interaction and rapport is shaped by the cognitive demands of assessment and the distinctive ecological
setting of the household.
Research limitations/implications –Observational data on assessment response processes and
interaction in real-life assessments can be integrated into and synthesized with other sources of “process
data”.
Practical implications –Assessment programs such as PIAACshould consider the significance of the
household setting on assessment qualityand observations of interaction in assessment as a valid source of
paradata.
Social implications –There is a place for small-scale observational studies of assessment to inform
public understandingof assessment quality and validity.
Originality/value –The paper provides qualitative insights into the significance of interaction and
“interviewereffects”in household assessment settings.
Keywords Interaction, Assessment, Rapport, Households, Interviewer effects
Paper type Research paper
Introduction
This paper examines the significance of interviewer-respondent interaction in the
Organisation for Economic Co-operation and Development’s (OECD’s) Programme for the
International Assessment of Adult Competencies (PIAAC) and its relationship to data
quality. It presents a qualitativecase study that questions the role of this interactionon data
quality and considers its significance in the context of household-based assessment. The
literature on conversationanalysis promotes the view that interaction in the formof talk and
gesture in face-to-face social situations is integral to the way that social situations are
produced and performed (Goodwin and Duranti, 1992). The importance of interaction has
not gone unnoticed by researchers in educational assessment, who note the significance of
talk in producing interviewer and respondent roles and relations, and its implications
for assessment performance and consequences (McNamara, 1997;Shohamy, 1998;
McNamara and Roever, 2006;Maddox, 2015). This raises questions about the merit of
interaction in assessment, why it occurs and how it is treated in interviewer training and in
quality assurancepractices.
QAE
26,2
182
Received26 May 2017
Revised5 October 2017
Accepted14 December 2017
QualityAssurance in Education
Vol.26 No. 2, 2018
pp. 182-195
© Emerald Publishing Limited
0968-4883
DOI 10.1108/QAE-05-2017-0022
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0968-4883.htm
To continue reading
Request your trial