When “Don’t Know” Indicates Nonignorable Missingness: Using the Estimation of Political Knowledge as an Example
| Author | Tsung-Han Tsai |
| DOI | http://doi.org/10.1177/14789299211058543 |
| Published date | 01 February 2023 |
| Date | 01 February 2023 |
| Subject Matter | Articles |
https://doi.org/10.1177/14789299211058543
Political Studies Review
2023, Vol. 21(1) 99 –126
© The Author(s) 2022
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/14789299211058543
journals.sagepub.com/home/psrev
When “Don’t Know” Indicates
Nonignorable Missingness:
Using the Estimation of Political
Knowledge as an Example
Tsung-Han Tsai
Abstract
The conventional procedure for measuring political knowledge is treating nonresponses such
as “don’t know” as incorrect responses and counting the number of “correct” responses. In
recent times, increasing attention has been paid to partial knowledge hidden within incorrect and
nonresponses. This article explores partial knowledge indicated by incorrect and nonresponses
and considers nonresponses as nonignorable missingness. We propose a model that combines
the shared-parameter approach presented in the literature on missing data mechanisms and the
methods of item response theory. We show that the proposed model can determine whether
the people with nonresponses should be treated as more or less knowledgeable and detect
whether it is appropriate to pool nonresponses and incorrect responses into the same category.
Furthermore, we find partial knowledge hidden within women’s nonresponses, which confirms
the possibility of the exaggeration of the gender gap in political knowledge.
Keywords
political knowledge, nonignorable missingness, missing not at random, item response theory,
Bayesian methods
Accepted: 21 October 2021
Introduction
In survey research, researchers typically design a battery of questions to measure some
unobserved, latent variables such as democratic values, political efficacy, and political
knowledge. Owing to the limitations of questionnaire length, interviewing time, and
implementation cost, three to five items are commonly used to measure a defined
Department of Political Science, Election Study Center, Taiwan Institute for Governance and
Communication Research, National Chengchi University, Taipei, Taiwan
Corresponding author:
Tsung-Han Tsai, National Chengchi University, Taipei 11605.
Email: thtsai@nccu.edu.tw
1058543PSW0010.1177/14789299211058543Political Studies ReviewTsai
research-article2021
Article
100 Political Studies Review 21(1)
construct. To measure political knowledge, for example, four multiple-choice items are
included in the Comparative Study of Electoral Systems (CSES) Module 4 (2011–2016)
and three multiple-choice items are included in the American National Election Studies
(ANES) 2016 Pilot Study.1
The conventional procedure for measuring political knowledge is categorizing
responses into correct and incorrect and counting the number of correct responses.
Usually, nonresponses such as “don’t know” (DK) are considered incorrect. However, in
recent times, increasing attention has been paid to partial knowledge hidden within DK
(e.g. Luskin and Bullock, 2011; Miller and Orr, 2008) and incorrect responses (e.g.
Gibson and Caldeira, 2009). Unfortunately, there is no agreement on whether DKs con-
ceal partial knowledge and whether it is appropriate to pool DKs and incorrect responses
together as a single absence-of-knowledge category.
To fill this gap, we propose a model termed the shared-parameter latent variable
model (SPLVM), which combines the shared-parameter approach presented in the
literature on missing data mechanisms and latent variable models, to estimate indi-
vidual political knowledge. The basic idea of the shared-parameter approach is to link
the outcome and missing value process through common latent variables. This is
especially useful in measuring political knowledge because both whether the answer
is correct and whether a response is provided are determined by knowledge levels.
Unlike the conventional approach, we treat DK responses as missing values, and on
the basis of the framework of missing data mechanisms, we assume that DK responses
are missing not at random (MNAR) or not missing at random (NMAR; Little and
Rubin, 2002; Rubin, 1976). We then apply the methods of item response theory (IRT)
to estimate knowledge levels that govern the responses and determine whether they
are observed under the framework of the shared-parameter approach.
This article has two primary contributions. First, the SPLVM can determine
whether people with nonresponses should be treated as more or less knowledgeable,
depending on the characteristics of items. As indicated by the analysis of multiple-
choice items, when the curve of the response model is located on the left side of that
of the outcome model, respondents with missing responses are treated as less knowl-
edgeable than those with incorrect responses. By contrast, the analysis of open-
ended items shows that when the curve of the response model is located on the right
side of that of the outcome model, respondents with missing responses are treated as
more knowledgeable than those with incorrect responses. Second, we can explicitly
detect the appropriateness of grouping DK and incorrect responses into the same
category by using a test to determine differences between item parameters. If the
location parameters of response and outcome models are not different from each
other, then the procedure of pooling DK and incorrect responses together is
acceptable.
The remainder of this article proceeds as follows. Section “Political Knowledge within
DK and Incorrect Responses” reviews the extant literature on political knowledge meas-
urement and discusses the circumstances where partial knowledge may be concealed.
Section “The Statistical Model” introduces the proposed statistical model, and Section
“Data and Analysis” illustrates analyzed survey data and presents analysis results. Section
“Conclusion” concludes the article.
To continue reading
Request your trial