Citizen perceptions of road smoothness: evidence from New York with implications for comparative performance measurement
| Author | Gregg G. Van Ryzin |
| Published date | 01 December 2008 |
| Date | 01 December 2008 |
| DOI | http://doi.org/10.1177/0020852308098469 |
| Subject Matter | Articles |
Review of
Administrative
Sciences
Citizen perceptions of road smoothness: evidence from New
York with implications for comparative performance
measurement
Gregg G. Van Ryzin
Abstract
Citizen surveys have many advantages for comparative performance measure-
ment, particularly across cities, regions or countries that often employ quite
different performance indicators and reporting systems. But much debate and
skepticism exists about the validity and therefore meaning of subjective ratings of
government performance. A recent study of street cleanliness in New York, how-
ever, found that citizen perceptions do strongly correlate with objective ratings by
trained observers. The present study uses the same New York survey data and
analytical approach to test the validity of citizen perceptions of another basic city
service, the condition of road surfaces, in comparison to a sophisticated, objective
measure of road smoothness conducted by the Fund for the City of New York. In
contrast to the street cleanliness findings, the present study finds almost no corre-
lation at all between objective and subjective measures of road smoothness.
These results suggest that the validity of citizen surveys depends a great deal on
the service or condition being measured. More empirical research on the specific
aspects of government performance that citizens can, or cannot, judge well is
needed.
Points for practitioners
Practitioners often rely on citizen surveys to measure service quality, but they
remain uncertain if survey results tell them much about actual government per-
formance. Do citizens perceive government performance as it really is? Or do they
view it through a largely subjective lens? Interestingly, it may be a bit of both. This
Gregg G. Van Ryzin is an associate professor at the School of Public Affairs and Administration at
Rutgers State University of New Jersey, USA.
Copyright © 2008 IIAS, SAGE Publications (Los Angeles, London, New Delhi and Singapore)
Vol 74(4):575–588 [DOI:10.1177/0020852308098469]
576 International Review of Administrative Sciences 74(4)
article builds on a prior study that showed how citizens of New York can be quite
good judges of the cleanliness of city streets. But using the same data and
method, this new study finds that citizens do a decidedly worse job at judging the
smoothness of road surfaces. Some guidance is offered about which kinds of
services citizens can judge best and how to interpret various service quality ratings
captured in a citizen survey.
Keywords: Citizen surveys, comparative performance measurement, service
quality, local government
Introduction and background
Sample surveys of citizens provide a systematic and economical way of measuring
government performance (or at least perceptions of performance) across public
sectors. Modern survey research methods are well established and scientific, and
respondents can be asked standardized questions about a uniform series of public
goods and services across locations, facilitating direct comparison of cities, regions,
and countries. As a result, there is growing interest in citizen surveys as a method-
ology for the study of public administration and governance (Bouckaert et al., 2005).
Moreover, often objective or ‘hard’ measures of government performance remain
unsuitable for international or even inter-jurisdictional comparisons within a country,
due largely to the fact that methods of performance measurement, accounting, and
reporting can be quite unique to each local government. Indeed, most of the inter-
national indicators of government performance — or bureaumetrics as they have
been called — must rely to a large extent on survey data rather than objective meas-
ures of performance (Bouckaert and Van de Walle, 2003; Van de Walle, 2006).
However, a great deal of skepticism exists in the field of public administration
regarding the validity of survey-based, subjective measures of government perform-
ance. ‘A potential danger of this measurement by subjective proxy’, warns Van de
Walle (2006: 446–7) in his review of international bureaumetrics, ‘is that the indica-
tors merely return the popular image of the administration, rather than actual func-
tioning.’ In fact, this points to a long-standing criticism of citizen surveys more
generally: that the results predominantly reflect the characteristics of the respondents
themselves — such as socioeconomic status, race, gender, and general attitudes
toward government — rather than actual service quality or government performance
(DeHoog et al., 1990; Stipak, 1979). There are various empirical studies that support
elements of this hypothesis to one degree or another (Brown and Coulter, 1983;
Durand, 1976; Hero and Durand, 1985; Lyons et al., 1992; Stipak, 1979; Van Ryzin et
al., 2004). In addition, the published empirical studies that explicitly test the corre-
spondence between subjective and objective measures of performance tend to find
little correlation (Brown and Coulter, 1983; Kelly and Swindell, 2002a,b).
But some recent empirical evidence has begun to emerge in favor of the corre-
spondence between subjective and objective measures of government performance.
For example, a study by Licari et al. (2005) found moderately strong correlations
between citizen ratings of streets and parks and the visual assessments of trained
observers in 99 small Iowa towns. The bivariate correlations between citizen satisfac-
Van Ryzin Citizen perceptions of road smoothness 577
tion and visual assessments were r = .49 for streets and r = .34 for parks. They con-
clude that these correlations ‘support the view that citizen assessments of service
quality can be valid’ (p. 365), at least in the sense of corresponding to the ratings of
a neutral, outside observer.
A recent empirical study of street cleanliness in New York, which the present study
directly builds on, demonstrates an even stronger correlation between citizen per-
ceptions and objective ratings of street cleanliness by trained observers (Van Ryzin et
al., 2008). The study compared findings from a large telephone survey of New York
citizens to the city’s official street cleanliness scorecard, a system of performance
measurement that employs trained observers and photographic standards to pro-
duce ratings for each of the city’s 59 districts. The New York street cleanliness score-
card is often considered a model of true outcome measurement in municipal
government administration (Hatry, 1999). The correlations between citizen percep-
tions and the street cleanliness scorecard turned out to be quite strong, r = .73 in both
2000 and in 2001 (the years for which survey data were available). Further analysis
of the individual variation in citizen ratings of street cleanliness, using multiple regres-
sion analysis, found that the street cleanliness scorecard emerged as the strongest
predictor — stronger than both a set of demographic factors (including age, gender,
race-ethnicity, education, and income) and a measure of overall trust of government.
These results provide clear evidence that citizen perceptions of street cleanliness in
New York do, indeed, reflect quite closely the city’s official scorecard measurement of
street cleanliness rather than personal characteristics or stereotypes of government.
The present study extends this work by employing the same New York telephone
survey data and analytical approach to test the validity of citizen perceptions of
another basic public service: the smoothness of road surfaces. The objective measure
for purposes of comparison is a sophisticated, scientific measurement of road
smoothness conducted by the Fund for the City of New York. As in the prior study
(Van Ryzin et al., 2008), the analysis is conducted both at...
To continue reading
Request your trial