Measuring and monitoring outputs in social care: the problem of measuring quality
| Author | Juliette Malley,Ann Netten |
| DOI | 10.1177/0020852308098467 |
| Date | 01 December 2008 |
| Published date | 01 December 2008 |
| Subject Matter | Articles |
Review of
Administrative
Sciences
Measuring and monitoring outputs in social care: the problem
of measuring quality
Juliette Malley and Ann Netten
Abstract
The focus of this article is the measurement of quality within the context of a new
approach to measuring government-funded social care services. The approach
builds on principles set out in a UK review led by Professor Sir Tony Atkinson. At
its heart are the concepts of capacity for benefit, describing the potential benefit
to users from services, and quality, reflecting the extent to which the benefit is
realized. We report on a study designed to explore the suitability of a weighted
user satisfaction measure to measure quality, examining in particular its sensitivity
to changes in the quality of social care over time. The study finds some problems
with using weighted user satisfaction. We discuss potential reasons for these
problems, including the possibility of gaming and the specification of the measure,
and discuss a way forward in the context of the requirement to draw upon routine
sources of data.
Points for practitioners
The article describes a practical approach to measuring and monitoring outcomes
and productivity in social care. Routine measures are used as far as possible,
weighted to reflect outcomes for services users. The concepts of Capacity for
Benefit and quality in this context potentially allow commissioners and managers
to distinguish the impact on productivity of: the potential of services as they are
currently configured, the dependency of service users, and the quality of delivery.
The article identifies key issues and problems in using measures of satisfaction
Juliette Malley is a Research Officer at the Personal Social Services Research Unit at the University of
Kent at Canterbury and the London School of Economics and Political Science. Ann Netten is
Professor of Social Welfare and Director of the Personal Social Services Research Unit at the University
of Kent at Canterbury.
Copyright © 2008 IIAS, SAGE Publications (Los Angeles, London, New Delhi and Singapore)
Vol 74(4):535–553 [DOI:10.1177/0020852308098467]
536 International Review of Administrative Sciences 74(4)
both as performance indicators and monitoring quality of services in the proposed
approach.
Keywords: government outputs, output measurement, quality, social care
Background
There is increasing pressure to devise a means of reflecting changes in productivity
and efficiency across the public sector. In the UK the Office for National Statistics
(ONS) review of the future development of government output, productivity and
associated price indices led by Atkinson (2005) followed Eurostat guidance that
countries should be developing direct measures of government services that are
individually consumed. A fundamental principle of the review was that:
Government output should be measured in a way that is adjusted for quality, taking
account of the attributable incremental contribution of the service to the outcome.
(Atkinson, 2005: 42)
If we are to be able to draw conclusions about changing productivity, the basis of any
output measure needs to draw on routinely available sources, ideally on an annual
basis. Moreover, the measure needs to be valid, reliable, transparent and to reflect the
understanding of those involved in commissioning and delivering public services,
particularly those involved in the provision of routine data sources.
As part of the Atkinson review and with the longer-term objective of improving
measurement and understanding of social care output and productivity, the Depart-
ment of Health (DH) funded work to develop new measures of personal social
services (PSS) output and productivity, reflecting best available practice. An approach
was developed for this sector (Netten et al., 2006) that has been identified in a recent
ONS strategy document as having a wider applicability across government (Office for
National Statistics, 2007).
In this article we start by outlining what we mean by social care before describing
the overall framework for measuring the quality-weighted outputs of government
funded social care. We demonstrate the application of the approach to home care of
older people, focusing in particular on the validity and reliability of the proposed
measure of quality.
Social care
Social care generally occurs when we compensate for physical, mental or emotional
impairment or limitations of others, either through meeting needs directly or increas-
ing individuals’ abilities or human capital. Formal social care interventions that meet
needs traditionally include home care, which substitutes for tasks usually undertaken
by the individual such as personal care, and care homes, which replace almost all
activities usually undertaken by the individual and care network. Interventions that
increase human capital would include equipment that helps people to do things for
themselves and training to improve the skills of informal carers (Netten and Davies,
1990).
Malley and Netten Outputs and quality in social care 537
Framework
The aim is to develop an outcome-weighted index that reflects changes in what
services are providing, the characteristics of those they are serving and quality of pro-
vision.
We define three types of social care outcomes:
● Increased individual/care network productivity in caring for themselves
● Increased individual knowledge and information
● Meeting needs
Individuals’ increased productivity would be the expected output of services that
have the objective of improving individuals’ health, functional capacity and prevention
of deterioration. Such interventions are usually characterized by relatively short-term
one-off expenditures that have benefits that may well extend beyond the year of
expenditure. This would include equipment services and adaptations to property as
well as rehabilitative interventions. Increased knowledge and information will be the
expected output from advice and information services but may also be an important
output of the assessment process. Our initial work has focused on mainstream serv-
ice provision, reflecting the bulk of current PSS activity and expenditure: meeting
needs or helping people.
Our basic unit of output is number of people helped. The critical question is how
much have they been helped? This will depend on the amount of the service
received (e.g. number of weeks of care), people’s needs and what has been facili-
tated or delivered (such as personal care, meals and so on). Rather than try to attrib-
ute what services have delivered at an individual level we represent the outputs of an
intervention or service as:
Σ xiitCfBitQit
where:
xit
is the quantity of service i produced during period t
CfBit
is the Capacity for Benefit of people using service i during period t
Qit
is the quality of service i in period t
Xi reflects the volume of services, for example the total number of weeks of service
package i delivered during the year. Capacity for Benefit (CfBi represents the poten-
tial of the service to deliver welfare to service users given the domains of outcome
that the service affects and the degree to which users are reliant or dependent on
that service. Quality (Qi) represents the degree to which the service is delivering the
potential benefit.
In order to identify CfB we need to identify first the relevant domains of outcome
for a given intervention and then the degree to which people depend on services in
these domains. Based on earlier work focusing on older people (Netten et al., 2002)
and subsequent consultations (Netten et al., 2005) the domains have been defined as:
● Personal cleanliness and comfort: Being personally clean and comfortable,
dressed and in bed or up at preferred times.
538 International Review of Administrative Sciences 74(4)
● Social participation and involvement: Emotional support, general social contact
and community participation.
● Control over daily life: Able to choose what to do and when to do it, having
control over daily life and activities.
● Meals and nutrition: Having a nutritious, varied and culturally appropriate diet
with meals at regular, timely intervals.
● Safety: Feeling safe and secure, including feeling free from fear of abuse, falling
or other physical harm and fear of being attacked or robbed.
● Accommodation cleanliness and comfort: The living environment is clean and
comfortable.
● Occupation and employment: Sufficiently occupied in meaningful activities
whether it be formal employment, caring for others, unpaid work or leisure
activities.
● Living at home: Enabling people to live in their own homes.1
To identify the degree to which individuals rely on services, four levels of need are
identified reflecting: no need for assistance; all needs met; low-level needs; and high-
level needs. The distinction between ‘no need for assistance’ and ‘all needs met’ for
most domains will be the assistance of others. The distinction between high- and
low-level needs is whether there would be mental or physical health implications if
these needs continued to be unmet.
When we are attempting to measure the impact of services, it is important that we
do not assume that each of these domains is of equal importance. For example, we
might attach greater importance to meeting needs in one domain rather than
another. Ideally, in order to best reflect the welfare gain resulting from government
expenditure on social care, we want to weight the domains and levels within the
domains to reflect their relative value to people....
To continue reading
Request your trial