Meta-analysis as a method of multi-site evaluation: An example from international development
| Author | Chris Coryn,Robert McCowen,Kristin Hobson |
| DOI | 10.1177/1035719X1501500302 |
| Published date | 01 September 2015 |
| Date | 01 September 2015 |
| Subject Matter | Refereed Article |
4
Evaluation Journal of Australasia Vol 15 | No 3 | 2015
REFEREED ARTICLE Evaluation Journal of Australasia Vol 15 | No 3 | 2015 | pp. 4–14
CHRIS CORYN | KRISTIN HOBSON | ROBERT MCCOWEN
Meta-analysis as a method
of multi-site evaluation
An example from international development
Interest in multi-site evaluation has seemingly grown, with a now nascent body of literature.
Even so, most of the literature on multi-site evaluation has come from writings about place-,
group-, and cluster-randomized controlled trials. Guidance on how to execute uncontrolled
multi-site evaluations is sparse and usually treats sites as homogeneous. In this paper, meta-
analysis is used to demonstrate the method’s potential for multi-site evaluation using Heifer
International sites in Albania, Nepal, and Uganda as an exemplar, with an emphasis on
nutritional outcomes for site participants.
Introduction
Although randomized controlled trials (RCTs) and
its various synonyms (e.g. randomized clinical trials,
randomized experiments), are widely acknowledged as
the methodological standard for establishing legitimate
causal claims and constructing unbiased estimates of the
eects of applied interventions (Cook et al. 2010; Coryn
& Hobson 2011; Shadish, Cook & Campbell 2002),
such designs are frequently infeasible (Bamberger, Rugh
& Mabry 2012; Stuebeam & Coryn 2014). However,
causal inferences with reasonable validity can often be
warranted through means other than RCTs. In fact, recent
studies and simulations have demonstrated that other
designs (e.g. regression discontinuity) are often capable
of producing unbiased causal estimates equivalent to
those obtained from RCTs (Cook, Shadish & Wong 2008;
Scriven 2012; Shadish et al. 2011).
Evaluation in international development contexts
International development evaluations are ideally
conducted in order to determine the merit, worth,
significance of and/or improvement of social and
economic projects and programs in developing countries
(Bamberger 2000; Patton 2011; Picciotto 2003; Scriven
1991). Historically however, self-assessments and the
monitoring of program and project inputs and outputs
have constituted the large majority of international
development evaluation eorts (Valadez & Bamberger
1994). Such evaluations are often dominated by: (1) the
influence on evaluation methodology by donor-sponsors;
(2) simple measures of program and project inputs and
outputs (rather than long-term outcomes and impacts);
(3) an increasing reliance on participatory evaluation
(Cullen & Coryn 2011; Cullen, Coryn & Rugh 2011)
as a means of giving a voice to program and project
participants—ultimately overlooking other methods that
might also address important evaluation questions; and
(4) the use of existing (often inadequate) baseline data
that hampers the assessment of social and economic
impacts of programs and projects. Equally important,
Bamberger notes:
The results of many of these eorts have been
disappointing, due in part to the pressures on
international development agencies to respond to the
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting