The wheelbarrow, the mosaic and the double helix
| Author | Lois-Ellin Datta |
| DOI | 10.1177/1035719X0100100210 |
| Published date | 01 December 2001 |
| Date | 01 December 2001 |
| Subject Matter | Mixed Methods Evaluation |
33
Lois-ellin Datta is a past President
of the Evaluation Research Society
(now the American Evaluation
Association), and is President of
Datta Analysis.
Lois-ellin Datta1
The wheelbarrow, the
mosaic and the double helix
mixed methods evaluation
Introduction
What is the best way to learn whether a program phasing out direct income supports,
while phasing in limited eligibility, employment transition, child care, health, housing
and food benefits, works? The discussions at the July 2000 forum on impact
evaluation held in Wellington, New Zealand, consistently indicated that there was no
one best method to which one would turn first, the others being defaults. The golden
rule, in Burt Perrin’s wonderful phrase, is that there is no golden rule in evaluation
methods.
Rather, the discussions emphasised that many approaches can be of great use,
depending on factors such as the policy context, including key questions as seen by
relevant stakeholders, program maturity, data availability and the desiderata to be
maximised (conference summary prepared by the New Zealand Government,
undated). It followed that scoping the evaluation could benefit from a multi-
disciplinary team, from front-end listening to diverse stakeholder groups, and from
early clarity on the evaluation desiderata to be maximized.
Some methods would shine against certain of the desiderata but have notable
limitations for others. Randomised designs, such as Boruch (1997) presents, would be
appropriate in some situations; quasi-experimental designs, such as those described
by Lipsey and Cordray (2000), in others; theory-based and intermediate variable
designs, such as Pawson and Tilley (1997) and Henry, Julnes and Mark (1998, 1999)
discuss, in still others. As a reminder, Table l illustrates some of these desiderata.
Many evaluations involve multiplism. The discussions also emphasised that in
many instances mixed methods would be required. This point is underscored in a
forthcoming discussion on evaluations of welfare reform by Julnes and Foster (2001).
They argue for using multiple methods to provide a more comprehensive view of the
outcomes experienced by welfare leavers, citing Cook’s (1985) explication of critical
multiplism. As seen by Cook, critical multiplism could involve:
■multiple value stances;
■multiple program theories;
■multiple operationalisation of constructs;
■multiple methodology paradigms;
■multiple professional affiliations of investigators;
■multiple contexts for inquiry.
This is hardly a tough argument to make. Rather, it seems fair to say that much
current evaluation practice involves, routinely and de minimis, multiple types of data.
Challenges and strategies for successfully carrying
out mixed methods evaluation
Datta – Mixed methods evaluation
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting