Evaluation in the ‘new’ knowledge age
| Author | Michael Quinn Patton |
| DOI | 10.1177/1035719X0100100208 |
| Published date | 01 December 2001 |
| Date | 01 December 2001 |
| Subject Matter | Opinion |
30 Evaluation Journal of Australasia, Vol. 1 (new series), No. 2, December 2001
Evaluation in the ‘new’
knowledge age
OPINION
Michael Quinn Patton
Evaluators are being swept up in the endlessly hyped
knowledge age of the new millennium. We’re being pressured to
generate lessons learned and identify ‘best practices.’ To do so with
authenticity, we’re going to need to bring some rigor to these powerful
sound-bite notions of ‘lessons learned’ and ‘best practices’.
The great lesson learned in the last decade of the last millennium
was that information is not the same as knowledge. (Wow! Who
knew?) In multinational corporations, the information age has given
way to the knowledge-hungry age. Chief Information Officers, all the
rage in the 1990s, have been replaced by Chief Knowledge Officers.
And what do Chief Knowledge Officers do? They capture lessons
learned and identify best practices.
Best practices have become the most sought after form of
knowledge. Not just effective practices, or decent practices, or better
practices – but best. It’s the American way. Be the best you can be.
How? Learn lessons (personal, local knowledge) and convert them to
best practices (universal knowledge).
The federal government publishes best practices for education,
health, highways, and welfare reform. Philanthropic foundations are
anxious to discover, fund, and disseminate best practices. Corporations
advertise that they follow best practices. Management consultants teach
best practices. One company of consultants, Best Practices, LLC, has
created a ‘Best Practice Database’ derived from ‘studying world-class
customer service practices [that] foster higher quality customer service
and satisfaction ...’ Best Practices BenchmarkingTM reports provide ‘fast
and effective access and intelligence to world-class excellence’
(www.best-in-class.com).
While the profession of evaluation will inevitably be affected by
concepts swirling in the larger political environment, we have an
obligation to examine those concepts with care and to educate users
about their deeper implications. For example, the assumptions
undergirding the phrase ‘best practices’ (e.g., that there must be a single
best way to do something) are highly suspect. In a world that values
diversity, many paths exist for reaching some destination; some may be
more difficult and some more
costly, but those are criteria that
take us beyond just getting there
and reveal the importance of
asking, ‘best’ from whose
perspective using what criteria?
From a systems point of
view, a major problem with many ‘best practices’ is the way they are
offered without attention to context. A lot of ‘best practices’ rhetoric
presumes context-free adoption. ‘Best practices’ that are highly
prescriptive and specific (e.g., ‘first graders should be read to by
teachers out loud at least fifteen minutes a day’ – to cite an example I
was shown by a teacher)
represent bad practice of best
practices. In contrast, ‘best
practices’ that are principles to
guide practice can be helpful. To
further illustrate (and be
provocative), I consider the
utilization-focussed mantra that
evaluations should be focussed
on ‘intended use by intended
users’ an evaluation ‘best
practice’ at the principle level.
However, identifying specific
intended uses with specific
intended users can only be
undertaken in a specific context
and situation. So, what’s one of
the most common questions I get
after presentations on
Utilization-Focussed Evaluation:
How many intended users
should an evaluation have? My
response: As many as it takes to
support intended uses, no more,
and no fewer. And how many
intended uses can an evaluation
support? As many as it takes to
meet the needs of primary
intended users. Circular
reasoning is a wonderful
antidote to linear, mechanistic
thinking – which characterizes
much (but not all) ‘best
practices’ practice. Going in
circles at least keeps people from
going some place where they’ll
do harm. All in all, I prefer to
eschew the language of ‘best
practices’. Calling something
‘best’ is typically more a political
assertion than an empirical
conclusion.
Going in circles at least keeps
people from going some place
where they’ll do harm.
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting