Quantifying changing understanding: Level 3 evaluation of leadership development interventions
| Author | Eric Stevenson,James Warn |
| Published date | 01 August 2003 |
| Date | 01 August 2003 |
| DOI | http://doi.org/10.1177/1035719X0300300109 |
| Published By | Sage Publications, Inc. |
| Subject Matter | Methodology Update |
56 Evaluation Journal of Australasia, Vol. 3 (new series), No. 1, August 2003
Quantifying changing understanding:
Level 3 evaluation of leadership
development interventions
Effective evaluation provides a vehicle for ensuring that organisations
deliver the training that enables managers to grow and gain mastery in
their various roles. However, evaluating the relevance of training for
developing generic skills such as leadership presents an enduring
difficulty for HR and training specialists. This is because it is relatively
easy to obtain feedback on the participant’s reaction to training, but
more difficult to determine whether they have improved their
understanding of leadership, or whether there is any subsequent benefit
to the organisation as a whole.
Trainers might typically evaluate their courses by eliciting
participant responses to course content and processes, and perhaps by
examining participant retention of specific skills and strategies. These
approaches are examples of Level 1 and 2 evaluation (Kirkpatrick
1998). Measuring individual development (Level 3) and organisational
change (Level 4) are also necessary to gain a more complete measure of
the effectiveness of a course, but they are more difficult. A number of
disparate, intervening variables not associated with the course may
account for an increase in participants’ performance, and isolating these
variables presents a challenge. This is particularly true when allowing
for the impact of experience in developing leaders.
The overall purpose of our research was to measure the extent to
which a participant’s ‘mental model’ of leadership changes as a result of
leadership development interventions such as education, training and
experience. Argyris and Schön (1974) define a mental model as the
individual’s personal ‘theory in use’ about the world that underpins
their behaviour. They may also be called ‘cognitive maps’, ‘schemas’ or
‘mental constructs’.
The mental model determines the selection, interpretation,
simplification and integration of information from the surrounding
world. The individual relies on the mental model to make sense of the
world and to formulate plans for influencing outcomes. Through
experiences, a manager develops a personal mental model of leadership
and it is this cognitive understanding that determines the observable
leadership behaviour. We propose that a training course needs to
enhance the leader’s mental model of leadership if it is to improve his or
her leadership performance. The purpose of this paper is to describe an
aspect of our research that entails an innovative method for quantifying
Level 3 changes in individuals undergoing leadership development.
Eric Stevenson
James Warn
Eric Stevenson is Assistant Director,
Strategic Education and Training in
the Australian Department of
Defence.1
James Warn lectures at
UNSW@ADFA in the fields of
Leadership and Organisational
Behaviour.2
The need for effective evaluation
METHODOLOGY UPDATE
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting