Evaluating as an Outsider or an Insider: A Two-Way Approach Guided by the Knowers of Culture
| Author | Bronwyn Rossingh,Yalmay Yunupingu |
| Published date | 01 September 2016 |
| Date | 01 September 2016 |
| DOI | http://doi.org/10.1177/1035719X1601600302 |
| Subject Matter | Refereed Article |
R E F E R E E D A R T I C L E Evaluation Journal of Australasia Vol 16 | No 3 | 2016 | p p . 5 – 1 4
BRONWYN ROSSINGH | YALMAY YUNUPINGU
Evaluating as an outsider or an insider:
a two-way approach guided by the
knowers of culture
Evaluating programs in remote Indigenous communities requires contextual consideration
and a degree of connection that goes beyond the usual focus on output measurement and
system monitoring. Evaluators who are experienced with working in remote communities
become more and more cognisant of the issues and complexities that impact the quality
of evaluations. As this reality presents itself, evaluators rely more and more on the help of
the local community to explain and reveal such realities. A learning journey takes place that
allows ‘outsiders’ to be accepted into the community and be part of a productive matrix of
support and ‘insiders’ begin to assume their legitimate rights to participate, advise, educate
and guide evaluators. Without a participative and collaborative styled process, evaluation
risks being non-reality based without a guarantee of quality and has the potential to impact
the community in a negative way.
This paper presents the views and experiences of ‘insiders’ and ‘outsiders’ who have been
involved in evaluations in remote communities. Their collective views demonstrate the
need for a two-way approach that establishes middle ground for a respectful and quality
evaluation process. This approach ensures evaluation remains relevant and purposeful for
those who are impacted by the program outcomes, providing a focus on the everyday reality
that depends on cultural responsiveness and ensures legitimacy for Indigenous people.
Introduction
process is legitimate according to the respective
Community engagement in the context of a remote
community’s cultural and social protocol system.
Indigenous community is about respect, understanding
It is important to work in collaboration with the
and building relationships within the cultural realms of a
community to ensure that the community understands:
community. The effectiveness of community engagement
what the evaluation is about; who is conducting it; who
depends on an open and honest sharing of information
will be involved; and how it will impact people in the
and a spirit of partnership between the parties involved.
workplace or in the community. It is also important
For evaluation to be effective in this context a degree of
to present and discuss findings and subsequent
connection that goes beyond the usual focus on output
reports to the community. An appropriate process of
measurement and system monitoring is needed. Seeking
community engagement is fundamental for program
guidance from the people who the program is targeting
development, implementation and evaluation in
is key to this process, as is ensuring that the engagement
Indigenous communities but is too often overlooked
or conveniently avoided. This then questions the usage
R o s s i n g h , Yu n u p i n g u — E v a l u a t i n g a s a n o u t s i d e r o r a n i n s i d e r
5
R E F E R E E D A R T I C L E
of data and statistics arising from program evaluation and the way
this information is used to potentially influence program and service
delivery (Malezer, 2012).
Evaluators who are experienced with working in remote
communities become more and more cognisant of the issues and
complexities that impact the quality of evaluations. As this reality
presents itself, evaluators seek guidance from the local community
so they may understand the factors that give rise to the outcomes
and therefore the findings. A learning journey takes place that allows
‘outsiders’ to be accepted into the community and be part of a
productive matrix of support and ‘insiders’ begin to assume their
legitimate rights to participate, advise, educate and guide evaluators.
Without a participative and collaborative styled process, evaluation
risks being non-reality based without a guarantee of quality and with
the potential to impact the community in a negative way.
The primary purpose of this paper is to present the voices of
Indigenous women who live in remote Indigenous communities,
Bronwyn Rossingh is Community
have operated under government-based and other highly politicised
Engagement Leader in the Office of Pro-Vice
programs and assisted numerous researchers and evaluators over a
Chancellor—Indigenous Leadership
number of years. The helpful and insightful contributions, discussed
at Charles Darwin University, Darwin,
later in this paper, arose from a panel-styled presentation that took
Northern Territory.
place at the 2015 Australasian Evaluation Society (AES) International
Email: bronwyn.rossingh@cdu.edu.au
Evaluation Conference in Melbourne. The discussion describes what
effective community engagement is and how evaluators can connect
with the community to conduct quality evaluations that draw on truth
and reality.
This paper is structured firstly to allow a discussion about
the field of evaluation, foundational principles of evaluation and
the evolution and process of evaluation in the context of remote
Indigenous communities. Further to this, the commencing discussion
includes the personal experiences of a non-Indigenous researcher
and evaluator who has worked in Indigenous communities for
many years undertaking evaluations and research alongside or
directed by community leaders. The discussion then draws on the
collective experiences of effective and poor community engagement
and evaluative processes conducted in communities and the critical
messages that are of significant value to the field of evaluation.
Key work that has guided Indigenous evaluation
The intent of this paper is to explore some of the foundational
Yalmay Yunupingu is Honorary University
literature of evaluation, how this has added value to evaluation in
Fellow in the Office of Pro Vice-Chancellor—
cultural contexts and also offer ways to gently push the boundaries
of evaluation by capturing the spirit of culture based on remote
Indigenous Leadership at Charles Darwin
Indigenous communities of Australia. Each year in the lead up to, and
University, Yirrkala, Northern Territory.
aftermath of, the annual AES International Evaluation Conference, the
Email: yunupingu@ntschools.net
messages of the society expands the horizons and reinforces evolving
principles and values of evaluation as a discipline premised on a
cycle of challenge and change. We are seeing more and more reviews
and evaluations of government programs located in Indigenous
communities across critical areas such as: education, health, housing,
employment, early childhood, youth, seniors, women’s business and
men’s business. These programs and their subsequent evaluation
have become an everyday reality in remote Indigenous communities.
Evaluation in Indigenous contexts needs to keep evolving as per the
6
E v a l u a t i o n J o u r n a l o f A u s t r a l a s i a Vo l 1 6 | N o 3 | 2 0 1 6
R E F E R E E D A R T I C L E
foundational work of Scriven (1997, 2012) and more
the success (or otherwise) of a program. This leads to a
specifically the seminal work of Wehipeihana (2008).
further quote from Greene (2013) referring to the essence
In 2003 Wehipeihana was an executive member of the
of Scriven’s principles seeking more than just the surface
AES Board and drove the process to have Indigenous
story and understanding the social basis of the program
evaluation as a strategic objective of the society. The fruits
being evaluated:
of her foundational work for the society continue today
Assessing the magnitude and contours of outcomes
and should be acknowledged as a major contribution.
against important needs of the intended beneficiaries or
Later work by Boulton et al. (2012), pays homage to
impactees. (p. 120)
Wehipeihana’s pivotal work and further reinforces the
importance of persevering and seeking answers to the
Over the past 10 to 15 years there has been much
necessary questions pertaining to program evaluation
discussion around who is best positioned to undertake
that gives rise to the must-have dialogue within the AES
an evaluation in an Indigenous community context. The
and its members concerning Indigenous evaluation. This
literature has demonstrated a relatively strong thread of
discourse remains of high interest to the AES and its
consistency concerning the key notions of community
members and has gained momentum due to the respectful
participation, understanding the influences of the
position that Wehipeihana (2008) and Boulton et al. (2012)
socio-political landscape, cultural differences and ethical
have operated from to cater for diversity relating to Maori
protocols. Taylor, who was a key note speaker at the 2003
and Australian Indigenous cultures. This earlier work has
AES International Evaluation Conference states that
engendered a more respectful and contextual position for
evaluations:
Indigenous evaluation but we still need to continue the
require proper exploration and consideration and extend
dialogue and build stronger foundations by respecting and
to the examination of the cultural settings pertaining to
acknowledging the voice of Australian Indigenous Peoples.
the activity, project or community service which is being
This is something that Wehipeihana and Boulton were
evaluated—as well as the questioning of those who are ...
To continue reading
Request your trial