Evaluation Journal of Australasia

Issue Number

Latest documents

• Online peer-group mentoring: An innovative mentoring program within an Australian professional association

  This article describes an online peer-group mentoring program to support career-long professional development, professional identity development, and stronger experiential peer-to-peer learning. It was developed in response to ongoing interest among members of the Australian Evaluation Society (AES), a professional association that aims to improve the theory, practice, and use of evaluation. It is intended to complement the technical training that can be provided through workshops or professional training and conventional one-on-one mentoring that members may access in other ways. Now in its third year, the program refined its focus and major objectives from promoting technical skills and knowledge to a broader focus on building professional networks, reducing professional isolation, and enhancing professional confidence. Two distinct approaches emerged from the first two years of the program, with each approach appealing to different participants. Some groups focused broadly on career development and professional network building, while others focused on topics of particular interest to participants. The peer-group approach is relevant to other professional associations in which practitioners tend to work alone or in small groups. Evaluation findings can inform other such associations about designing and implementing online group mentoring programs for their members.

• Evaluator perspective: A conversation with Australian Evaluation Society fellow – John Guenther
• Utilising existing data for a pilot social return on investment analysis of the family wellbeing empowerment program: A justification and framework
• ‘Fanning the flame’: Aboriginal girls, women, and their communities define success for the Shooting Stars empowerment program

  Education and engagement programs for Aboriginal young people tend to propagate Western, colonial paradigms: Aboriginal populations are targeted, yet Aboriginal voices are rarely heard throughout the planning, delivery, and evaluation stages of program implementation. It is essential that the voices of program participants – and the communities within which these programs serve – are provided a platform to determine what those outcomes are to be, and how they are to be achieved. Shooting Stars is a holistic engagement program for Aboriginal and Torres Strait Islander girls based in 20 remote and regional schools in Western Australia and South Australia. Shooting Stars staff (85% Aboriginal and Torres Strait Islander) facilitate yarning circles with program participants, local steering committees, and broader community groups in order to tailor program delivery to local needs and assess the effectiveness of the program. From 2020 to 2021, we yarned with our participants, communities, and localised steering committees about their definitions of success and how Shooting Stars can help to grow successful young women. In this article, we explore a complex, multidimensional definition of success defined by program participants and their communities, and describe how we are reflecting and implementing this definition across program delivery and evaluation.

• Mapping benefits and value from evaluations
• Book Review: Evaluation management: How to commission and conduct evaluations that matter
• Evaluation at the cutting edge: Driving innovation and quality
• Meta-evaluation: Validating program evaluation standards through the United Nations Evaluation Quality Assessment (EQAs)

  The 2015 United Nation’s ‘Eval Year’ declaration heightened program evaluation’s significance in achieving the Sustainable Development Goals. However, uncertainties persist regarding evaluating high-quality evaluations and addressing social justice concerns in meta-evaluations. The field lacks consensus on both conducting meta-evaluation and the standards to use. To address this, we reviewed meta-evaluation literature, mapped the American Evaluation Association’s foundational documents with the United Nations Evaluation Group’s Norms and Standards to explore their intersectionality on social justice, and analysed 62 United Nations Population Fund evaluation reports alongside their management responses. Our findings indicate that addressing social justice concerns in meta-evaluation is contingent on context rather than established standards. Thus, it’s crucial for evaluators to prioritise social justice in evaluation design and implementation, and to select quality assurance tools that match the evaluation context and professional association guidelines, especially in the absence of standardised guidelines.

• Four suppositions about emergent properties of complex interventions that operate and function as systems: A call for evaluation research

  This article focuses on evaluating the emergent properties of complex interventions that operate and function as systems. After defining key terms, four suppositions of emergence are presented. The suppositions are based on the authors’ reflections on evaluating several complex interventions using a systems approach. Supposition 1 posits that there is an operational emergent property, paralleling the theory-driven evaluation concept of implementation theory noted by Carol Weiss. Supposition 2 suggests that every complex intervention also has a functional emergent property that speaks to the effectiveness of the intervention. Several important distinctions between a functional emergent property and the long-term outcomes derived based on a reductionist worldview are noted. Supposition 3 builds on the work of Chalmers to posit that it is possible for a complex intervention to simultaneously have more than one operational emergent property and more than one functional emergent property. Supposition 4 is grounded in the system definition to propose that operational emergence is a prerequisite for functional emergence. Suppositions by definition require proof to become theories. The article calls for evaluators to share whether their experiences support these suppositions and encourages investments in evaluation research to test them.

• Book Review: Policy evaluation in the era of COVID-19

Featured documents

• A Stagecraft of New Zealand Evaluation

  Since the 1960s there has been a growing interest in evaluation shown by most Western countries. Alongside discussion of practical and theoretical issues of evaluation, such as methodological developments, best practice, and cross-cultural practice, there has also been increased interest in mapping ...

• Generalising from qualitative evaluation

  Evaluations are often focused on assessing merit, value, outcome or some other feature of a programme, project, policy or some other object. Evaluation research is then more concerned with the particular rather than the general – even more so, when qualitative methods are used. But does this mean...

• How to Design Better Programs: A Staff Centred Stakeholder Approach to Program Logic Modeling
• Book Review: What Counts as Credible Evidence in Applied Research and Evaluation Practice?
• Evaluating Unitec’s ALLY workshop on diverse sexuality and gender inclusivity

  This article presents findings from a mixed-method, utilisation-focused evaluation of an education workshop on diverse sexuality and gender (DSG) inclusivity offered at Unitec Institute of Technology in Auckland, New Zealand. Workshop completion enables participants to make a public commitment to...

• Notes to contributors
• Making two plus two equal five

  This paper examines ways in which internal data can be used in the evaluation research through data matching. It illustrates this through two case studies involving a three-year longitudinal and a cross-sectional evaluation of two large-scale Australian assistance programs. Discussion centres...

• The wheelbarrow, the mosaic and the double helix
• Aes Contacts 2002-2003
• An evaluation of governance mechanisms as determinants of performance: A case study of Australian universities

  The premise underlying the introduction of corporate governance principles in the Australian higher education sector is that good governance results in good performance. The purpose of this paper is to evaluate the influence of internal and external corporate governance mechanisms as determinants...