Exploiting a natural experiment in assessing student learning outcomes in public and nonprofit administration: A demonstration
| Published date | 01 October 2024 |
| DOI | http://doi.org/10.1177/01447394231223071 |
| Author | Reynold Galope,Robert Bilyk,Daniel Woldeab |
| Date | 01 October 2024 |
| Subject Matter | Articles |
Article
Teaching Public Administration
2024, Vol. 42(3) 352–375
© The Author(s) 2023
Article reuse guidelines:
sagepub.com/journals-permissions
DOI: 10.1177/01447394231223071
journals.sagepub.com/home/tpa
Exploiting a natural experiment
in assessing student learning
outcomes in public and
nonprofit administration: A
demonstration
Reynold Galope
Metropolitan State University, USA
Robert Bilyk
LodeStar Learning Corporation, USA
Daniel Woldeab
Metropolitan State University, USA
Abstract
This study exploits a natural experiment to establish the equivalence and/or difference in
student learning outcomes between online and face-to-face public and nonprofit ad-
ministration courses. Its main contribution is thus methodological. We will reexamine the
online v. classroom debate - the face-to-face lecture is still the most effective method to
deliver course content to students - using a new dataset and estimation approach. Our
research addresses this causal inference question: Does the format of course delivery
impact student learning? The most robust empirical strategy to rule out alternati ve
explanations in causal studies is the experimental approach. In this study, we did not
employ the experimental research design or any standard techniques, for example,
regression analysis, available to the program evaluator or policy analyst. Instead, we
exploited a naturally occurring phenomenon in a classroom environment to approximate
statistical equivalence in the characteristics of students in the online and classroom
formats and satisfy the exogeneity assumption of the treatment variable. Its more
practical contribution is the use of learning theory and new research in online pedagogy to
discuss the study’s conclusions and implications for online programming, instruction, and
Corresponding author:
Reynold Galope, Department of Public and Nonprofit Leadership, Metropolitan State University, 1450 Energy
Park Drive, Suite 147B, Saint Paul, MN 55108-5218, USA.
Email: reynold.galope@metrostate.edu
program coordination. We developed the feedback as teaching philosophy or approach to
close the gap between the learning outcomes of completely asynchronous online and
entirely face-to-face classes in public administration.
Keywords
Natural experiment, online teaching, assessment of student learning outcomes, task-
related feedback, process-related feedback, self-referenced feedback, public affairs
education, program evaluation, quasi-experiment, as-if-random assignment,
endogeneity, treatment effects analysis
Introduction
Online learning in higher education continues to expand (Allen and Seaman, 2017) with
advances in information and communications technology (ICT) as its primary platform.
1
ICT is indeed disruptive (Christ ensen, 1997); its speed, 24/7 convenience, and widespread
diffusion have fundamentally changed the market for post-secondary education. Internet
technology, mobile computing, and social media have provided an efficient alternative to
the traditional, brick-and-mortar delivery of post-secondary academic credentialing,
colleges.
2
Higher education institutions (HEIs) welcome online learning because of its potential
to “bend the cost curve”(Deming et al., 2015). For example, Georgia Institute of
Technology offers its pioneering online master’s degree in computer science at an “ultra-
low”tuition of about $7,000, which is only about a sixth of the cost of an on-campus
degree (McKenzie, 2018).
3
More importantly, online programming attracts non-
traditional and underserved students who require some flexibility in their academic
journeys (Woldeab et al., 2020). It effectively expands tuition revenues and manages the
risks and variations in enrollments by appealing to a segment of the student population
who would not otherwise have pursued higher education because of the time, geographic,
and other physical constraints of attending face-to-face classes. Online learning is thus a
cost-saving and market-expanding product in higher education.
However, despite its vast potential to expand access to higher education, the com-
pletely online format could be difficult. Its quality cannot be guaranteed. Online edu-
cation’s ability to meet students’academic needs may depend on the convergence of at
least three “success”factors - technology, student quality, and faculty acceptance of their
new roles in a virtual academic environment (Meine and Dunn, 2017).
First, HEIs entering the online market must invest in a learning management system
and associated software applications that facilitate online course design, development,
delivery, and evaluation of student learning outcomes. These online learning technologies
must be user-friendly to students and instructors alike and sufficiently robust to facilitate
more frequent and profound student-to-student and student-to-instructor interactions.
Galope et al. 353
Get this document and AI-powered insights with a free trial of vLex and Vincent AI
Get Started for FreeStart Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
Start Your Free Trial of vLex and Vincent AI, Your Precision-Engineered Legal Assistant
-
Access comprehensive legal content with no limitations across vLex's unparalleled global legal database
-
Build stronger arguments with verified citations and CERT citator that tracks case history and precedential strength
-
Transform your legal research from hours to minutes with Vincent AI's intelligent search and analysis capabilities
-
Elevate your practice by focusing your expertise where it matters most while Vincent handles the heavy lifting
