Comparing online with brick and mortar course learning outcomes

Article
Teaching Public Administration
Comparing online with
2014, Vol. 32(1) 95–107
ª The Author(s) 2014
brick and mortar course
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/0144739414523284
learning outcomes: An
tpa.sagepub.com
analysis of quantitative
methods curriculum in
public administration
Ronald A Harris and Gleb O Nikitenko
University of San Francisco, CA, USA
Abstract
Teaching graduate students in an intensive adult-learning format presents a special
challenge for quantitative analytical competencies. Students often lack necessary back-
ground, skills and motivation to deal with quantitative-skill-based course work. This
study compares learning outcomes for graduate students enrolled in three course
sections (cohorts) taking a quantitative methods course in a public administration
program. One cohort of students was taught online, while two student cohorts were
taught face-to-face in a traditional classroom setting. Most of the online students resided
in the same geographic location as the ‘ brick-and-mortar’’ students. While student
backgrounds and demographics were comparable, there were notable differences in
their levels of self-directed learning readiness and persistence. These differences illus-
trate both course design and modality features for a comparison between online and
traditional brick-and-mortar learning environments. We find that predictors of student
performance in an online environment are rather well described by the Self-Directed
Learning Theory (SDL) and Self-Regulated Learning Theory (SRL). A statistically signif-
icant difference was found in the pretest-posttest mean scores, which indicates that
students learned the course content for quantitative methods in the online section
differently from those in the brick-and-mortar section. Overall, students enrolled in the
online section (cohort) performed better on the posttest than did students enrolled in
traditional ‘ brick-and-mortar’’ classes. An age variable shows that older students per-
formed much better than younger students on the posttest. Other differences in learning
Corresponding author:
Gleb O. Nikitenko, University of San Francisco, 2130 Fulton Street, San Francisco, CA 94118, USA.
Email: nikitenko@usfca.edu; raharris4@usfca.edu

---

96
Teaching Public Administration 32(1)
outcomes between the online and brick-and-mortar sections are analyzed in the study.
Stakeholders in online education should be interested in these outcomes.
Keywords
online education, brick and mortar, learning, theory, students, public administration
The problem
Research is needed on how context influences learning (Garrison, 1997, 2003;
Gunawardena and McIsaac, 2003; Song and Hill, 2007). Higher education takes place
in a variety of contexts, ranging from face-to-face classrooms to virtual classrooms.
Within these settings, a variety of methods are used to enable interactions, including
100 per cent physical classroom interactions and 100 per cent online interactions. The
issues and attributes in various learning contexts (i.e. physical classroom instruction, a
web-based course, a computer-based instructional unit, et cetera), require further
exploration (Song and Hill, 2007).
More specifically, the impact of course content – such as quantitative analysis and
research methods – on student motivation and success, and faculty effectiveness in a
variety of contexts is drawing scholarly attention in different academic journals, such as
the Journal of Public Affairs Education, Teaching Public Administration, Public
Administration Review, and other publications. The revised National Association of
Schools of Public Policy and Administration standards and the quest for better student
learning assessment and experiences continued by most accrediting bodies in the United
States contribute to the academic and scholarly interest, as well as the sense of urgency,
in the area of teaching and evaluating student quantitative knowledge and performance.
Analytical skills make up one of the five principal competencies continuously assessed
by the Western Association of Schools and Colleges. They remain perhaps the most
important skills for overall student success and job marker performance. Yet, they are
rather difficult areas of instruction and assessment (Wright et al., 2004).
The need for empirical research into online student learning competencies, such as
quantitative skills, is recognized. Unfortunately, recent academic literature is replete
with accounts of personal learning, teaching experiences, and anecdotal observations.
Empirical research into online learning is at the cutting edge of pedagogical innovation
(Bonk et al., 2006; Boyer and Kelly, 2005; Dziuban et al., 2004; Hiemstra, 2003; Ya Ni,
2012). Previous research on learning outcomes in the cognitive domain usually measure
factors, such as course performance, content processing, levels of student interaction,
and retention.
Research has also been conducted to analyze various socio-demographic characteristics
of online learners. Those characteristics include personal demographic characteristics,
learners’ experiences and satisfaction with e-learning, and prior experiences in
computer-related activities, such as electronic mail, online course work, and internet use.
Learning styles and the quality of learners’ social interactions in an online environment
were commonly investigated (Bee and Usip, 1998; Gunawardena and Duphorne, 2001;
96

---

Harris and Nikitenko
97
Mortensen and Young, 2000; Muilenburg and Berge, 2005; Neuhauser, 2002; Swan et al.,
2001; Wells, 2000).
Some scholars found independent variables that statistically significantly affected
student perceptions of e-learning; the variables included gender, age, ethnicity, type of
learning institution, self-rating of online learning skills, effectiveness of learning online,
online learning enjoyment, prejudicial treatment in traditional classes, and the number of
online courses completed. These findings show that people with more prior experience
and training in computer-related activities reported more satisfaction and comfort with
the online environment (Muilenburg and Berge, 2005; Swan et al., 2001; Wells, 2000).
Other scholars have found no statistically significant differences in test scores,
assignments, participation grades, and final grades based on gender, age, learning pre-
ferences and styles, and media familiarity (Neuhauser, 2002; Ya Ni, 2012). Evidence
shows that student performance as measured by grade or score is not related to mode of
instruction. Rather, lower student persistence was found to be typical of educational
performance in an online environment as compared to an in-class format. A generally
higher level of interaction was observed in online courses, specifically those in research
methods and quantitative analysis (Tallent-Runnels et al., 2006; Ya Ni, 2012). No
consensus exists among researchers about relationships between demographic characteris-
tics and learning performance in online courses (Tallent-Runnels et al., 2006). Therefore,
further investigation is warranted.
Learning theory
Self-directed learning theory (SDL) has been linked conceptually with internet-based
learning since the 1990s (Caffarella, 1993; Long, 2001; Monolescu and Schifter,
2001). SDL is defined by Conner et al. (1995) as ‘learning initiated and directed by the
learner’ (p. 62); SDL includes self-paced, independent, individualized learning, as well
as self-instruction (Caffarella, 1993). An SDL strategy is effective, as it forces the learner
to take the initiative, resulting in a more active learning process and a deeper understand-
ing of the assigned course material (Brockett, 2001).
E-learning, by its very learner-centered nature, is where SDL can and does occur
(Garrison, 2003; Gunawardena and McIsaac, 2003; Shapley, 2000). Instructors teaching
asynchronous online classes and providing guidelines for e-learning allow students to
study at their own pace, in their own environment,...