Developing a Scratch-based coding achievement test
| Date | 13 May 2019 |
| Published date | 13 May 2019 |
| Pages | 383-406 |
| DOI | https://doi.org/10.1108/ILS-08-2018-0078 |
| Author | Ömer Demir,Süleyman Sadi Seferoğlu |
| Subject Matter | Library & information science |
Developing a Scratch-based
coding achievement test
Ömer Demir and Süleyman Sadi Sefero
glu
Department of Computer Education and Instructional Technology,
Hacettepe University, Ankara, Turkey
Abstract
Purpose –The lack of a reliable and valid measurementtool for coding achievement emerges as a major
problem in Turkey. Therefore, thepurpose of this study is to develop a Scratch-based coding achievement
test.
Design/methodology/approach –Initially, an item pool with31 items was created. The item pool was
classified withinthe framework of Bayman and Mayer’s (1988) types of coding knowledge to supportcontent
validity of the test. Then the item pool was appliedto 186 volunteer undergraduates at Hacettepe University
during the spring semester of the 2017-2018 academicyear. Subsequently, the item analysis was conducted
for constructvalidity of the test.
Findings –In all, 13 items were discarded from the test, leaving a total of 18 items. Out of the 18-item
version of the coding achievement test, 4, 5 and 9 items measured syntactic, conceptual and strategic
knowledge, respectively, among the types of coding knowledge. Furthermore, average item discrimination
index (0.531),average item difficulty index (0.541) and Cronbach Alpha reliabilitycoefficient (0.801) of the test
were calculated.
Practical implications –Scratch users, especiallythose who are taking introductory courses at Turkish
universities,could benefit from a reliable and valid coding achievementtest developed in this study.
Originality/value –This paper has theoretical and practicalvalue, as it provides detailed developmental
stages of a reliableand valid Scratch-based coding achievement test.
Keywords Validity and reliability, Scratch, Coding achievement test, Coding education,
Measuring coding skills, Types of coding knowledge
Paper type Research paper
Introduction
As first put forward by Perlis (1962), and later by Papert (1980), coding education has long
been an important area of research and pedagogical development. The spread of coding
education gained significant momentumparticularly after Wing’s(2006)introduction of the
concept of computational thinking (CT). Today, coding education is regarded as a
requirement for digital natives (Prensky, 2001) to attain CT skills (Kanbul and Uzunboylu,
2017;Wing, 2006).
Aho (2012) defines CT as thinking processes containing the use of computational steps
and algorithms in solving problems.CT is a skill that can be associated with Computational
Thinker, a standard identified by International Society for Technology in Education (ISTE)
in 2016 as one of the seven characteristics that should be possessed digital natives. Coding
education can also be indirectly associated with the Knowledge Constructor standard of
ISTE, 2016. This standardincludes individuals’presentations of informationand innovative
products by using digital tools.It can be argued that coding education makes it possible for
users to achieve bothof these standards.
Coding education is being included in more primary and secondary education curricula
because coding education can make important contributions to a country’s development
Scratch-based
coding
383
Received7 August 2018
Revised20 February 2019
Accepted25 February 2019
Informationand Learning
Sciences
Vol.120 No. 5/6, 2019
pp. 383-406
© Emerald Publishing Limited
2398-5348
DOI 10.1108/ILS-08-2018-0078
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/2398-5348.htm
(Sayin and Seferoglu, 2016). The developments in coding education have positively
contributed to the coding education in Turkey and, as a result, a course titled Information
Technologies and Software has been incorporated in the curricula of Grades 5 and above
since 2012-2013 academicyear.
Coding education has become readily accessible for teachers in recent years. Tools such
as Scratch (Scratch.mit.edu, 2018), Alice (Alice.org, 2018), Code.org (Code.org, 2018) and
others, along with globalcoding activities such as Bebras (Bebras.org, 2018) and the hour of
code (TheHourofCode.org, 2018),played important roles in the process. Not only has coding
become available but its positive academic effects have also become more evident. For
example, numerous scholars have linked coding activities to increased student motivation
(Akpinar and Altun, 2014;Calder, 2010;Kelleher et al.,2007;Kert and Ugras, 2009). Coding
education is also connectedto the development of students’higher-order thinking skillssuch
as analysing, assessing, relating, interpreting, problem-solving, critical thinking and
algorithmic thinking (Akpinar and Altun, 2014;Kert and Ugras, 2009). A meta-analysis
study examining the effect of coding indicates that coding mostly has a positive effect on
problem-solving and other higher-order cognitive skills (Liao and Bright, 1991). This is
because coding intenselyincludes mathematical and logical thinking skills.
In the rest of the introduction, we examine block-based coding languages, then, types of
coding knowledge, followed by our argument for validated coding achievement tests. We
also discuss the perceived shortcomings of existing coding achievement tests and articulate
the rationale for our study and how it buildson prior scholarshipin this area.
Block-based coding languages
Block-based coding languagesare based on the idea that beginners should be able to master
basic coding concepts more easily. This way, beginnerscan get used to professional coding
languages more quickly. In block-basedcoding languages, coders do not write codes and do
not need to be familiar with the confusing syntax of professional coding languages. They
just drag and drop code blocks to create a programme, which is not intimidating for
beginners. Scratch (Scratch.mit.edu, 2018), Blockly (Blockly-games.appspot.com, 2018) and
Snap (Snap.berkeley.edu,2018) are good examples of block-based codinglanguage tools.
Scratch is the most well-known among these block-based languages. Scratch was
developed by the Lifelong KindergartenGroup at the MIT Media Lab in 2007. Scratch is free
and easy to learn. Thus, Scratch is an appropriate tool to teach coding to the beginners.
Scratch is used not only by children but alsoby freshmen and sophomores at universities to
ease the difficulties in learning coding.Therefore, it has millions of users all over the world.
As of 20 February 2019, the number of registered users in the official Scratch website is
35,584,584 (Scratch.mit.edu, 2018). It should be noted that Scratch can also be used offline,
without registering in the official Scratch website,further increasing the number of Scratch
users. Scratch is widely used in Turkey,where the authors have based their work. There are
484,137 Scratchers in Turkey, which make up 1.44 per cent of Scratchers all around the
world, as of 20 February 2019.Turkey is second only to the USA in Scratch usage, according
to Scratchers worldwide map. Furthermore, nine Scratch days were celebrated all around
Turkey in 2018, which demonstratesa strong commitment to coding education (Scratch.mit.
edu, 2018). Nevertheless, it is unknown how many universitystudents in Turkey are using
Scratch and learning introductoryprogramming through Scratch.
Types of coding knowledge
Because coding is a unique knowledge domain including several sub-knowledge types,
scholars have developed a distinct coding knowledge classification. According to
ILS
120,5/6
384
To continue reading
Request your trial