Supporting Skill Assessment in Learning Experiences Based on Serious Games Through Process Mining Techniques Juan Antonio Caballero-Hernández, Manuel Palomo-Duarte, Juan Manuel Dodero, Dragan Gaševic International Journal of Interactive Multimedia and Artificial Intelligence, 2024 Learning experiences based on serious games are employed in multiple contexts. Players carry out multiple interactions during the gameplay to solve the different challenges faced. Those interactions can be registered in logs as large data sets providing the assessment process with objective information about the skills employed. Most assessment methods in learning experiences based on serious games rely on manual approaches, which do not scalewell when the amount of data increases. We propose an automated method to analyse students’ interactions and assess their skills in learning experiences based on serious games. The method takes into account not only the final model obtained by the student, but also the process followed to obtain it, extracted from game logs. The assessment method groups students according to their in-game errors and in-game outcomes. Then, the models for the most and the least successful students are discovered using process mining techniques. Similarities in their behaviour are analysed through conformance checking techniques to compare all the students with the most successful ones. Finally, the similarities found are quantified to build a classification of the students’ assessments. We have employed this method with Computer Science students playing a serious game to solve design problems in a course on databases. The findings show that process mining techniques can palliate the limitations of skill assessment methods in game-based learning experiences.
Comparative Study of Moodle Plugins to Facilitate the Adoption of Computer-Based Assessments Milagros Huerta, Juan Antonio Caballero-Hernández, Manuel Alejandro Fernández-Ruiz Applied Sciences Switzerland, 2022 The use of Learning Management Systems (LMS) has had rapid growth over the last decades. Great efforts have been recently made to assess online students’ performance level, due to the COVID-19 pandemic. Faculty members with limited experience in the use of LMS such as Moodle, Edmodo, MOOC, Blackboard and Google Classroom face challenges creating online tests. This paper presents a descriptive and comparative study of the existing plugins used to import questions into Moodle, classifying them according to the necessary computing resources. Each of the classifications were compared and ranked, and features such as the support for gamification and the option to create parameterised questions are explored. Parameterised questions can generate a large number of different questions, which is very useful for large classes and avoids fraudulent behaviour. The paper outlines an open-source plugin developed by the authors: FastTest PlugIn, recently approved by Moodle. FastTest PlugIn is a promising alternative to mitigate the detected limitations in analysed plugins. FastTest PlugIn was validated in seminars with 230 faculty members, obtaining positive results about expectations and potential recommendations. The features of the main alternative plugins are discussed and compared, describing the potential advantages of FastTest PlugIn.
Upgrading to 3D of a serious game for conceptual data modeling Ceur Workshop Proceedings, 2021
Learning analytics to detect evidence of fraudulent behaviour in online examinations Antonio Balderas, Manuel Palomo-Duarte, Juan Antonio Caballero-Hernández, Mercedes Rodriguez-Garcia, Juan Manuel Dodero International Journal of Interactive Multimedia and Artificial Intelligence, 2021 Lecturers are often reluctant to set examinations online because of the potential problems of fraudulent behaviour from their students. This concern has increased during the coronavirus pandemic because courses that were previously designed to be taken face-to-face have to be conducted online. The courses have had to be redesigned, including seminars, laboratory sessions and evaluation activities. This has brought lecturers and students into conflict because, according to the students, the activities and examinations that have been redesigned to avoid cheating are also harder. The lecturers' concern is that students can collaborate in taking examinations that must be taken individually without the lecturers being able to do anything to prevent it, i.e. fraudulent collaboration. This research proposes a process model to obtain evidence of students who attempt to fraudulently collaborate, based on the information in the learning environment logs. It is automated in a software tool that checks how the students took the examinations and the grades that they obtained. It is applied in a case study with more than 100 undergraduate students. The results are positive and its use allowed lecturers to detect evidence of fraudulent collaboration by several clusters of students from their submission timestamps and the grades obtained.
Analysis of Learning Records to Detect Student Cheating on Online Exams: Case Study during COVID-19 Pandemic Antonio Balderas, Juan Antonio Caballero-Hernández ACM International Conference Proceeding Series, 2020 In March 2020, due to the Covid19 pandemic, higher education had to switch from face-to-face to exclusively virtual mode overnight. In this unexpected scenario, supervisors also had to adapt the assessment procedures, including the exams. This caused a significant controversy, as, according to many students, supervisors were more concerned about how to prevent students from cheating, than actually measuring their learning. This paper introduces an experience that implemented several of the students' requests in an online exam and conducts a comprehensive analysis of students’ behavior according to the virtual learning environment records. Different existing software tools are used for the analysis, complemented with a Python application ad-hoc developed. The objective indicators gathered provide evidence that some students cheated and invite focusing on evidence-based assessment.
Discovering player profile behavior models in serious games Ceur Workshop Proceedings, 2020
Teamwork assessment in collaborative projects through process mining techniques International Journal of Engineering Education, 2020
Learning personalization in block-based programming languages using clustering and static code analysis T Person, JA Caballero-Hernández, C Romero, I Ruiz-Rube, JM Dodero Computers and Education Open, 100333 , 2026 2026 Citations: 1
Wondercoins: dissemination of the ancient numismatic archaeological heritage of the western Mediterranean as open data M Palomo-Duarte, A Contreras-Puerto, C Boueri-Martínez, A Balderas, ... 2025 International Symposium on Computers in Education (SIIE), 1-4 , 2025 2025
Wondercoins: difusión del patrimonio arqueológico numismático antiguo del occidente mediterráneo como datos abiertos M Palomo Duarte, A Contreras Puerto, C Boueri Martínez, ... 2025
Análisis de Sentimientos y Polarización de Reseñas de Videojuegos a Través de Inteligencia Artificial A Contreras Puerto, A Muñoz Ortega, M Palomo Duarte, ... Sociedad Española para las Ciencias del Videojuego , 2025 2025
Integración de Instagram y TikTok en la docencia universitaria: visibilidad, creatividad y aprendizaje L Gutiérrez Madroñal, R Baena Pérez, JA Caballero Hernández, ... 2025
There is no place like tomorrow: videojuego para sensibilizar sobre utopías desde la perspectiva de género LI Benitez Eyzaguirre, JA Roa Domínguez, M Palomo Duarte, ... 2025
Serious Game Adventure for Learning Data Exploration Skills M Palomo-Duarte, JA Caballero-Hernández, F Díaz-Brotons, A Balderas, ... Innovation and Technologies for the Digital Transformation of Education … , 2024 2024
Opinión de los estudiantes sobre el uso de un Modelo Grande de Lenguaje en una asignatura de Bases de Datos M Palomo-Duarte, R Baena-Pérez, A Balderas, JA Caballero-Hernández, ... Book of Abstracts. SIIE 2024. XXVI International Symposium on Computers in … , 2024 2024 Citations: 2
Uso de modelo grande de lenguaje en un curso de base de datos: una visión vía TAM M Palomo Duarte, R Baena Pérez, A Balderas Alberico, ... Instituto Politécnico de Setúbal , 2023 2023
Exploring the impact of ChatGPT on education: A case of study on Spanish Tweets A Muñoz, R Baena-Pérez, A Balderas, M Palomo-Duarte, ... International conference on technological ecosystems for enhancing … , 2023 2023 Citations: 1
Supporting skill assessment in learning experiences based on serious games through process mining techniques JA Caballero Hernández, M Palomo Duarte, JM Dodero Beardo, ... 2023 Citations: 16
Videoaventura para aprender lenguaje SQL M Palomo-Duarte, F Díaz-Brotons, JA Caballero-Hernández, JA Ortega, ... Universidad de Zaragoza, Servicio de Publicaciones , 2023 2023
Herramienta para un Análisis Visual de las Colaboraciones entre Estudiantes en la Realización de Cuestionarios de Moodle A Balderas Alberico, F Carrillo Chaves, M Palomo Duarte, ... 2022
Comparative study of Moodle plugins to facilitate the adoption of computer-based assessments M Huerta, JA Caballero-Hernández, MA Fernández-Ruiz Applied Sciences 12 (18), 8996 , 2022 2022 Citations: 29
Comparative Study of Moodle Plugins to Facilitate the Adoption of Computer-Based Assessments M Huerta Gómez de Merodio, JA Caballero Hernández, ... MDPI , 2022 2022
Learning Analytics to Detect Evidence of Fraudulent Behaviour in Online Examinations. A Balderas, MP Duarte, JAC Hernández, MR García, JM Dodero International Journal of Interactive Multimedia and Artificial Intelligence … , 2021 2021 Citations: 14
Ampliación a 3D de un juego serio para modelado conceptual de datos JA Caballero Hernández, M Palomo Duarte, JM Dodero Beardo, ... CEUR Workshop Proceedings , 2021 2021
Learning Analytics to Detect Evidence of Fraudulent Behaviour in Online Examinations A Balderas Alberico, M Palomo Duarte, JA Caballero Hernández, ... UNIV INT RIOJA , 2021 2021
Análisis de eventos para evaluar competencias en experiencias de aprendizaje basadas en serious games JA Caballero Hernández 2020
Analysis of learning records to detect student cheating on online exams: Case study during COVID-19 pandemic A Balderas, JA Caballero-Hernández Eighth international conference on technological ecosystems for enhancing … , 2020 2020 Citations: 76
MOST CITED SCHOLAR PUBLICATIONS
Skill assessment in learning experiences based on serious games: A Systematic Mapping Study JA Caballero-Hernández, M Palomo-Duarte, JM Dodero Computers & Education 113, 42-60 , 2017 2017 Citations: 98
Assessing foreign language learning through mobile game-based learning environments M Palomo-Duarte, A Berns, A Cejas, JM Dodero, JA Caballero, ... International Journal of Human Capital and Information Technology … , 2016 2016 Citations: 80
Analysis of learning records to detect student cheating on online exams: Case study during COVID-19 pandemic A Balderas, JA Caballero-Hernández Eighth international conference on technological ecosystems for enhancing … , 2020 2020 Citations: 76
Comparative study of Moodle plugins to facilitate the adoption of computer-based assessments M Huerta, JA Caballero-Hernández, MA Fernández-Ruiz Applied Sciences 12 (18), 8996 , 2022 2022 Citations: 29
Teamwork assessment in collaborative projects through process mining techniques JA Caballero-Hernández, A Balderas, M Palomo-Duarte, P Delatorre, ... The International journal of engineering education 36 (1), 470-482 , 2020 2020 Citations: 21
Supporting skill assessment in learning experiences based on serious games through process mining techniques JA Caballero Hernández, M Palomo Duarte, JM Dodero Beardo, ... 2023 Citations: 16
An architecture for skill assessment in serious games based on Event Sequence Analysis JAC Hernández, MP Duarte, JM Dodero Proceedings of the 5th international conference on technological ecosystems … , 2017 2017 Citations: 16
Learning Analytics to Detect Evidence of Fraudulent Behaviour in Online Examinations. A Balderas, MP Duarte, JAC Hernández, MR García, JM Dodero International Journal of Interactive Multimedia and Artificial Intelligence … , 2021 2021 Citations: 14
Integrating external evidences of skill assessment in Virtual Learning Environments JA Caballero, M Palomo, JM Dodero, G Rodríguez, MS Ibarra Fifth International Conference on e-Learning, Belgrade, Serbia , 2014 2014 Citations: 13
Discovering bottlenecks in a computer science degree through process mining techniques JA Caballero-Hernández, JM Dodero, I Ruiz-Rube, M Palomo-Duarte, ... 2018 International Symposium on Computers in Education (SIIE), 1-6 , 2018 2018 Citations: 12
Model-driven skills assessment in knowledge management systems A Balderas, JA Caballero-Hernández, JM Dodero, M Palomo-Duarte, ... Journal of web engineering 18 (4–6), 353-379 , 2019 2019 Citations: 10
A Community-Driven Mobile System to Support Foreign Language Learning M Palomo-Duarte, A Berns, A Cejas, JM Dodero, ... Multidisciplinary Perspectives on Human Capital and Information Technology … , 2018 2018 Citations: 8
Evaluación de competencias en Serious games mediante analítica de aprendizaje con Process Mining JA Caballero-Hernández, T Person, M Palomo-Duarte, JM Dodero 2019 Citations: 6
Integrating quantitative and qualitative data in assessment of wiki collaborative assignments MP Duarte, A Balderas, JM Dodero, AJ Reinoso, JA Caballero, ... Proceedings of the Sixth International Conference on Technological … , 2018 2018 Citations: 5
Métodos de evaluación de competencias en serious games: estudio y análisis sobre su estado actual JA Caballero-Hernández, M Palomo-Duarte, JM Dodero IV Congreso Internacional Sobre Aprendizaje, Innovación y Competitividad … , 2017 2017 Citations: 5
Domain-driven competence assessment in virtual learning environments. Application to planning and time management skills A Balderas, A Galán-Pinero, JAC Hernández, GR Gómez, J Dodero, ... 2014 International Symposium on Computers in Education (SIIE), 121-126 , 2014 2014 Citations: 4
Opinión de los estudiantes sobre el uso de un Modelo Grande de Lenguaje en una asignatura de Bases de Datos M Palomo-Duarte, R Baena-Pérez, A Balderas, JA Caballero-Hernández, ... Book of Abstracts. SIIE 2024. XXVI International Symposium on Computers in … , 2024 2024 Citations: 2
Assessment of Generic Skills through an Organizational Learning Process Model. A Balderas, JAC Hernández, JM Dodero, M Palomo-Duarte, I Ruiz-Rube WEBIST, 293-300 , 2018 2018 Citations: 2
Métodos de evaluación de competencias en serious games: estudio y análisis sobre su estado actual [Skill assessment methods in serious games: a state of the art study and analysis] JA Caballero-Hernández, M Palomo-Duarte, JM Dodero 2017 Citations: 2
Learning personalization in block-based programming languages using clustering and static code analysis T Person, JA Caballero-Hernández, C Romero, I Ruiz-Rube, JM Dodero Computers and Education Open, 100333 , 2026 2026 Citations: 1