Stefan Marks

Verified email at

Auckland University of Technology



XR, data visualisation


Scopus Publications

Scopus Publications

  • Humanness lies in unpredictability: Role of Theory of Mind on anthropomorphism in human-computer interactions
    Julia Ayache, Andrew M. Connor, Stefan Marks, Alexander Sumich, and Nadja Heym

    HAI 2022 - Proceedings of the 10th Conference on Human-Agent Interaction, Pages: 306-308, Published: 5 December 2022 ACM
    Predictability is a core aspect of human-computer interaction (HCI), but an excess of predictability can lead to stereotypical behaviors and decreases the perceived agentivity (i.e., anthropomorphism) of a virtual agent. Yet, it remains unclear if inter-individual variability in predicting the behavior of other is modulating tendencies for anthropomorphism. The present study investigated the interaction between Theory of Mind (ToM) abilities and agent predictability and their association with anthropomorphism. Participants (N=38) completed self-reports of Perspective Taking capabilities and played the Matching Pennies Game with a virtual agent. The agent’s predictability was manipulated across four conditions, and participants reported their perception of the agent’s humanness and predictability. Results revealed that perceived predictability rather than self-reported Perspective Taking capabilities were positively associated with ToM abilities in predicting virtual agent’s behavior. However, increasing the virtual agent’s predictability decreased tendencies for anthropomorphism, stressing the role of randomness in perceiving humanness.

  • Evaluation of a Multi-agent “Human-in-the-loop” Game Design System
    Jan Kruse, Andy M. Connor, and Stefan Marks

    ACM Transactions on Interactive Intelligent Systems, ISSN: 21606455, eISSN: 21606463, Published: 26 July 2022 Association for Computing Machinery (ACM)
    Designing games is a complicated and time-consuming process, where developing new levels for existing games can take weeks. Procedural content generation offers the potential to shorten this timeframe, however, automated design tools are not adopted widely in the game industry. This article presents an expert evaluation of a human-in-the-loop generative design approach for commercial game maps that incorporates multiple computational agents. The evaluation aims to gauge the extent to which such an approach could support and be accepted by human game designers and to determine whether the computational agents improve the overall design. To evaluate the approach, 11 game designers utilized the approach to design game levels with the computational agents both active and inactive. Eye-tracking, observational, and think-aloud data was collected to determine whether designers favored levels suggested by the computational agents. This data was triangulated with qualitative data from semi-structured interviews that were used to gather overall opinions of the approach. The eye-tracking data indicates that the participating game level designers showed a clear preference for levels suggested by the computational agents, however, expert designers in particular appeared to reject the idea that the computational agents are helpful. The perception of computational tools not being useful needs to be addressed if procedural content generation approaches are to fulfill their potential for the game industry.

  • Exploring the “Dark Matter” of Social Interaction: Systematic Review of a Decade of Research in Spontaneous Interpersonal Coordination
    Julia Ayache, Andy Connor, Stefan Marks, Daria J. Kuss, Darren Rhodes, Alexander Sumich, and Nadja Heym

    Frontiers in Psychology, eISSN: 16641078, Published: 11 October 2021 Frontiers Media SA
    Interpersonal coordination is a research topic that has attracted considerable attention this last decade both due to a theoretical shift from intra-individual to inter-individual processes and due to the development of new methods for recording and analyzing movements in ecological settings. Encompassing spatiotemporal behavioral matching, interpersonal coordination is considered as “social glue” due to its capacity to foster social bonding. However, the mechanisms underlying this effect are still unclear and recent findings suggest a complex picture. Goal-oriented joint action and spontaneous coordination are often conflated, making it difficult to disentangle the role of joint commitment from unconscious mutual attunement. Consequently, the goals of the present article are twofold: (1) to illustrate the rapid expansion of interpersonal coordination as a research topic and (2) to conduct a systematic review of spontaneous interpersonal coordination, summarizing its latest developments and current challenges this last decade. By applying Rapid Automatic Keyword Extraction and Latent Dirichlet Allocation algorithms, keywords were extracted from PubMed and Scopus databases revealing the large diversity of research topics associated with spontaneous interpersonal coordination. Using the same databases and the keywords “behavioral matching,” “interactional synchrony,” and “interpersonal coordination,” 1,213 articles were identified, extracted, and screened following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses protocol. A total of 19 articles were selected using the following inclusion criteria: (1) dynamic and spontaneous interactions between two unacquainted individuals (2) kinematic analyses, and (3) non-clinical and non-expert adult populations. The results of this systematic review stress the proliferation of various definitions and experimental paradigms that study perceptual and/or social influences on the emergence of spontaneous interpersonal coordination. As methods and indices used to quantify interpersonal coordination differ from one study to another, it becomes difficult to establish a coherent picture. This review highlights the need to reconsider interpersonal coordination not as the pinnacle of social interactions but as a complex dynamical process that requires cautious interpretation. An interdisciplinary approach is necessary for building bridges across scattered research fields through opening a dialogue between different theoretical frameworks and consequently provides a more ecological and holistic understanding of human social cognition.

  • Discrete versus Continuous Colour Pickers Impact Colour Selection in Virtual Reality Art-Making
    Marylyn Alex, Danielle Lottridge, Jisu Lee, Stefan Marks, and Burkhard Wüensche

    ACM International Conference Proceeding Series, Pages: 158-169, Published: 2 December 2020 ACM
    Colour selection is an important task in digital art and 3D modelling applications. Most colour pickers are based on continuous colour spaces or a representative sampling of them, such as the Munsell colour palette. Continuous colour space-based pickers enable users to select from all colours by displaying full saturation hues with options to lower saturation and modify value. The two-step process of colour selection from continuous pickers requires understanding of 3D colour space, e.g., where to find “brown” or “sand”. In this research we investigate how continuous versus discrete pickers affect colour selection in virtual art. We compared an HSV picker with a discrete picker in a study with 40 participants aged 16 - 60. We found that the colour picker impacted the kinds of colours used in artworks, with significant differences in colour distribution characteristics. We discuss implications of colour selection tools for virtual reality art-making.

  • Development of a virtual construction approach for steel structures considering structural and nonstructural elements, and installation equipment
    Proceedings of the International Conference of Architectural Science Association, eISSN: 22093850, Volume: 2020-November, Pages: 405-414, Published: 2020

  • Active Learning and Teaching through Digital Technology and Live Performance: ‘Choreographic Thinking’ as Art Practice in the Tertiary Sector
    Jennifer R. A. Nikolai, Gregory Bennett, Stefan Marks, and Geoff Gilson

    International Journal of Art and Design Education, ISSN: 14768062, eISSN: 14768070, Pages: 137-152, Published: Februaryy 2019 Wiley

  • Evaluation of a Virtual Reality Nasal Cavity Education Tool
    Stefan Marks, David White, and Milan Mazdics

    Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018, Pages: 193-198, Published: 16 January 2019 IEEE
    In this paper, we present a user study to evaluate the suitability of a virtual reality (VR) application for teaching the anatomy of the human nasal cavity. We discuss the benefits of VR technology in the domain of anatomy education and present our application and the user study. Our results seem to indicate a disadvantage of the VR application with respect to assessment results, but also raise questions about the nature of assessment of spatial knowledge. With respect to usability and motivation, VR demonstrates advantages over “traditional” teaching material.

  • From von Neumann architecture and Atanasoffs ABC to neuro-morphic computation and Kasabov’s neuCube: Principles and implementations
    Neelava Sengupta, Josafath Israel Espinosa Ramos, Enmei Tu, Stefan Marks, Nathan Scott, Jakub Weclawski, Akshay Raj Gollahalli, Maryam Gholami Doborjeh, Zohreh Gholami Doborjeh, Kaushalya Kumarasinghe, Vivienne Breen, and Anne Abbott

    Studies in Computational Intelligence, ISSN: 1860949X, Volume: 756, Pages: 1-28, Published: 2018 Springer International Publishing
    During the 1940s John Atanasoff with the help of one of his students Clifford E. Berry, at Iowa State College, created the ABC (Atanasoff-Berry Computer) that was the first electronic digital computer. The ABC computer was not a general-purpose one, but still, it was the first to implement three of the most important ideas used in computers nowadays: binary data representation; using electronics instead of mechanical switches and wheels; using a von Neumann architecture, where the memory and the computations are separated. A new computational paradigm, named as Neuromorphic, utilises the above two principles, but instead of the von Neumann principle, it integrates the memory and the computation in a single module a spiking neural network structure. This chapter first reviews the principles of the earlier published work by the team on neuromorphic computational architecture NeuCube. NeuCube is not a general purpose machine but is still the first neuromorphic spatio/spectro-temporal data machine for learning, pattern recognition and understanding of spatio/spectro-temporal data. The chapter further presents the software/hardware implementation of the NeuCube as a development system for efficient applications on temporal or spatio/spectro-temporal across domain areas, including: brain data (EEG, fMRI), brain computer interfaces, robot control, multi-sensory data modelling, seismic stream data modelling and earthquake prediction, financial time series forecasting, climate data modelling and personalised, on-line risk of stroke prediction, and others. A limited version of the NeuCube software implementation is available from

  • Getting up your nose: a virtual reality education tool for nasal cavity anatomy
    Stefan Marks, David White, and Manpreet Singh

    SIGGRAPH Asia 2017 Symposium on Education, SA 2017, Published: 27 November 2017 ACM
    This article explores the application of virtual reality (VR) to the area of anatomical education, specifically the shape of and the airflow through the human nasal cavity. We argue the benefits of VR technology in this specific domain, and describe the creation of the VR application which is intended to be used in future courses. Through two preliminary case studies, we describe our experiences, and discuss advantages and disadvantages of the use of VR in this area.

  • Immersive visualisation of 3-dimensional spiking neural networks
    Stefan Marks

    Evolving Systems, ISSN: 18686478, eISSN: 18686486, Pages: 193-201, Published: 1 September 2017 Springer Science and Business Media LLC
    Recent development in artificial neural networks has led to an increase in performance, but also in complexity and size. This poses a significant challenge for the exploration and analysis of the spatial structure and temporal behaviour of such networks. Several projects for the 3D visualisation of neural networks exist, but they focus largely on the exploration of the spatial structure alone, and are using standard 2D screens as output and mouse and keyboard as input devices. In this article, we present NeuVis, a framework for an intuitive and immersive 3D visualisation of spiking neural networks in virtual reality, allowing for a larger variety of input and output devices. We apply NeuVis to NeuCube, a 3-dimensional spiking neural network learning framework, significantly improving the user’s abilities to explore, analyse, and also debug the network. Finally, we discuss further venues of development and alternative render methods that are currently under development and will increase the visual accuracy and realism of the visualisation, as well as further extending its analysis and exploration capabilities.

  • Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications
    Nikola Kasabov, Nathan Matthew Scott, Enmei Tu, Stefan Marks, Neelava Sengupta, Elisa Capecci, Muhaini Othman, Maryam Gholami Doborjeh, Norhanifah Murli, Reggio Hartono, Josafath Israel Espinosa-Ramos, Lei Zhou, Fahad Bashir Alvi, Grace Wang, Denise Taylor, Valery Feigin, Sergei Gulyaev, Mahmoud Mahmoud, Zeng-Guang Hou, and Jie Yang

    Neural Networks, ISSN: 08936080, eISSN: 18792782, Pages: 1-14, Published: 1 June 2016 Elsevier BV
    The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include 'on the fly' new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.

  • Design of a virtual trainer for exergaming
    Lindsay Alexander Shaw, Romain Tourrel, Burkhard Claus Wunsche, Christof Lutteroth, Stefan Marks, and Jude Buckley

    ACM International Conference Proceeding Series, Volume: 01-05-February-2016, Published: 1 February 2016 ACM
    Exergames are becoming increasingly popular as a way of motivating people to exercise. However, merely adding exercise elements to a game may not achieve the desired level of motivation and long term adherence. By designing an exergame which takes into account the user's personality profile, the user's level of motivation to play the game and thus exercise may be increased. In this paper, we present an exergame using a virtual trainer system which can be customized for the personality of the user. The trainer system supports two modes: a competitive mode for players who are motivated by pushing themselves to beat an opponent, and a cooperative mode for players who enjoy working with another player to perform well. We conduct a brief pilot study to evaluate our virtual trainers in which participants' personalities are evaluated using the Sport Orientation Questionnaire. They then play three short sessions of the exergame: a control condition without a trainer system, and one for each of the two trainer system. Our initial results indicate that the training systems are highly motivating when matching the personality of the user, particularly for competitive individuals.

  • Creating creative technologists: Playing with(in) education
    Andy M. Connor, Stefan Marks, and Charles Walker

    Springer Series on Cultural Computing, ISSN: 21959056, eISSN: 21959064, Pages: 35-56, Published: 2015 Springer London
    The Bachelor of Creative Technologies (BCT) degree is offered by Colab, a unique academic unit at Auckland University of Technology. The unit is a research-teaching nexus or ‘collaboratory’ at the intersection of four existing schools (Art and Design, Communications and Media Studies, Computer and Mathematical Sciences and Engineering) in the Faculty of Design and Creative Technologies. The goal of Colab is to develop new experimental alliances, research collaborations and learning experiences across these overlapping disciplines. Its researchers, students and stakeholders are encouraged to imagine, construct and navigate rapidly changing social, economic, technological and career environments.

  • Challenges in virtual reality exergame design
    Conferences in Research and Practice in Information Technology Series, ISSN: 14451336, Volume: 162, Pages: 61-68, Published: 2015

  • An intuitive tangible game controller
    Jacques Foottit, Dave Brown, Stefan Marks, and Andy M. Connor

    ACM International Conference Proceeding Series, Volume: 02-03-December-2014, Published: 2 December 2014 ACM
    This paper outlines the development of a sensory feedback device providing a tangible interface for controlling digital environments, in this example a flight simulator, where the intention for the device is that it is relatively low cost, versatile and intuitive. Gesture based input allows for a more immersive experience, so rather than making the user feel like they are controlling an aircraft the intuitive interface allows the user to become the aircraft that is controlled by the movements of the user's hand. The movements are designed to allow a sense of immersion that would be difficult to achieve with an alternative interface. A vibrotactile based haptic feedback is incorporated in the device to further enhance the connection between the user and the game environment by providing immediate confirmation of game events. When used for navigating an aircraft simulator, this device invites playful action and thrill. It bridges new territory on portable, low cost solutions for haptic devices in gaming contexts.

  • Towards the holodeck: Fully immersive virtual reality visualisation of scientific and engineering data
    Stefan Marks, Javier E. Estevez, and Andy M. Connor

    ACM International Conference Proceeding Series, Volume: 19-21-November-2014, Pages: 42-47, Published: 19 November 2014 ACM Press
    In this paper, we describe the development and operating principles of an immersive virtual reality (VR) visualisation environment that is designed around the use of consumer VR headsets in an existing wide area motion capture suite. We present two case studies in the application areas of visualisation of scientific and engineering data. Each of these case studies utilise a different render engine, namely a custom engine for one case and a commercial game engine for the other. The advantages and appropriateness of each approach are discussed along with suggestions for future work.

  • Experimental study of steer-by-wire ratios and response curves in a simulated high speed vehicle
    Conferences in Research and Practice in Information Technology Series, ISSN: 14451336, Volume: 139, Pages: 123-124, Published: February 2013

  • An ethnographic study of a high cognitive load driving environment
    Conferences in Research and Practice in Information Technology Series, ISSN: 14451336, Volume: 139, Pages: 121-122, Published: February 2013

  • Using game engine technology for virtual environment teamwork training
    20th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG 2012 - Conference Proceedings, Issue: PART 1, Pages: 169-178, Published: 2012

  • Design and evaluation of a medical teamwork training simulator using consumer-level equipment
    S. Marks, J. Windsor and B. Wünsche

    Studies in Health Technology and Informatics, ISSN: 09269630, eISSN: 18798365, Volume: 173, Pages: 273-279, Published: 2012
    Virtual environments (VE) are increasingly used for teamwork training purposes, e.g., for medical teams. One shortcoming is lack of support for nonverbal communication channels, essential for teamwork. We address this issue by using an inexpensive webcam to track the user's head and using that data for controlling avatar head movement, thereby conveying head gestures and adding a nonverbal communication channel. In addition, navigation and orientation within the virtual environment is simplified. We present the design and evaluation of a simulation framework based on a game engine and consumer-level hardware and the results of two user studies showing, among other results, an improvement in the usability of the VE and in the perceived quality of realism and communication within the VE by using head tracking avatar and view control.

  • Head tracking based avatar control for virtual environment teamwork training
    GRAPP 2011 - Proceedings of the International Conference on Computer Graphics Theory and Applications, Pages: 257-269, Published: 2011

  • Evaluation of the effectiveness of head tracking for view and avatar control in virtual environments
    S. Marks, J. A. Windsor, and B. Wunsche

    International Conference Image and Vision Computing New Zealand, ISSN: 21512191, eISSN: 21512205, Published: 2010 IEEE
    Virtual environments (VE) are gaining in popularity and are increasingly used for teamwork training purposes, e.g., for medical teams. We have identified two shortcomings of modern VEs: First, nonverbal communication channels are essential for teamwork but are not supported well. Second, view control in VEs is usually done manually, requiring the user to learn the controls before being able to effectively use them. We address those two shortcomings by using an inexpensive webcam to track the user's head. The rotational movement is used to control the head movement of the user's avatar, thereby conveying head gestures and adding a nonverbal communication channel. The translational movement is used to control the view of the VE in an intuitive way. Our paper presents the results of a user study designed to investigate how well users were able to use our system's advantages.

  • Optimisation and comparison framework for monocular camera-based face tracking
    Stefan Marks, John Windsor, and Burkhard Wunsche

    2009 24th International Conference Image and Vision Computing New Zealand, IVCNZ 2009 - Conference Proceedings, Pages: 243-248, Published: 2009 IEEE
    Tracking the position and orientation of the human face with respect to a camera has valuable applications in human computer interaction (HCI). Examples are navigating through a virtual environment, controlling objects using head gestures, and enabling avatars in a virtual environment to reflect the user's behaviour. Tracking performance can be heavily influenced by environmental parameters. Developers and users of face tracking plugins without computer vision experience need guidelines how to optimise face tracking performance in real world set-ups and they need measures how environmental parameters influence the results. In this paper we develop a qualitative framework for determining ideal working conditions of face tracking algorithms. We apply our framework to a commercially available face tracking solution and present the results of this analysis.

  • Enhancing Virtual Environment-Based Surgical Teamwork Training With Non-Verbal Communication
    GRAPP 2009 - Proceedings of the 4th International Conference on Computer Graphics Theory and Applications, Pages: 361-366, Published: 2009

  • Evaluation of game engines for simulated clinical training
    New Zealand Computer Science Research Student Conference, NZCSRSC 2008 - Proceedings, Pages: 92-99, Published: 2008