Stefan Marks

Verified email at

Auckland University of Technology



XR, data visualisation


Scopus Publications

Scopus Publications

  • Discrete versus Continuous Colour Pickers Impact Colour Selection in Virtual Reality Art-Making
    Marylyn Alex, Danielle Lottridge, Jisu Lee, Stefan Marks, and Burkhard Wüensche

    PervasiveHealth: Pervasive Computing Technologies for Healthcare, ISSN: 21531633, Pages: 158-169, Published: 2 December 2020 ACM
    Colour selection is an important task in digital art and 3D modelling applications. Most colour pickers are based on continuous colour spaces or a representative sampling of them, such as the Munsell colour palette. Continuous colour space-based pickers enable users to select from all colours by displaying full saturation hues with options to lower saturation and modify value. The two-step process of colour selection from continuous pickers requires understanding of 3D colour space, e.g., where to find “brown” or “sand”. In this research we investigate how continuous versus discrete pickers affect colour selection in virtual art. We compared an HSV picker with a discrete picker in a study with 40 participants aged 16 - 60. We found that the colour picker impacted the kinds of colours used in artworks, with significant differences in colour distribution characteristics. We discuss implications of colour selection tools for virtual reality art-making.

  • Development of a virtual construction approach for steel structures considering structural and nonstructural elements, and installation equipment
    Proceedings of the International Conference of Architectural Science Association, eISSN: 22093850, Volume: 2020-November, Pages: 405-414, Published: 2020

  • Active Learning and Teaching through Digital Technology and Live Performance: ‘Choreographic Thinking’ as Art Practice in the Tertiary Sector
    Jennifer R. A. Nikolai, Gregory Bennett, Stefan Marks, and Geoff Gilson

    International Journal of Art and Design Education, ISSN: 14768062, eISSN: 14768070, Pages: 137-152, Published: Februaryy 2019 Wiley

  • Evaluation of a Virtual Reality Nasal Cavity Education Tool
    Stefan Marks, David White, and Milan Mazdics

    Proceedings of 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering, TALE 2018, Pages: 193-198, Published: 16 January 2019 IEEE
    In this paper, we present a user study to evaluate the suitability of a virtual reality (VR) application for teaching the anatomy of the human nasal cavity. We discuss the benefits of VR technology in the domain of anatomy education and present our application and the user study. Our results seem to indicate a disadvantage of the VR application with respect to assessment results, but also raise questions about the nature of assessment of spatial knowledge. With respect to usability and motivation, VR demonstrates advantages over “traditional” teaching material.

  • From von Neumann architecture and Atanasoffs ABC to neuro-morphic computation and Kasabov’s neuCube: Principles and implementations
    Neelava Sengupta, Josafath Israel Espinosa Ramos, Enmei Tu, Stefan Marks, Nathan Scott, Jakub Weclawski, Akshay Raj Gollahalli, Maryam Gholami Doborjeh, Zohreh Gholami Doborjeh, Kaushalya Kumarasinghe, Vivienne Breen, and Anne Abbott

    Studies in Computational Intelligence, ISSN: 1860949X, Volume: 756, Pages: 1-28, Published: 2018 Springer International Publishing
    During the 1940s John Atanasoff with the help of one of his students Clifford E. Berry, at Iowa State College, created the ABC (Atanasoff-Berry Computer) that was the first electronic digital computer. The ABC computer was not a general-purpose one, but still, it was the first to implement three of the most important ideas used in computers nowadays: binary data representation; using electronics instead of mechanical switches and wheels; using a von Neumann architecture, where the memory and the computations are separated. A new computational paradigm, named as Neuromorphic, utilises the above two principles, but instead of the von Neumann principle, it integrates the memory and the computation in a single module a spiking neural network structure. This chapter first reviews the principles of the earlier published work by the team on neuromorphic computational architecture NeuCube. NeuCube is not a general purpose machine but is still the first neuromorphic spatio/spectro-temporal data machine for learning, pattern recognition and understanding of spatio/spectro-temporal data. The chapter further presents the software/hardware implementation of the NeuCube as a development system for efficient applications on temporal or spatio/spectro-temporal across domain areas, including: brain data (EEG, fMRI), brain computer interfaces, robot control, multi-sensory data modelling, seismic stream data modelling and earthquake prediction, financial time series forecasting, climate data modelling and personalised, on-line risk of stroke prediction, and others. A limited version of the NeuCube software implementation is available from

  • Getting up your nose: a virtual reality education tool for nasal cavity anatomy
    Stefan Marks, David White, and Manpreet Singh

    SIGGRAPH Asia 2017 Symposium on Education, SA 2017, Published: 27 November 2017 ACM
    This article explores the application of virtual reality (VR) to the area of anatomical education, specifically the shape of and the airflow through the human nasal cavity. We argue the benefits of VR technology in this specific domain, and describe the creation of the VR application which is intended to be used in future courses. Through two preliminary case studies, we describe our experiences, and discuss advantages and disadvantages of the use of VR in this area.

  • Immersive visualisation of 3-dimensional spiking neural networks
    Stefan Marks

    Evolving Systems, ISSN: 18686478, eISSN: 18686486, Pages: 193-201, Published: 1 September 2017 Springer Science and Business Media LLC
    Recent development in artificial neural networks has led to an increase in performance, but also in complexity and size. This poses a significant challenge for the exploration and analysis of the spatial structure and temporal behaviour of such networks. Several projects for the 3D visualisation of neural networks exist, but they focus largely on the exploration of the spatial structure alone, and are using standard 2D screens as output and mouse and keyboard as input devices. In this article, we present NeuVis, a framework for an intuitive and immersive 3D visualisation of spiking neural networks in virtual reality, allowing for a larger variety of input and output devices. We apply NeuVis to NeuCube, a 3-dimensional spiking neural network learning framework, significantly improving the user’s abilities to explore, analyse, and also debug the network. Finally, we discuss further venues of development and alternative render methods that are currently under development and will increase the visual accuracy and realism of the visualisation, as well as further extending its analysis and exploration capabilities.

  • Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications
    Nikola Kasabov, Nathan Matthew Scott, Enmei Tu, Stefan Marks, Neelava Sengupta, Elisa Capecci, Muhaini Othman, Maryam Gholami Doborjeh, Norhanifah Murli, Reggio Hartono, Josafath Israel Espinosa-Ramos, Lei Zhou, Fahad Bashir Alvi, Grace Wang, Denise Taylor, Valery Feigin, Sergei Gulyaev, Mahmoud Mahmoud, Zeng-Guang Hou, and Jie Yang

    Neural Networks, ISSN: 08936080, eISSN: 18792782, Pages: 1-14, Published: 1 June 2016 Elsevier BV
    The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include 'on the fly' new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.

  • Design of a virtual trainer for exergaming
    Lindsay Alexander Shaw, Romain Tourrel, Burkhard Claus Wunsche, Christof Lutteroth, Stefan Marks, and Jude Buckley

    ACM International Conference Proceeding Series, Volume: 01-05-February-2016, Published: 1 February 2016 ACM
    Exergames are becoming increasingly popular as a way of motivating people to exercise. However, merely adding exercise elements to a game may not achieve the desired level of motivation and long term adherence. By designing an exergame which takes into account the user's personality profile, the user's level of motivation to play the game and thus exercise may be increased. In this paper, we present an exergame using a virtual trainer system which can be customized for the personality of the user. The trainer system supports two modes: a competitive mode for players who are motivated by pushing themselves to beat an opponent, and a cooperative mode for players who enjoy working with another player to perform well. We conduct a brief pilot study to evaluate our virtual trainers in which participants' personalities are evaluated using the Sport Orientation Questionnaire. They then play three short sessions of the exergame: a control condition without a trainer system, and one for each of the two trainer system. Our initial results indicate that the training systems are highly motivating when matching the personality of the user, particularly for competitive individuals.

  • Creating creative technologists: Playing with(in) education
    Andy M. Connor, Stefan Marks, and Charles Walker

    Springer Series on Cultural Computing, ISSN: 21959056, eISSN: 21959064, Pages: 35-56, Published: 2015 Springer London
    The Bachelor of Creative Technologies (BCT) degree is offered by Colab, a unique academic unit at Auckland University of Technology. The unit is a research-teaching nexus or ‘collaboratory’ at the intersection of four existing schools (Art and Design, Communications and Media Studies, Computer and Mathematical Sciences and Engineering) in the Faculty of Design and Creative Technologies. The goal of Colab is to develop new experimental alliances, research collaborations and learning experiences across these overlapping disciplines. Its researchers, students and stakeholders are encouraged to imagine, construct and navigate rapidly changing social, economic, technological and career environments.

  • Challenges in virtual reality exergame design
    Conferences in Research and Practice in Information Technology Series, ISSN: 14451336, Volume: 162, Pages: 61-68, Published: 2015

  • An intuitive tangible game controller
    Jacques Foottit, Dave Brown, Stefan Marks, and Andy M. Connor

    ACM International Conference Proceeding Series, Volume: 02-03-December-2014, Published: 2 December 2014 ACM
    This paper outlines the development of a sensory feedback device providing a tangible interface for controlling digital environments, in this example a flight simulator, where the intention for the device is that it is relatively low cost, versatile and intuitive. Gesture based input allows for a more immersive experience, so rather than making the user feel like they are controlling an aircraft the intuitive interface allows the user to become the aircraft that is controlled by the movements of the user's hand. The movements are designed to allow a sense of immersion that would be difficult to achieve with an alternative interface. A vibrotactile based haptic feedback is incorporated in the device to further enhance the connection between the user and the game environment by providing immediate confirmation of game events. When used for navigating an aircraft simulator, this device invites playful action and thrill. It bridges new territory on portable, low cost solutions for haptic devices in gaming contexts.

  • Towards the holodeck: Fully immersive virtual reality visualisation of scientific and engineering data
    Stefan Marks, Javier E. Estevez, and Andy M. Connor

    ACM International Conference Proceeding Series, Volume: 19-21-November-2014, Pages: 42-47, Published: 19 November 2014 ACM Press
    In this paper, we describe the development and operating principles of an immersive virtual reality (VR) visualisation environment that is designed around the use of consumer VR headsets in an existing wide area motion capture suite. We present two case studies in the application areas of visualisation of scientific and engineering data. Each of these case studies utilise a different render engine, namely a custom engine for one case and a commercial game engine for the other. The advantages and appropriateness of each approach are discussed along with suggestions for future work.

  • Experimental study of steer-by-wire ratios and response curves in a simulated high speed vehicle
    Conferences in Research and Practice in Information Technology Series, ISSN: 14451336, Volume: 139, Pages: 123-124, Published: February 2013

  • An ethnographic study of a high cognitive load driving environment
    Conferences in Research and Practice in Information Technology Series, ISSN: 14451336, Volume: 139, Pages: 121-122, Published: February 2013

  • Using game engine technology for virtual environment teamwork training
    20th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, WSCG 2012 - Conference Proceedings, Issue: PART 1, Pages: 169-178, Published: 2012

  • Design and evaluation of a medical teamwork training simulator using consumer-level equipment
    S. Marks, J. Windsor and B. Wünsche

    Studies in Health Technology and Informatics, ISSN: 09269630, eISSN: 18798365, Volume: 173, Pages: 273-279, Published: 2012
    Virtual environments (VE) are increasingly used for teamwork training purposes, e.g., for medical teams. One shortcoming is lack of support for nonverbal communication channels, essential for teamwork. We address this issue by using an inexpensive webcam to track the user's head and using that data for controlling avatar head movement, thereby conveying head gestures and adding a nonverbal communication channel. In addition, navigation and orientation within the virtual environment is simplified. We present the design and evaluation of a simulation framework based on a game engine and consumer-level hardware and the results of two user studies showing, among other results, an improvement in the usability of the VE and in the perceived quality of realism and communication within the VE by using head tracking avatar and view control.

  • Head tracking based avatar control for virtual environment teamwork training
    GRAPP 2011 - Proceedings of the International Conference on Computer Graphics Theory and Applications, Pages: 257-269, Published: 2011

  • Evaluation of the effectiveness of head tracking for view and avatar control in virtual environments
    S. Marks, J. A. Windsor, and B. Wunsche

    International Conference Image and Vision Computing New Zealand, ISSN: 21512191, eISSN: 21512205, Published: 2010 IEEE
    Virtual environments (VE) are gaining in popularity and are increasingly used for teamwork training purposes, e.g., for medical teams. We have identified two shortcomings of modern VEs: First, nonverbal communication channels are essential for teamwork but are not supported well. Second, view control in VEs is usually done manually, requiring the user to learn the controls before being able to effectively use them. We address those two shortcomings by using an inexpensive webcam to track the user's head. The rotational movement is used to control the head movement of the user's avatar, thereby conveying head gestures and adding a nonverbal communication channel. The translational movement is used to control the view of the VE in an intuitive way. Our paper presents the results of a user study designed to investigate how well users were able to use our system's advantages.

  • Optimisation and comparison framework for monocular camera-based face tracking
    Stefan Marks, John Windsor, and Burkhard Wunsche

    2009 24th International Conference Image and Vision Computing New Zealand, IVCNZ 2009 - Conference Proceedings, Pages: 243-248, Published: 2009 IEEE
    Tracking the position and orientation of the human face with respect to a camera has valuable applications in human computer interaction (HCI). Examples are navigating through a virtual environment, controlling objects using head gestures, and enabling avatars in a virtual environment to reflect the user's behaviour. Tracking performance can be heavily influenced by environmental parameters. Developers and users of face tracking plugins without computer vision experience need guidelines how to optimise face tracking performance in real world set-ups and they need measures how environmental parameters influence the results. In this paper we develop a qualitative framework for determining ideal working conditions of face tracking algorithms. We apply our framework to a commercially available face tracking solution and present the results of this analysis.

  • Enhancing Virtual Environment-Based Surgical Teamwork Training With Non-Verbal Communication
    GRAPP 2009 - Proceedings of the 4th International Conference on Computer Graphics Theory and Applications, Pages: 361-366, Published: 2009

  • Evaluation of game engines for simulated clinical training
    New Zealand Computer Science Research Student Conference, NZCSRSC 2008 - Proceedings, Pages: 92-99, Published: 2008

  • Camera based face tracking for enhancing surgical teamwork training with non-verbal communication
    S. Marks, J. Windsor, and B. Wunsche

    2008 23rd International Conference Image and Vision Computing New Zealand, IVCNZ, Published: 2008 IEEE
    In recent years, the increased use of simulation for the training of surgical skills has improved the medical curriculum and the overall patient outcome. Advances in hardware and simulation techniques have resulted in many commercial applications for training technical skills. However, most of these simulators are extremely expensive and do not consider non-technical skills like teamwork and communication. This is a major drawback since recent research suggests that a a large percentage of mistakes in clinical settings are due to communication problems. In addition training teamwork can also improve the efficiency of a surgical team and as such reduce costs and workload. We present an inexpensive camera-based system for the acquisition of aspects of non-verbal communication of users participating in virtual environment-based teamwork simulations. This data can be used for the enhancement of virtual-environment-based simulations to increase the realism and effectiveness of team communication.

  • Evaluation of game engines for simulated surgical training
    Stefan Marks, John Windsor, and Burkhard Wünsche

    Proceedings - GRAPHITE 2007, 5th International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, Pages: 273-280, Published: 2007 ACM Press
    The increasing complexity and costs of surgical training and the constant development of new surgical procedures has made virtual surgical training an essential tool in medical education. Unfortunately, commercial tools are very expensive and have a small support base. Game engines offer unique advantages for the creation of highly interactive and collaborative environments. This paper examines the suitability of currently available game engines for developing applications for medical education and simulated surgical training. We formally evaluate a list of available game engines for stability, availability, the possibility of custom content creation and the interaction of multiple users via a network. Based on these criteria, three of the highest ranked engines are used for further case studies. We found that in general it is possible to easily create scenarios with custom medical models that can be cooperatively viewed and interacted with. Limitations in physical simulation capabilities make some engines unsuitable for fully interactive applications, but they can be used in combination with predefined animations. We show that overall game engines represent a good foundation for low cost virtual surgery applications and we discuss technologies which can be used to further extend their physical simulation capabilities.