Stefan Marks
@aut.ac.nz
Auckland University of Technology
RESEARCH INTERESTS
XR, data visualisation
Scopus Publications
Scopus Publications
Stefan Marks and Sebastián Gil Parga
ACM
This paper describes the challenges and solutions to teaching computer graphics as well as extended reality concepts to students from a variety of backgrounds in the context of the School of Future Environments at the Auckland University of Technology, New Zealand. Examples are provided for the content and assessment strategies for two courses, as well as a summary of student work and feedback collected over the last three years.
Sebastian Gil Parga, Umang Singh, Jairo Gutierrez, and Stefan Marks
Informa UK Limited
Julia Ayache, Andrew M. Connor, Stefan Marks, Alexander Sumich, and Nadja Heym
ACM
Predictability is a core aspect of human-computer interaction (HCI), but an excess of predictability can lead to stereotypical behaviors and decreases the perceived agentivity (i.e., anthropomorphism) of a virtual agent. Yet, it remains unclear if inter-individual variability in predicting the behavior of other is modulating tendencies for anthropomorphism. The present study investigated the interaction between Theory of Mind (ToM) abilities and agent predictability and their association with anthropomorphism. Participants (N=38) completed self-reports of Perspective Taking capabilities and played the Matching Pennies Game with a virtual agent. The agent’s predictability was manipulated across four conditions, and participants reported their perception of the agent’s humanness and predictability. Results revealed that perceived predictability rather than self-reported Perspective Taking capabilities were positively associated with ToM abilities in predicting virtual agent’s behavior. However, increasing the virtual agent’s predictability decreased tendencies for anthropomorphism, stressing the role of randomness in perceiving humanness.
Jan Kruse, Andy M. Connor, and Stefan Marks
Association for Computing Machinery (ACM)
Designing games is a complicated and time-consuming process, where developing new levels for existing games can take weeks. Procedural content generation offers the potential to shorten this timeframe, however, automated design tools are not adopted widely in the game industry. This article presents an expert evaluation of a human-in-the-loop generative design approach for commercial game maps that incorporates multiple computational agents. The evaluation aims to gauge the extent to which such an approach could support and be accepted by human game designers and to determine whether the computational agents improve the overall design. To evaluate the approach, 11 game designers utilized the approach to design game levels with the computational agents both active and inactive. Eye-tracking, observational, and think-aloud data was collected to determine whether designers favored levels suggested by the computational agents. This data was triangulated with qualitative data from semi-structured interviews that were used to gather overall opinions of the approach. The eye-tracking data indicates that the participating game level designers showed a clear preference for levels suggested by the computational agents, however, expert designers in particular appeared to reject the idea that the computational agents are helpful. The perception of computational tools not being useful needs to be addressed if procedural content generation approaches are to fulfill their potential for the game industry.
Julia Ayache, Andy Connor, Stefan Marks, Daria J. Kuss, Darren Rhodes, Alexander Sumich, and Nadja Heym
Frontiers Media SA
Interpersonal coordination is a research topic that has attracted considerable attention this last decade both due to a theoretical shift from intra-individual to inter-individual processes and due to the development of new methods for recording and analyzing movements in ecological settings. Encompassing spatiotemporal behavioral matching, interpersonal coordination is considered as “social glue” due to its capacity to foster social bonding. However, the mechanisms underlying this effect are still unclear and recent findings suggest a complex picture. Goal-oriented joint action and spontaneous coordination are often conflated, making it difficult to disentangle the role of joint commitment from unconscious mutual attunement. Consequently, the goals of the present article are twofold: (1) to illustrate the rapid expansion of interpersonal coordination as a research topic and (2) to conduct a systematic review of spontaneous interpersonal coordination, summarizing its latest developments and current challenges this last decade. By applying Rapid Automatic Keyword Extraction and Latent Dirichlet Allocation algorithms, keywords were extracted from PubMed and Scopus databases revealing the large diversity of research topics associated with spontaneous interpersonal coordination. Using the same databases and the keywords “behavioral matching,” “interactional synchrony,” and “interpersonal coordination,” 1,213 articles were identified, extracted, and screened following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses protocol. A total of 19 articles were selected using the following inclusion criteria: (1) dynamic and spontaneous interactions between two unacquainted individuals (2) kinematic analyses, and (3) non-clinical and non-expert adult populations. The results of this systematic review stress the proliferation of various definitions and experimental paradigms that study perceptual and/or social influences on the emergence of spontaneous interpersonal coordination. As methods and indices used to quantify interpersonal coordination differ from one study to another, it becomes difficult to establish a coherent picture. This review highlights the need to reconsider interpersonal coordination not as the pinnacle of social interactions but as a complex dynamical process that requires cautious interpretation. An interdisciplinary approach is necessary for building bridges across scattered research fields through opening a dialogue between different theoretical frameworks and consequently provides a more ecological and holistic understanding of human social cognition.
Marylyn Alex, Danielle Lottridge, Jisu Lee, Stefan Marks, and Burkhard Wüensche
ACM
Colour selection is an important task in digital art and 3D modelling applications. Most colour pickers are based on continuous colour spaces or a representative sampling of them, such as the Munsell colour palette. Continuous colour space-based pickers enable users to select from all colours by displaying full saturation hues with options to lower saturation and modify value. The two-step process of colour selection from continuous pickers requires understanding of 3D colour space, e.g., where to find “brown” or “sand”. In this research we investigate how continuous versus discrete pickers affect colour selection in virtual art. We compared an HSV picker with a discrete picker in a study with 40 participants aged 16 - 60. We found that the colour picker impacted the kinds of colours used in artworks, with significant differences in colour distribution characteristics. We discuss implications of colour selection tools for virtual reality art-making.
Stefan Marks and David White
ACM
We present a multi-device collaboration principle for virtual environments, using a combination of virtual and augmented reality (VR/AR) technology, used in the context of two educational applications, a virtual nasal cavity, and a visualisation of earthquake data. A head-mounted display (HMD) and a 3D-tracked tablet create two views of a shared virtual space. This allows two users to collaborate, utilising the strengths of each of the two technologies, e.g., intuitive spatial navigation and interaction in VR, and touch control of the visualisation parameters via the AR tablet. Touch gestures on the tablet are translated into a pointer ray in VR, so the users can easily indicate spatial features. The underlying networking infrastructure allows for an extension of this application to more than two users and across different rendering platforms.
Yemon Lee, Stefan Marks, and Andy M. Connor
ACM
This research is an exploratory study that evaluates the potential for using a three-dimensional (3D) virtual reality headset in air traffic control scenarios by considering whether they offer advantages in identifying potential in-flight collisions in comparison to traditional 2-dimensional (2D) displays. Presenting large volumes of data on 2D displays may limit speed and efficiency of air traffic control work. By comparison, virtual reality (VR) allows users to experience immersion within a virtual environment which facilitates different modes of interaction with large and complex datasets. Fifteen participants were involved in this explorative study, none of whom were trained air traffic controllers. Each participant observed a number of simulated flight scenarios using both a 2D display and a 3D VR headset. A combination of quantitative and qualitative data was collected using simulation event logs and post-observation questionnaires. The quantitative data from the simulation logs generally shows that potential collisions are detected more quickly using VR. Despite this, participants did not feel as able to detect potential collisions using virtual reality.
Jennifer R. A. Nikolai, Gregory Bennett, Stefan Marks, and Geoff Gilson
Wiley
Stefan Marks, David White, and Milan Mazdics
IEEE
In this paper, we present a user study to evaluate the suitability of a virtual reality (VR) application for teaching the anatomy of the human nasal cavity. We discuss the benefits of VR technology in the domain of anatomy education and present our application and the user study. Our results seem to indicate a disadvantage of the VR application with respect to assessment results, but also raise questions about the nature of assessment of spatial knowledge. With respect to usability and motivation, VR demonstrates advantages over “traditional” teaching material.
Neelava Sengupta, Josafath Israel Espinosa Ramos, Enmei Tu, Stefan Marks, Nathan Scott, Jakub Weclawski, Akshay Raj Gollahalli, Maryam Gholami Doborjeh, Zohreh Gholami Doborjeh, Kaushalya Kumarasinghe,et al.
Springer International Publishing
During the 1940s John Atanasoff with the help of one of his students Clifford E. Berry, at Iowa State College, created the ABC (Atanasoff-Berry Computer) that was the first electronic digital computer. The ABC computer was not a general-purpose one, but still, it was the first to implement three of the most important ideas used in computers nowadays: binary data representation; using electronics instead of mechanical switches and wheels; using a von Neumann architecture, where the memory and the computations are separated. A new computational paradigm, named as Neuromorphic, utilises the above two principles, but instead of the von Neumann principle, it integrates the memory and the computation in a single module a spiking neural network structure. This chapter first reviews the principles of the earlier published work by the team on neuromorphic computational architecture NeuCube. NeuCube is not a general purpose machine but is still the first neuromorphic spatio/spectro-temporal data machine for learning, pattern recognition and understanding of spatio/spectro-temporal data. The chapter further presents the software/hardware implementation of the NeuCube as a development system for efficient applications on temporal or spatio/spectro-temporal across domain areas, including: brain data (EEG, fMRI), brain computer interfaces, robot control, multi-sensory data modelling, seismic stream data modelling and earthquake prediction, financial time series forecasting, climate data modelling and personalised, on-line risk of stroke prediction, and others. A limited version of the NeuCube software implementation is available from http://www.kedri.aut.ac.nz/neucube/.
Stefan Marks, David White, and Manpreet Singh
ACM
This article explores the application of virtual reality (VR) to the area of anatomical education, specifically the shape of and the airflow through the human nasal cavity. We argue the benefits of VR technology in this specific domain, and describe the creation of the VR application which is intended to be used in future courses. Through two preliminary case studies, we describe our experiences, and discuss advantages and disadvantages of the use of VR in this area.
Stefan Marks
Springer Science and Business Media LLC
Recent development in artificial neural networks has led to an increase in performance, but also in complexity and size. This poses a significant challenge for the exploration and analysis of the spatial structure and temporal behaviour of such networks. Several projects for the 3D visualisation of neural networks exist, but they focus largely on the exploration of the spatial structure alone, and are using standard 2D screens as output and mouse and keyboard as input devices. In this article, we present NeuVis, a framework for an intuitive and immersive 3D visualisation of spiking neural networks in virtual reality, allowing for a larger variety of input and output devices. We apply NeuVis to NeuCube, a 3-dimensional spiking neural network learning framework, significantly improving the user’s abilities to explore, analyse, and also debug the network. Finally, we discuss further venues of development and alternative render methods that are currently under development and will increase the visual accuracy and realism of the visualisation, as well as further extending its analysis and exploration capabilities.
Nikola Kasabov, Nathan Matthew Scott, Enmei Tu, Stefan Marks, Neelava Sengupta, Elisa Capecci, Muhaini Othman, Maryam Gholami Doborjeh, Norhanifah Murli, Reggio Hartono,et al.
Elsevier BV
The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include 'on the fly' new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.
Lindsay Alexander Shaw, Romain Tourrel, Burkhard Claus Wunsche, Christof Lutteroth, Stefan Marks, and Jude Buckley
ACM
Exergames are becoming increasingly popular as a way of motivating people to exercise. However, merely adding exercise elements to a game may not achieve the desired level of motivation and long term adherence. By designing an exergame which takes into account the user's personality profile, the user's level of motivation to play the game and thus exercise may be increased. In this paper, we present an exergame using a virtual trainer system which can be customized for the personality of the user. The trainer system supports two modes: a competitive mode for players who are motivated by pushing themselves to beat an opponent, and a cooperative mode for players who enjoy working with another player to perform well. We conduct a brief pilot study to evaluate our virtual trainers in which participants' personalities are evaluated using the Sport Orientation Questionnaire. They then play three short sessions of the exergame: a control condition without a trainer system, and one for each of the two trainer system. Our initial results indicate that the training systems are highly motivating when matching the personality of the user, particularly for competitive individuals.
Andy M. Connor, Stefan Marks, and Charles Walker
Springer London
The Bachelor of Creative Technologies (BCT) degree is offered by Colab, a unique academic unit at Auckland University of Technology. The unit is a research-teaching nexus or ‘collaboratory’ at the intersection of four existing schools (Art and Design, Communications and Media Studies, Computer and Mathematical Sciences and Engineering) in the Faculty of Design and Creative Technologies. The goal of Colab is to develop new experimental alliances, research collaborations and learning experiences across these overlapping disciplines. Its researchers, students and stakeholders are encouraged to imagine, construct and navigate rapidly changing social, economic, technological and career environments.
Jacques Foottit, Dave Brown, Stefan Marks, and Andy M. Connor
ACM
This paper outlines the development of a sensory feedback device providing a tangible interface for controlling digital environments, in this example a flight simulator, where the intention for the device is that it is relatively low cost, versatile and intuitive. Gesture based input allows for a more immersive experience, so rather than making the user feel like they are controlling an aircraft the intuitive interface allows the user to become the aircraft that is controlled by the movements of the user's hand. The movements are designed to allow a sense of immersion that would be difficult to achieve with an alternative interface. A vibrotactile based haptic feedback is incorporated in the device to further enhance the connection between the user and the game environment by providing immediate confirmation of game events. When used for navigating an aircraft simulator, this device invites playful action and thrill. It bridges new territory on portable, low cost solutions for haptic devices in gaming contexts.
Stefan Marks, Javier E. Estevez, and Andy M. Connor
ACM Press
In this paper, we describe the development and operating principles of an immersive virtual reality (VR) visualisation environment that is designed around the use of consumer VR headsets in an existing wide area motion capture suite. We present two case studies in the application areas of visualisation of scientific and engineering data. Each of these case studies utilise a different render engine, namely a custom engine for one case and a commercial game engine for the other. The advantages and appropriateness of each approach are discussed along with suggestions for future work.
S. Marks, J. Windsor and B. Wünsche
Virtual environments (VE) are increasingly used for teamwork training purposes, e.g., for medical teams. One shortcoming is lack of support for nonverbal communication channels, essential for teamwork. We address this issue by using an inexpensive webcam to track the user's head and using that data for controlling avatar head movement, thereby conveying head gestures and adding a nonverbal communication channel. In addition, navigation and orientation within the virtual environment is simplified. We present the design and evaluation of a simulation framework based on a game engine and consumer-level hardware and the results of two user studies showing, among other results, an improvement in the usability of the VE and in the perceived quality of realism and communication within the VE by using head tracking avatar and view control.