Current scholarly interests are in models of complexity, simulation, imagination and art, and virtual and real spaces of nature, especially as sources for educational applications. My research agenda is a multi-dimensional investigation of game technology as both an aesthetic medium and as a scientific tool; this game technology is used to model the human-computer environment interaction relationships between environmental signals, emotional reactions, and learning. Research focus is on knowledge acquisition and creativity-mediated and supported actions with high fidelity information systems, educational simulations, virtual environments, virtual/augmented reality, immersive learning simulations, and serious games. Such games are used for informal learning, formal education, knowledge stores, collaboration, decision support, and creativity. Quantitative and causal models are used to explain the role of context in human perception, signal processing, memory, decision processing, and action in the real world. Applications of this research are relevant to the fields of simulation, educational technology, creativity, and robotics. Many of my grant applications have resulted in funding, and my publications and presentations include peer-reviewed journals and conferences. I plan to write and publish books, file for IP patents, and release commercial software with notable impact.
Dissertation Findings Overview:
My dissertation, defended and published in 2008, “Simulated Ecological Environments for Education (SEEE): A Tripartite Model Framework of HCI Design Parameters for Situational Learning in Virtual Environments,” (major dissertation advisor: Dr. Peter Brusilovsky), investigated the empirical inter-relationships between humans, computers, and the environment. My dissertation discussed the environmental factors that cause learning and creativity to occur, specifically the environmental factors with respect to computer systems and simulations. My research showed that the Real is better than the Virtual, and that there are priming, transfer, and reinforcement effects. These findings thus indicated the Virtual-Real-Virtual order is justified as best practice, and that increased visual fidelity resulted in significantly increased learning activity and knowledge gains, and that increased freedom of navigation showed a strong trend toward increased learning activity and knowledge gains.
In my dissertation, I analyzed system design factors of virtual environments used for learning. Employing a combination of ethnographic and empirical methods grounded in human-computer interaction and human factors information processing, I investigated the impact of visual fidelity and navigational freedom on children’s exploration and inquiry activities and learning outcomes. The main research aim was to decompose the simulation and user interface into design parameters that influence emotion, attention, curiosity, information-seeking behavior, knowledge acquisition, learning, and creativity. First, I built a high fidelity photo-realistic simulation, “The Virtual Trillium Trail,” for which I could control, isolate, and test design factors. Then, I conducted an ethnographic and empirical analysis of users engaged in tasks in-situ, comparing and contrasting the Real Environment (control) to the Virtual Environment (experimental). Next, I investigated the system design parameters, using a planned orthogonal contrast (one system with four system states required to have internal empirical validity), and used a 2x2 ANOVA statistical test on the factors of Visual Fidelity (set to Low and High) and Navigational Freedom (set to Low and High).
The contribution of my dissertation provided a new approach to the question of learning in virtual reality, not from the long-standing educational pedagogy or philosophical paradigm of “immersion” or “presence,” nor the new “gamification” approach of serious games. Instead, I focused on the design factors in the system, isolated those factors, measured their impact, and reported the results. I adopted a human-computer environment interaction and a human information processing view, rather than “educational systems to meet the opportunities and challenges of the 21st century” perspective. I also illustrated that the historical focus on the “educational outcomes and standards” left relevant human-computer interaction and system design factors understudied. This approach brought the focus of students’ needs, teachers’ abilities, knowledge stores, context/environment, and the factors of the computer system—such as visual fidelity and navigational freedom—back into the analysis, thereby opening a new direction for an empirical study of the child learning in context, both real and virtual.
My research has made significant theoretical contributions at this intersection of the simulation, the user interface, and the learner. These contributions occur on five fronts:
1. Technical and Research Design Matters: Technical Contribution of the Virtual Trillium Trail as Data Generated Virtual Ecologies for Virtual Learning Environments and as a Planned Orthogonal Contrast Statistical Framework” documented how to build data visualization of ecological information (terrain data, satellite images, plant population data, and land gathered data) in real-time, 3D graphical, interactive platforms. Such platforms included high fidelity, photo-realistic game engines. The study also underscored the importance of software systems design choices, as they will impact statistical tests and thus conclusions. The main empirical contribution produced by the interaction of the system design (POC 2x2 ANOVA research design) was to isolate the factors of Visual Fidelity and Navigational Freedom, and to measure the impacts on output variables, Knowledge Gained (i.e., the difference between pre and post tests) and Salient Events (i.e., count the number of times there is a change in learning behavior from exploration-mode to inquiry-mode).
2. Reality Matters: Comparison of Real and Virtual Learning Environments: Ethnographic and empirical evidence prove the transfer of knowledge from Real to Virtual and Virtual to Real, thus demonstrating how to use virtual environments for priming, transfer, and reinforcement for maximum learning gain. These conclusions indicate the Virtual-Real-Virtual order is the best practice. I proved the Real Environment is superior to the Virtual Environment for learning activity, but when the Virtual is identical to and compared to the subset of the Real, learning activity outcomes are the same (i.e., for real world data, Real is superior, but for plant-only data, Real and Virtual are identical).
3. Gender Differences in Information Seeking and Learning Behavior of Young Children in Virtual Learning Environments: I showed that boys and girls learn differently in their information-seeking behavior and activity, and discovered one gender-neutral learning profile that is highly effective. I then examined its implications for both software design choices and teaching methods and concluded that girls benefit from High Visual Fidelity, but they are penalized by Low Visual Fidelity interfaces.
4. Visual Fidelity and Navigational Freedom and as Design Factors: There is significant interaction, ( F(1,60) = 4.85, p = 0.0315), between Visual Fidelity and Navigational Freedom, as design factors in virtual reality or games used as educational simulations, thus proving that both of these factors must be present to have the greatest impact on changes in test scores, as measured in the dependent variable, Knowledge Gained. Visual Fidelity is strong and significant (F(1,60) = 10.54, p = 0.0019) as a factor, and Navigational Freedom shows a trend, (F(1.60) = 2.71, p = 0.105). The combined conditions of both High Visual Fidelity and High Navigational Freedom result in far superior Knowledge Gained on tests, (cell mean = 37.44, SD = 13.88) when compared to the Low Visual Fidelity x Low Navigational Freedom conditions (cell mean = 20.93, SD = 13.36), and an unpaired t-test (t = 3.4280, p = 0.0018) is significant.
There is no interaction, (F(1,60) = 1.48, p = 0.2285), between Visual Fidelity and Navigational Freedom, as design factors in virtual reality used as educational systems, thus proving independence of these two factors on Salient Events. Salient Events measure the number of times student behavior changes from exploration to deep inquiry. Visual Fidelity is strong and significant, (F(1,60) = 4.35, p = 0.00413), proving to be a critical design factor for increased learning activity in virtual reality or games used as educational simulations. It alone, is responsible for significantly increasing learning activity. Navigational Freedom, as a factor, shows a strong trend, (F(1,60) = 3.23, p = 0.0773. The data show that the High Visual Fidelity condition (Row Mean = 14.46, SD = 6) resulted in more Salient Event counts than did the Low Visual Fidelity condition (Row Mean=11.31, SD = 6.37). The more a virtual reality environment, simulation, or serious game looks high fidelity and photo-realistic, the more times a child’s behavior will change from exploration to inquiry. Thus, Visual Fidelity increases a child’s desire to learn, to understand, and to stop in order to independently and actively inquire.
A design choice, High Visual Fidelity, has the strongest impact on this change of information-seeking behavior, indicative of a desire to learn, curiosity. Visual Fidelity is a powerful, significant, and independent factor affecting Salient Events. Additionally, Salient Events are significantly higher in the High Visual Fidelity x High Navigational Freedom conditions (cell mean =16.75, SD = 6.27) when compared to the Low Visual Fidelity x Low Navigational Freedom conditions (cell mean = 10.87, SD = 5.91). In an unpaired t-test (t = 2.7297, p = 0.0105), the results appear strong and significant. High Visual Fidelity and High Navigational Freedom conditions increase Salient Events (which shows a very strong trend at p = 0.05, or is significant at a p value of 0.10), and are thus critical design features for virtual reality or games used as educational simulations, especially since Salient Events are moderately positively correlated with Knowledge Gained (rho = 0.455, p = 0.000) in the SEEE Model.
These findings have significant impacts on both cartoon-only educational applications and film with a linear navigation, in that both forms of media are inferior to a photo-realistic virtual environment with an open, 360 degrees of freedom to explore.
5. Simulated Ecological Environments for Education (SEEE) Model: This model launched an investigation into the empirical models with the data generated by the simulations in order to understand the causal models, and to gauge the effect of the design factors of complex user interfaces in these virtual environments on human thought and action. Future research—in the form of regression equations and Markov models—will investigate the complexity and causality of such interactions between the child’s mental model, the virtual environment, and the user interface.
The original framing of the research question and the literature review presented at SIGGRAPH 2006.
Harrington, Maria, C. R. (2006). Situational learning in real and virtual space: Lessons learned and future directions. ACM SIGGRAPH’06. July 30-August 3, 2006 Boston, MA, USA. ISBN:1-59593-364-6 doi: 10.1145/1179295.1179344