Tracy Dorrington-Skinner, Victims of Institutional Childhood Exploitation Society (VOICES)
Gerald Morrison, Victims of Institutional Childhood Exploitation Society (VOICES)
Tony Smith, Victims of Institutional Childhood Exploitation Society (VOICES)
and
The DOHR Team
Abstract
Relational presence is the core principle of a new approach to designing virtual learning environments (VLEs), which has been developed by the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca). Presence, normally understood as the sense of being in a virtual environment to the extent that one forgets the environment is virtual, is thought to have significant pedagogical benefits in K–12 experiential learning projects aiming to develop spatial and social competencies that learners can translate into actual-world contexts. DOHR, by contrast, aims to build the understanding needed for learners to address systemic racism in Nova Scotia, through an oral history and restorative justice–based curriculum. To serve this alternative learning goal, relational presence replaces presence. The usual emphasis in VLE design on simulation, interactivity, identity construction, agency, and satisfaction is replaced with new values of impression, witnessing, self-awareness and awareness of difference, interpretation and inquiry, and affective dissonance. This paper introduces relational presence in order to help establish, in the field of VLE design, a productive discourse around issues of justice, representation of marginalized communities, and pedagogy-led design.
Introduction
This article introduces relational presence, the core principle of a new approach to designing virtual learning environments (VLEs) that has been developed by the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca). DOHR has worked in partnership with the Nova Scotia Home for Colored Children Restorative Inquiry (restorativeinquiry.ca). The Restorative Inquiry was a four-year, provincially-mandated public inquiry into the history and legacy of the Nova Scotia Home for Colored Children (NSHCC), including the lived experiences of its residents. The Home was a segregated care institution for African Nova Scotian children that operated in Dartmouth, Nova Scotia from 1921 until the early 2000s. Established to meet the care needs of African Nova Scotian children, the Home was a site of significant abuse and harm for many of its residents. Over the decades of its operations, former residents experienced neglect and abuse (Province of Nova Scotia 2019, 153–172). The Restorative Inquiry was established to examine the experience of the Nova Scotia Home for Colored Children in relation to systemic and institutionalized racism, both historic and current, in Nova Scotia. In order to “contribute to the goal of social change to end the harmful legacy of abuse and ensure the conditions, context and causes that contributed to it are not repeated” (Province of Nova Scotia 2015, 4–5), among its goals the Inquiry aimed to:
(a) Empower those involved in, and affected by, the history and legacy of the NSHCC to learn about what happened and the contexts, causes, circumstances and ongoing legacy of the harms related to the NSHCC.
(b) Educate the public about the history and legacy of the NSHCC.
(c) Publicly share the truth and understanding established through the RI and the actions taken, planned, and recommended to address systemic and institutionalized racism and build more just relationships for the future (Province of Nova Scotia 2019, 23).
The DOHR project was an important mechanism through which the Inquiry pursued this part of its mandate (Province of Nova Scotia 2019, 504–505). The DOHR project has brought former residents of the Home, representatives of the Nova Scotia education system, and members of the Inquiry’s Council of Parties together with artists and researchers from seven universities across Canada (Waterloo, Dalhousie, New Brunswick, McGill, Ottawa, Alberta, and British Columbia) to develop a two-week grade eleven Canadian History curriculum unit that supports students in learning about the historical harms experienced by former residents of the Home. In this way, it has served to support the mandate of education and the broader goal of moving toward reconciliation by building the understanding needed to address systemic racism in Nova Scotia.
DOHR is thus a community-driven project. It arises from a need articulated by a community and works to co-create the project with community members. This community mandate is central to the need for designing relational presence in the virtual reality (VR) experience. The Restorative Inquiry, from which the DOHR project was created, pursued a restorative vision of justice that was reflective of a relational worldview focused on connectedness. It sought justice in the form of just relations between individuals, groups, communities, and at the level of institutions and systems (Llewellyn 2011). As a restorative process, the Inquiry was “future focused, yet concerned with getting a comprehensive understanding of the past in order to know how to move forward toward a just future” (Province of Nova Scotia 2019, 26). It focused on learning about past harms in order to build more just relations going forward. This is, in simplistic terms, the impetus for a restorative approach to learning in the DOHR curriculum. A restorative approach, as DOHR members Jennifer Llewellyn and Kristina Llewellyn have articulated, is grounded in relational theory. Relational theory holds that human beings exist in and through relationship with one another (Llewellyn, J. 2011; Llewellyn and Llewellyn 2015). The DOHR project reflects the premise that relationality is at the core of learning about such difficult knowledge as systemic racism in the Home and its legacy. Learning requires attention to the fact that we exist in and through relations, and this fact has implications for justice. Recognizing the relational nature of the historical harms of the Home, requires that learners listen to the lived experiences of former residents.
The DOHR project therefore co-created, as part of its curriculum, a placed-based oral history experience in virtual reality, with three former residents—Gerry Morrison, Tony Smith, and Tracy Dorrington-Skinner—who are recognized leaders and activists in the community.[1] Scholars have demonstrated the many ways that oral history in education, both in conducting interviews and in listening to pre-recorded interviews, builds relational connections that are intergenerational and support reconciliation across divides (Llewellyn and Ng-A-Fook 2017; 2019). Unlike other oral history projects in schools, however, the DOHR project required that learners listen to stories in a contextual way that would connect them to a sense of place and the human experience of it—specifically, to the Home. Yet the DOHR team knew that former residents could not, nor should they be expected to, share their stories in-person with all students. The DOHR team also knew that not all students could visit the site of the Home and, even if it were possible, the site of the Home itself has changed significantly over the decades. While part of the Home’s building still stands, its present structure is considerably different from the structures in which the former residents lived (Morrison from 1954–60, Smith from 1965–68, and Dorrington-Skinner, who lived in the original Home building from 1972–78, and in the newer building now known as the Akoma Family Centre from 1978–84). Indeed, since early 2019, the site of the Old Home has been undergoing yet another phase of major renovations (see Figure 1). Since students cannot interact with the former residents or the site of the Home directly, the DOHR curriculum exemplifies the kind of experiential learning that is consistently identified in the VLE literature as likely to benefit from a virtual learning environment (VLE), and ideally one that is VR-based. DOHR wants to deliver experiential “learning tasks which are expensive” (Dalgarno and Lee 2010, 19) or even “impossible” (Bulu 2012, 153; Kwon 2019, 105) in real life. In order to provoke new, relational understandings of the Home and of systemic racism, which are further supported in the fuller DOHR curriculum, the former residents’ oral histories are shared in DOHR’s VR-based VLE.
Figure 1. At top left, the original Nova Scotia Home for Colored Children (the “Old Home”) on the occasion of its official opening in June 1921. Top right, a large brick extension was added in 1961. Bottom left, the “New Home” building built in 1978 (now the site of the Akoma Family Centre). At bottom right, the Old Home as it appeared during the DOHR research team’s site visit in April 2019. (For further details see Chapter 3 of the Restorative Inquiry Report [Province of Nova Scotia 2019].)
However, DOHR has taken an unusual approach to the design of its VR-based VLE, because the learning outcomes that the DOHR project fosters—grounded in the commitment to restorative justice which seeks to foster just relations (Llewellyn 2011; Llewellyn and Llewellyn 2015)—are unusual in the context of VR-based VLE design. Specifically, we have developed a different approach to presence, which is broadly understood to be the element of VR design that contributes most to student learning in VLEs, and is hence the principal design aim of most VLE projects. Presence is the sense of “being there” (Slater and Wilbur 1997) in a virtual experience or “the psychological state where virtual experiences feel authentic” (McCreery, Schrader, Krach, and Boone 2013, 1635). Outside of the context of VLE design for experiential education, virtual environments have been designed with quite different aims. The digital humanities, for example, has emphasized the use of virtual reconstruction in research contexts as “not a neutral representation of ‘the past,’ but the scholar’s interpretation of specific aspects of a place at a certain time—an interpretation that can be challenged, revised, or rejected” (Sullivan, Nieves, and Snyder 2017, 301; emphasis original). Since the goal of creating a virtual environment in the digital humanities has been to make an argument and provide the locus for future argumentation (Sullivan, Nieves, and Snyder 2017, 301; see Roberts-Smith et al. 2016; Roberts-Smith 2017), attention has been paid to VLE design that encourages critical creativity rather than presence (Roberts-Smith et al. 2013). There is also some very recent, parallel work in VR design for K–12 education exploring the ways students productively fill in the gaps in imperfect historical simulations without compromising their sense of historicity (Papanastasiou et al. 2019). By contrast, in what has come to be known as “immersive journalism” (de la Peña et al. 2010; Reis and Coelho 2018), the use of 360º video to place the viewer in the “center” of a documented event is designed not to help participants learn to do anything specific, but to encourage them to empathize with victims of injustices (see for example Torsei and Philippe 2019). To date, however, neither these alternative approaches to virtual environment design nor their critiques (e.g. Reis and Coelho 2018; Mabrook and Singer 2019) are well integrated into the discourse around VLE design for experiential education. Similarly, 3D graphical approaches to the representation of marginalized communities remain under-interrogated in the VLE design literature, despite some robust work in this area emerging from game studies (e.g. Reis and Coelho 2018; Malkowski, Russwork, and Trea 2017; Taylor and Voorhees 2018).
Our aim for the DOHR project and this paper is to broaden the conversation about VLE design—which has largely followed technology- and psychology-driven lines of thought originating in the early 1990s[2]—to accommodate issues that have arisen more recently in fields outside of VLE design, through a discussion of the DOHR VLE. Since perspectives on how to achieve presence in VLEs, and why such presence is effective, are quite dispersed even in the VLE literature, we begin our discussion here with a synthesis of the most influential concepts. We then provide a description of the DOHR VR experience, and a discussion of how it approaches presence differently, consistent with the relational principles of a restorative approach. In conclusion, we offer some preliminary reflections on the DOHR VR experience’s effectiveness as a learning tool and suggest next steps for future development.
Presence and Learner Engagement in VLE Design for Experiential Learning
VR-based VLEs are most commonly designed to help learners (whether in school contexts or in public education contexts) to develop either spatial or social competencies that are impractical and/or dangerous to teach, especially at introductory levels, in the actual world. Widely-publicized examples include VR-based small motor-skill training for surgeons, in which learners use physical surgical instruments as controllers of avatars of the same instruments, to perform virtual surgeries on digitally-simulated bodies; such systems are increasingly used not just to train new surgeons but also to refresh the skills of practicing surgeons before performing actual-world surgeries (as in Surgical Science’s VR training system for laproscopy and endoscopy). In socially oriented VLEs, students typically learn how to avoid or respond positively to harmful behaviors, such as racial stereotyping, by rehearsing actions in virtually simulated scenarios (as in Kaplana’s 2015 Injustice); or how to develop empathy for the suffering of others by experiencing a simulation of their hardships (as in Kors et al.’s 2016 A Breathtaking Journey). Since the expectation in both kinds of VLE is that students will be able to transfer competencies developed in virtual reality into actual-world situations, these VLEs strive to simulate real-world experiences as vividly and accurately as possible, often incorporating actual-world material objects as well as virtual simulations, such as the surgical instruments used by Surgical Science. Kwon (2019) argues that immersive VLEs are especially relevant to the first stage of Kolb’s (1984) model of the four circulative processes of experiential learning, “concrete experience,” which could be followed by “reflective observation, abstract conceptualization, and active experimentation” in the classroom.[3]
There is general agreement in the literature on VLE design for experiential learning that the closer a simulated experience is to an actual-world experience, the better it functions as a replacement for real-world experience. When a simulation is effective, it produces in the participant the feeling referred to as “telepresence,” which is usually abbreviated to “presence”: “the psychological state where virtual experiences feel authentic” (McCreery, Schrader, Krach, and Boone 2013, 1635). If a technologically mediated experience is effective in generating the sense of presence, the perception of the person experiencing it “fails to accurately acknowledge the role of the technology in the experience” (International Society of Presence Research). In other words: it is generally accepted in the field that if a VR experience is effective in generating presence, the participant forgets they have a VR headset on, and instead feels like they are “there” in the illusion the headset is creating. In education, this experience of forgetting you are in a VLE has been seen as an advantage for experiential learning. Since experiential learning is thought to have been achieved when virtual experience is recognized as similar to an actual experience, “enhanced presence” is an ambition of VR-based VLE design (Kwon 2019, 105).
The sense of presence in VR-based VLE design is often understood to arise from immersive hardware systems (Fowler 2015, 416). Immersion here is understood as the “degree to which a virtual environment submerges the perceptual system of the user in computer-generated stimuli” (Biocca and Delaney 1995). In this understanding, the perceptual system is submerged physically, by the technical hardware employed to create the illusion of the virtual environment. For example, we might think of a VR headset as more immersive than a desktop computer screen, because it excludes the perception of visual stimuli that are not part of the virtual illusion (Dalgarno and Lee 2010, 11). However, as the examples of Surgical Science and Injustice demonstrate, hardware alone is not the greatest determiner of perceptual submersion in immersive systems (Archer and Finger 2018); rather, perceptual submersion is achieved by the degree to which a virtual illusion explicitly mimics the actual world. Although there is no consensus in the literature on the most effective design practices for achieving presence in VR-based VLEs, three design factors are regularly identified as having a significant ability to increase perceptual submersion: representational fidelity (the degree to which a virtual illusion looks or sounds like reality), interactivity (the degree to which the virtual illusion responds realistically to the embodied actions of a spectator), and identity construction (the degree to which spectators can associate themselves with characters represented in the virtual environment).[4] These factors are normally differentiated from one another in the literature, but are also understood to be interdependent in ways that are not yet consistently articulated.
Representational fidelity, for example, is often understood to be achieved by one or more of the following four factors:
(a) The vividness, or “abundance of reenactment … providing information to the senses” (Kwon 2019, 102; see Steuer 1992). On a sliding scale, at the low end of what VR systems can deliver, only the sense of sight is engaged; in more sophisticated systems, hearing is engaged; then at the cutting edge of what is currently possible, touch is manipulated. Taste and smell remain beyond the capacity of existing virtual systems, available only in “actual reality” (see Figure 3).
(b) The realism of the virtual illusion (Bessa, Melo, Sousa, and Vasconcelos-Raposo 2018, 35; Bulu 2012, 156), including its three-dimensionality (Bulu 2012, 154; Dalgarno and Lee 2010, 11); the ways in which it represents users (Fowler 2015, 413); and “the consistency of object behaviour” (Fowler 2015, 413; see Dalgarno and Lee 2010).
(c) The plausibility and dynamics of the virtual environment, including such technical effects as reflecting mirrors or shadows (Sanchez-Vives and Slater 2005).
(d) The “quality of the display, with high-fidelity displays being most realistic or photorealistic” (Fowler 2015, 413; see Dalgarno and Lee 2010). In other words, the sophistication of the equipment used to deliver a VR experience.
Similarly, interactivity is often understood to be achieved by one or more of the following:
(a) The range of embodied actions available to the VR participant (Dalgarno and Lee 2010; Kwon 2019); in other words: the degree to which a participant can use their body in the ways they would in actual reality, by touching, speaking, or moving around, for example.
(b) The degree to which the virtual illusion responds to the participant’s actions (Murray 1997; Dalgarno and Lee 2010; Kwon 2019), when, for example, objects move or other avatars engage in conversation.
(c) The degree to which the participant can create new elements of the virtual illusion (Dalgarno and Lee 2010; Kwon 2019).
(d) The technical ability of the system to respond to action through, for example, head-tracking (Sanches-Vives and Slater 2005) or update rate (Barfield and Hendrix 1995, 3).
However, the first of the four principles thought to contribute to representational fidelity is also sometimes treated independently as a factor that interacts with interactivity to increase a virtual illusion’s ability to simulate reality. Kwon, for example, sees VR-based VLEs that leverage the sense of touch and enable a wide range of bodily gestures as more “authentic” in the sense that they provide a more vivid experience closer to actual reality (see Figure 2). VR that is “authentic” in this way is particularly good at generating a sense of “place illusion” or “place presence” (Bulu 2012), the sensation of being and operating at a remote or virtual place (Slater 2009), or “being there” in the place depicted by the virtual display (Slater and Wilbur 1997). Hence “place presence” is often a design goal of VLEs whose intended learning outcomes include spatial competencies that can be translated to actual-world scenarios (such as, for example, Surgical Science).
Figure 2. “Relationship between virtual reality and actual reality based on the degree of presence” (Kwon 2019, Figure 2).[5]
The third major design factor, identity construction, by comparison, is often understood to be an outcome of the first two, representational fidelity and interactivity (see Figure 3). Identity construction normally refers to the sense of personal “body ownership” (Slater 2009) that learners develop by associating themselves with a manipulable avatar in a virtual environment (Bulu 2012, 154; Fowler 2015, 414). It can also refer to a learner’s ability to construct identities for other player-participants through their respective avatars (Fowler 2015, 414; Bulu 2012, 155; Biocca et al. 2003; Schroeder 2002). Identity construction is often leveraged in educational contexts to help generate a sense of “co-presence”, or “being there together” (Fowler 2015, 414) in a virtual environment. Co-presence has two dimensions: “perceiving others and having a sense or feeling that others [are] actively perceiving us and being part of a group” (Goffman 1963; Slater, Sadagic, Usoh, and Schroeder 2000). Co-presence is normally understood to involve a sense that there is “psychological interaction” among individuals (Nowak 2001; Schroeder 2002; Bulu 2012, 155). As a result, identity construction is often a design goal of VLEs whose intended learning outcomes include social competencies that can be translated to actual-world scenarios (such as Injustice).
Figure 3. Dalgarno and Lee’s elaborated model of learning in a 3D VLE (Dalgarno and Lee 2010, Figure 1).
In general, however, whether designed for spatial or social learning tasks, presence is thought to have three major pedagogical benefits for learners. First, presence helps students focus on the learning tasks they are encountering in a VLE, developing a task-oriented “flow.” When students experience “flow”—the term used in the literature to describe “the state of being absorbed by an activity” (Scoresby and Shelton 2011, 227), which “mediates the relationship between presence and enjoyment” (Weibel, Wissmath, et al. 2008, 2274)—they learn better (Kwon 2019; see Figure 4).
Figure 4. Influence relations among vividness, tactile interactivity, locomotive interactivity, simulator sickness, presence, flow, and learning effect (Kwon 2019, Figure 8). Note that simulator sickness is a counter-indicator of flow here.
Second, presence helps create the sense of agency that learners need to have in order for learning to be experiential. Triberti and Riva, for example, describe presence as “a core neuropsychological phenomenon whose goal is to produce a sense of agency and control: I am present in a real or virtual space if I manage to put my intentions into action (enacting them)” (2016, 2). As Janet Murray puts it, agency is “the satisfying power to take meaningful action” (1997). The third benefit, which arises from the first and second, is that presence is also frequently associated with students’ satisfaction with their own learning (e.g. Bulu 2012). Student satisfaction is a measure frequently used to determine the effectiveness of a virtual learning activity (see Kwon 2019, for example).
If we were to extract the best practices for VR-based VLE design from the literature survey above, we might end up with something like: Make a high-fidelity simulation of the relevant actual-world environment; give learners a way of affecting the virtual learning environment and make the environment respond; and provide representations of learners themselves in the world. Thanks to the resulting sense of agency they will then feel, learners will forget they are in a virtual environment, getting into a flow where they are totally focused on their learning tasks. The outcome will be that they learn effectively and feel satisfied with their learning experience. According to the current state of the VLE literature, then, in an effective VR-based VLE a learner perceives themselves acting in a simulation and perceives the simulation responding; the resulting agency, presence, and flow lead to learning and satisfaction. While the literature describes best practices for VLE design to support the kinds of spatial and social learning outcomes commonly intended in experiential learning curricula, it does not support the DOHR curriculum’s intended learning outcomes.
Designing Presence in the DOHR VR Experience
The DOHR VR experience is a thirteen- to fifteen-minute individual learning activity that is embedded early in a five-lesson curriculum designed to run, typically, over the course of ten history classes. The VR experience was designed to support learning activities outside of the VLE that are constructed based on the principles of historical thinking and oral history education, and on a restorative approach to learning (Llewellyn and Llewellyn 2015; Llewellyn and Ng-A-Fook 2017; 2019; see also Gibson and Case 2019; Epstein and Peck 2019; and the Historical Thinking Project). The first two lessons invite students to join the former residents in their decades-long journey to bring their stories of the Home to light in order to build a better future. Students are introduced to a brief history of the Home and then asked to actively inquire about the historical significance, causes, and consequences of the Home. Their engagement in this inquiry is centered on an examination of oral history as a primary source in itself and in relation to other primary historical evidence (such as social worker reports, newspaper articles, and photographs). The lessons culminate with students developing a “restorative plan” that asks them, among other aims, to share what they have learned in a way that will do justice to the historical experiences of the former residents and contribute to the future-focused goal of reconciliation.
In the third lesson, learners are on-boarded in small groups to the DOHR VR experience in person, by a trained facilitator, at individual stations. The facilitator advises students how to end the experience if they find its content distressing. The facilitator leads a short “sharing circle” (a key activity in a restorative approach to learning) to debrief about the experience afterwards. The VR experience itself begins with a short, documentary-like 360º video segment in which learners see the storytellers, Smith, Morrison, and Dorrington-Skinner, and hear their voices in voiceover. The introduction of the storytellers is followed by a set of oral histories rendered in a multi-modal blend of 3D graphics, 360º and 2D video, 2D images, environmental and spatially-located sound, voiceover narrative, and text. There are 12 stories in total, but each learner can only choose to witness three, one from each storyteller. Following those three stories, all learners witness a short sequence in which the three storytellers reflect together about their memories of one common room in the Home. Finally, learners witness another 360º, documentary-like video for a concluding sequence in which the storytellers describe directly (that is, without the use of voiceover) how they came to be the activists they are today.
From its inception, then, the challenge for the DOHR team in developing the VR experience has been that it is intended for a different kind of use-case than other VR-based VLE projects. DOHR is a project that seeks for students to build a relational understanding of the historical harm of the Home by hearing the oral histories of former residents. The intent of the curriculum is for students to ask: What relationship do they have to, and thus what responsibility do they have to address, the history of harm, based on systemic racism, that is the legacy of the Nova Scotia Home for Colored Children? The aim of the curriculum, including the VR experience, is for students to build a sense of relationship to the lived experience of a place, even though they will never likely be in the actual Home nor meet the former residents in person. This means, in part, that a traditional sense of place presence in the VR experience is not important, since spatial awareness and spatial skills are not primary learning outcomes. Similarly, traditional social presence is not useful, since our aim is not to help students practice social behaviors in a virtual environment so that they can adopt them more confidently or consistently in the real world. What we need students to do is to consider their stance in relation to the stories of the former residents in order to inform their understanding and efforts toward just relations with those whose lived experiences are different from their own. In other words, instead of generating a sense of presence as it is traditionally understood, where a virtual experience feels like an experience that could be replicated in the real world, we need students to remember that they are witnessing a story being told through the perspective of another person, which they could never experience themselves in the real world. The term we are using to describe this form of presence is relational presence.
Relational presence has had several concrete implications for the design of the DOHR VR experience. Since we need to help foster an understanding of what it was like to live in the Home, for different people at different times, we are not invested in representational fidelity in the way that other projects are. Although we have worked extensively with architectural drawings, photographs, and other archival and archaeological evidence of the past structure of its site and buildings, in our renderings, we express the Home in a multi-modal, impressionist aesthetic that reinforces the former residents’ oral histories. Most stories, for example, begin in a line-drawn, white-on-blue rendering of the site of the Home that is intended to evoke a three-dimensional version of the architectural drawings used to structure 3D space at each point in the narrative. The invocation of the documentary record only becomes substantial—opaque 3D graphics, 360º video, light, and environmental sounds helping to establish a more specific impression of place, time, mood, and activities—as the voice of the storyteller (the sound that appears closest to a participant’s ear) begins to recount the story. The representational media are in turn combined in ways that neither attempt to mask the differences among them, nor their individual differences from the actual-world phenomena they evoke. Our aim is to privilege lived experience over the fragmentary documentary record, making it clear that the world learners are encountering is not an attempt to reconstruct the past through simulation. Instead, it is an attempt to construct a present encounter with oral histories about past experiences in the Home and the long-term impacts of those experiences. In contrast to the traditionally sought sense of “place presence”, then, the DOHR VR experience seeks to foster a sense of what we are calling “relational place.” Relational place is an invocation of what a place means—in the case of the DOHR VR experience, what it means for storytellers and learners—rather than a simulation of how a place looked or was configured at any given point time (see Figure 5).
Figure 5. Screen captures showing three different approaches to multi-modal impressionist rendering in excerpts from Morrison’s story, “Swamp Water” (top left); Dorrington-Skinner’s story, “Mrs. Johnson’s Helper” (top right); and Smith’s story, “The Switch” (bottom). See Roberts-Smith et al. 2019.
Since learners need to maintain a sense of the difference between their own perspectives and the perspectives they are learning about, we are, similarly, not invested in identity construction in the sense of the identification of self or others with avatars to enhance social presence as it is understood in the VLE literature. Rather, we seek to support in students the development of a sense of relation to the stories rendered in our VR experiences and the storytellers from their own position and perspective. The VR experience does not create the illusion that the storytellers are really “there” with learners in the virtual environment. This means there are no anthropomorphic avatars in our VLE, and we make no attempt to create roles or characters for the storytellers or for learners to “play.” Learners are characterized by means of an avatar that is a literal representation of the story-selection controller held in the learner’s hand—the only avatar in the entire build—only as the force that uses the controller to select a story. Instead of creating virtual representations of either storyteller or student, we make space for each to occupy their own, actual-world perspectives. For storytellers, that means that their oral histories are told in recordings of the adult storytellers themselves, and for learners, it means witnessing those stories as grade eleven Canadian History students themselves, and not in the kind of role-play scenario that is common in social competency VLEs.
The emphasis on witnessing oral histories of the Home, then, means that there is also very little traditional interactivity in the DOHR VR experience. Since the world of our VLE represents the lived experiences of the storytellers, students do not need agency in the sense of being able to take action that initiates a response from objects or characters in the virtual world. If they were able to do that, the world would no longer represent the storytellers’ perspectives, and would not help learners understand the difference between their own perspectives and those of the storytellers. It would also give students the illusion of having power to change the stories, which, since justice in the DOHR project depends on hearing stories that have not been heard before, would subvert the project’s aim of encouraging an active listening that may provoke new understandings of the past. It would undermine the students’ ability to consider how lessons from the past can contribute to future just relations, as required of them in the restorative plan they develop as part of the curriculum. In the DOHR VLE, interactivity is hence extremely limited in traditional VLE design terms. To the extent that it is available at all, it is designed to characterize learners as witnesses to the stories. Students are able to choose one story from each of the three former residents whose stories are represented, and then listen to it. However, learners are not inactive, because listening to the stories is itself an important cognitive activity that has been recognized, for example, in Indigenous studies as an active “inhabiting” of representational worlds (Ridington 1998), and in performance studies as an active self-reflection on one’s “role and experience as a spectator” (Rokem 2002). Bronwen Low, a member of the DOHR research team, has written about “the pedagogy of listening” in similar terms (Low 2015). Drawing upon the work of Jean-Luc Nancy, Low describes listening as a learning process that extends the ear towards the other. This is not a silent nor passive process, but rather, one that builds relations of deep listening between storyteller and listener (Low 2015, 270–75; Llewellyn and Cook 2017). We differentiate this form of “witnessing” from “co-presence” as it is understood in the field of VLE design: namely, as the perception of others in a virtual environment, and the perception that others perceive us, giving us the sense that we are “part of a group” (Bulu 2012, 155; see Goffman 1963; Slater, Sadagic, Usoh, and Schroeder 2000). In VLE design, “co-presence” normally refers to the presence of other users in the virtual environment, and it is achieved through synchronous user-manipulated avatar interactions. By contrast, we are interested in giving students a sense of relation not to other VR participants, but to the lived experience of the non-player characters of the storytellers, Gerry, Tony, and Tracy, who are not simultaneously present, but represented through pre-recorded media and pre-fabricated digital assets. Relational interactivity, then—the invitation to witness—places agency in the context of relationship. The learner’s power is to relate across differences in perspective.
In these different approaches to representational fidelity, identity construction, and interactivity, DOHR takes an approach that is also different from precedents in fields outside VLE design. For example, despite a shared lack of interest in in-world interactivity, our conception of “witnessing” also differs significantly from the concept of non-interactive “immersive witness” (Nash 2018) that has been taken up in 360º video-based journalism inspired by the work of Nonny de la Peña (2010). In this context, 360º video is understood as a means of simulating a distant event, and consequently as offering the experience, rather than a representation, of that event; this experience is “immersive witnessing” (Nash 2018). While immersive witnessing has been critiqued for its relative lack of interest in the distance or disinterestedness normally expected in journalistic reporting (Nash 2018; Reis and Coelho 2018), it has been taken up by activist and humanitarian organizations because it was thought to instill a sense of responsibility for others. However, immersive witness makes different assumptions than the DOHR project does about the nature and aims of witnessing. Immersive witness is interested in “providing the audience with something of an experience that is linked in various ways to the experiences of others” (Nash 2018) through passive reception of a photo-realistic simulation. DOHR, by contrast, avoids simulation to encourage the active exploration of differences of perspective arising from differences in lived experience.
Finally, another important difference between the DOHR VLE and other VLEs is that witnessing the different perspectives of former residents of the Home can be an uncomfortable experience. The stories are about the harms that former residents suffered there, and the resilience that they and other children drew upon to survive those harms. So, another difference between relational presence and traditional presence is that, although relational presence can be absorbing and lead to the kind of “flow” where students are fully engaged in their learning task, it does not necessarily offer the pleasant kind of self-satisfaction related to taking action within the VLE intended by Janet Horowitz Murray (1997) and others. Instead, relational presence can lead to affective dissonance, which is the discomfort we feel when we experience difficult knowledge (Zembylas 2015; Zembylas and Bekerman 2008; Simon 2015). That discomfort prompts thought-provoking questions for learners, providing a different opportunity and experience of agency, which learners explore in the fuller classroom curriculum. These questions include: How could this have happened? Why didn’t I know about this before? What is my responsibility now that I know these stories? That kind of questioning is a learning agency—the agency to inquire and to reconsider how we act in and through relationship with others in the world.
Designing the DOHR VR experience has suggested to us that presence need not necessarily be understood as a simulation-based forgetting that we are witnessing an illusion, nor as an erasure of our awareness of the technology that delivers it. The project does not use a model of presence that requires the reconstruction of spaces for us to “be” in, identifying with representations of ourselves or others, and feeling satisfying agency by interacting with place and social context. Instead, we think of presence as the unsettling agency to witness a different perspective on meaning, which offers an opportunity to consider, and possibly change, our actual-world understanding and behavior as a result. If presence is thought of relationally, an alternative model of effective VLE design emerges. Instead of acting in a simulation, a learner occupies a relational place to witness a story. The resulting sense of relational presence fosters forms of agency and affect that are critical for learners to inquire further and seek restorative actions for justice in their actual-world contexts. For DOHR to achieve relational presence, our VLE needed to offer the opportunity to witness a past world described by those who lived it and provoke questions, based on the opportunity to witness, that would otherwise be impossible to formulate. In the case of the DOHR project, relational presence was achieved by means of a mixed-mode, impressionistic representation of the lived experience of a real-world environment, which avoided avatars, and limited interactivity to opportunities to witness (see Table 1).
Traditional Presence
Relational Presence
simulation (representational fidelity)
impression (representation of meaning)
interactivity
witnessing
identity construction (recognizing self)
self-awareness, awareness of difference
agency
interpretation, inquiry
satisfaction
affective dissonance
Table 1. A comparison of traditional and relational approaches to presence in VR-based VLE design.
Conclusion
In the DOHR VR experience, we have developed a theory of relational presence, and one approach to achieving it, which have yet to be validated through empirical study of student learning, or in other VR-based VLE design projects. At the time of this publication, the DOHR team has conducted a study of the DOHR curriculum, including the VR experience, and is analyzing the data. Preliminary results from the data indicate positive learning outcomes. Students reported sensations that indicated they did experience a strong sense of flow, and acquired important new knowledge, despite our unconventional approach to designing the VLE. Future study of the delivery of the curriculum will help the team to understand how diverse social and geographical factors affect learning outcomes, and the need to address the accessibility limitations of our current design, beyond physical and auditory enhancements. An analysis of the data from classroom implementation will provide us with the evidence required to determine how the VR experience with a focus on relational presence, embedded within the curriculum as a whole, may lead to learners’ increased relational competency; that is, to an increase in students’ ability to engage in the work of building more just relations in their worlds.
Two avenues for further research have already suggested themselves to the DOHR team. First, there may be productive research to be done on the role of aesthetics as a support for learning in VLEs. The concept of representational fidelity has so far been limited to a very narrow subsection of what might be more fully understood as the representational aesthetics of a virtual learning experience (by which we mean the intentional manipulations of the media of expression to both represent and generate experiential phenomena). This may be a result of the strong influence, to date, of STEM disciplines on the development of existing VLEs; STEM-based work typically thinks of aesthetics as a means of enhancing user experience and usability (e.g. Tuch et al. 2012). In the DOHR VR experience, we found that a significant investment in representational aesthetics was essential to the pedagogical goals of the project—both in terms of the theatre-based design experts gathered to work with former residents of the Home and other members of the DOHR steering committee on the VR development team, and also in terms of financial and material resources. What the DOHR build lacks in traditional presence-inducing features, it perhaps makes up for in aesthetic features. This may also explain why the DOHR team found some preliminary precedents for some components of relational presence in Indigenous studies and performance studies, two fields that are deeply invested in what we might think of as a twenty-first century development of what Nicholas Bourriaud first termed “relational aesthetics.” For Bourriaud, “the possibility of a relational art (an art taking as its theoretical horizon the realm of human interactions and its social context, rather than the assertion of an independent and private symbolic space), points to a radical upheaval of the aesthetic, cultural and political goals introduced by [its predecessor] modern art” (2002, 14; emphases original). The DOHR VR experience’s stylistic gesture of intermedial, impressionist representation to achieve relational presence may be one way VLEs can begin to explore social systems (or “formations,” as Bourriaud calls them) in much richer ways than the field of VLE design has yet done. Similarly, DOHR’s engagement of a relational approach may encourage others to explore the ways in which Bourriaud’s articulation of “relational aesthetics” might be altered or expanded to better serve the aims of projects fostering restorative justice.
In addition, there is certainly more work to be done on the contextualization of VR-based VLEs within classroom-based curricula and with reference to in-class teaching strategies. Work in this area currently consists of a decade or more of advocacy in pedagogical game studies (see, most recently, Hébert and Jenson 2019 for both context and evidence of best practices). Although VLE designers have devoted a great deal of energy to the technical design of stand-alone VLEs, the field has not yet taken full advantage of the opportunity to apply best practices in pedagogical design to VLEs (Fowler 2015). This could be done in stand-alone VLEs but could also be approached by situating a VLE as one in a series of classroom learning activities, as the DOHR project has done. A significant advantage of considering VLEs in the context of an overall blended (in-person and virtual) curriculum design is that it avoids “technological determinism” (Reis and Coelho 2018, 1093) whereby virtual experiences are “considered both a product and an outcome of technology” (Reis and Coelho 2018, 1093), rather than an outcome of the ways designers have manipulated the technologies in question. Understanding a VLE as one learning activity in the context of a larger curriculum necessarily makes its technology secondary and emphasizes the agency of educators to design and use the VLE in the ways that best serve their students. Relational presence is one offering that DOHR can make to the larger project of reconsidering the role of VLEs in K–12 and public education, with a view to addressing issues of pedagogy, representation, and justice that are not yet well accounted-for in the field.
Notes
[1]The DOHR VR experience was designed using a process that aligns generally with the principles of “co-design” articulated in the seminal work of Steen (2013), whereby parties characterized as “stakeholders” are actively involved throughout the design process and afterwards (Steen 2013); as distinct from “participatory design,” in which stakeholders are consulted only at key points (Schuler and Namoika 1993; Björgvinsson, Pelle, and Hillgren 2010). However, DOHR’s process differs from this and other activist, participatory artistic practices leveraging digital media (e.g. Gubraim, Harper, and Otañez 2015) in its centering of a relational approach to all project activities. The full citation for the DOHR VR experience, acknowledging specific roles of individual co-design participants, can be found in our reference list under Roberts-Smith et al. 2019.
[2]See Steuer 1992 (also cited below) for an example of an influential early technology-focused work; Chittaro 2013 for an example of work using psychological concepts to better understand human-computer action; and Riva 2018 for a compelling example of the integration of philosophy, human-computer interaction, and psychology in current work. Lombard and Ditton 1997 offer a survey of early 1990s trends; Fowler 2015 and Reis and Coelho 2018 critique the outcomes of the emphasis on technology in particular.
[3]We note, however, that the classical conditions under which virtual experience is advantageous (i.e. where embodied experience is “expensive, dangerous, or impossible” [Dalgarno and Lee 2010]) may be as likely to occur at the active experimentation stage as at the concrete experience stage.
[4]An important additional factor, beyond the scope of our discussion here, is immersive tendency, which operates outside of the context of the VLE itself. Immersive tendency refers both to the pre-disposition of some participants “to involve and focus on the [sic] common activities in real life” (Bulu 2012, 159), and also to participants’ desire to immerse because they “have specific expectations about what the outcome should be” (Shin 2017, 71; citing Weibel et al. 2010; Burns & Fairclough 2015; Hou, Nam, Peng, and Lee 2012).
[5]Kwon’s scale addresses only the five most familiar senses. There is also a great deal of work being done on proprioception in research related to motion sickness in VR, which Kwon acknowledges as a counter-indication of presence. For a substantial review of the relevant literature, see Weech, Kenny, and Barnett-Cowan 2019.
Bibliography
Archer, Dan and Katharina Finger. 2018. “Walking in Another’s Virtual Shoes: Do 360-degree Video News Stories Generate Empathy in Viewers?” TOW Reports. Columbia Journalism Review. Tow Center for Journalism, Columbia University Graduate School of Journalism. March 18.
Barfield, Woodrow, and Claudia Hendrix. 1995. “The Effect of Update Rate on the Sense of Presence within Virtual Environments.” Virtual Reality 1, no. 1: 3–15. https://doi.org/10.1007/BF02009709.
Bessa, Maximino, Miguel Melo, A. Augusto de Sousa, and José Vasconcelos-Raposo. 2018. “The Effects of Body Position on Reflexive Motor Acts and the Sense of Presence in Virtual Environments.” Computers & Graphics 71 (April): 35–41. https://doi.org/10.1016/j.cag.2017.11.003.
Biocca, Frank, and Ben Delaney. 1995. “Immersive Virtual Reality Technology.” In LEA’s Communication Series: Communication in the Age of Virtual Reality, 57–124. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.
Björgvinsson, Erling, Pelle Ehn, and Per-Anders Hillgren. 2010. “Participatory Design and ‘Democratizing Innovation.’” PDC ’10: Proceedings of the 11th Biennial Participatory Design Conference (November): 41–50. https://doi.org/10.1145/1900441.1900448.
Bulu, Saniye Tugba. 2012. “Place Presence, Social Presence, Co-Presence, and Satisfaction in Virtual Worlds.” Computers & Education 58, no. 1: 154–61. https://doi.org/10.1016/j.compedu.2011.08.024.
Dalgarno, Barney, and Mark J W Lee. 2010. “What Are the Learning Affordances of 3‐D Virtual Environments?” British Journal of Educational Technology 41, no. 1: 10–32. https://doi.org/10.1111/j.1467-8535.2009.01038.x.
Epstein, Terrie, and Carla L. Peck. 2018. Teaching and Learning Difficult Histories in International Contexts a Critical Sociocultural Approach. New York: Routledge.
Fowler, Chris. 2015. “Virtual Reality and Learning: Where Is the Pedagogy?” British Journal of Educational Technology 46, no. 2: 412–22. https://doi.org/10.1111/bjet.12135.
Gibson, Lindsay, and Roland Case. 2019. “Reshaping Canadian History Education in Support of Reconciliation.” Canadian Journal of Education 42, no. 1: 251.
Goffman, Erving. 1963. Behavior in Public Places. New York: The Free Press.
Gubrium, Aline, Krista Harper, and Marty Otañez, eds. 2015. Participatory Visual and Digital Research in Action. Walnut Creek, California: Left Coast Press, Inc.
Kwon, C. 2019. “Verification of the possibility and effectiveness of experiential learning using HMD-based immersive VR technologies.” Virtual Reality 23: 101–118.
Llewellyn, Jennifer J. 2011. “Restorative Justice: Thinking Relationally about Justice.” In Being Relational: Reflections on Relational Theory & Health Law, edited by Jocelyn Downie and Jennifer J. Llewellyn, 89–108. Vancouver: UBC Press.
Llewellyn, Kristina R. and Sharon Cook. 2017. “Oral History as Peacebuilding Pedagogy.” In Oral History and Education: Theories, Dilemmas, and Practices, edited by Kristina Llewellyn and Nicholas Ng-A-Fook, 17–41. New York: Palgrave Macmillan.
Llewellyn, Kristina R. and Jennifer J. Llewellyn. 2015. “A Restorative Approach to Learning: Relational Theory as Feminist Pedagogy in Universities.” In Feminist Pedagogy in Higher Education, edited by Tracy Penny Light, Jane Nicholas, and Renee Bondy, 11–31. Waterloo, Ontario: Wilfrid Laurier UP.
Llewellyn, Kristina R. and Nicholas Ng-A-Fook, eds. 2017. Oral History and Education: Theories, Dilemmas, and Practices. New York: Palgrave Macmillan.
Llewellyn, Kristina R. and Nicholas Ng-A-Fook, eds. 2019. Oral History, Education, and Justice: Limitations and Possibilities for Redress and Reconciliation. New York: Routledge.
Low, Bronwen E., and Emmanuelle Sonntag. 2015. “Towards a Pedagogy of Listening: Teaching and Learning from Life Stories of Human Rights Violations.” In The Canadian Oral History Reader, edited by Kristina Llewellyn, Alexander Freund, and Nolan Reilly, 266–281. Montreal and Kingston: McGill-Queen’s University Press.
Mabrook, Radwa, and Jane B. Singer. 2019. “Virtual Reality, 360° Video, and Journalism Studies: Conceptual Approaches to Immersive Technologies.” Journalism Studies 20, no. 14: 2096–112. https://doi.org/10.1080/1461670x.2019.1568203.
Malkowski, Jennifer, and Treaandrea M. Russworm. 2017. Gaming Representation: Race, Gender, and Sexuality in Video Games. Bloomington: Indiana University Press.
McCreery, Michael P., P. G. Schrader, S. Kathleen Krach, and Randy Boone. 2013. “A Sense of Self: The Role of Presence in Virtual Environments.” Computers in Human Behavior 29, no. 4: 1635–40. https://doi.org/10.1016/j.chb.2013.02.002.
Murray, Janet Horowitz. 1997. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Cambridge, Massachusetts: MIT Press.
Nash, Kate. 2018. “Virtual Reality Witness: Exploring the Ethics of Mediated Presence.” Studies in Documentary Film 12, no. 2: 119–31. https://doi.org/10.1080/17503280.2017.1340796.
Nowak, Kristine. 2001. “Defining and Differentiating Copresence, Social Presence, and Presence as Transportation.” A Comparison of Four New Communication Technologies, Proceedings of HCI International Conference on Human-Computer Interaction, Lawrence Erlbaum Associates.
Papanastasiou, George, Athanasios Drigas, Charalabos Skianis, Miltiadis Lytras, and Effrosyni Papanastasiou. 2018. “Virtual and Augmented Reality Effects on K-12, Higher and Tertiary Education Students’ Twenty-First Century Skills.” Virtual Reality 23, no. 4: 425–36. https://doi.org/10.1007/s10055-018-0363-2.
Reis, António Baía, and António Fernando Vasconcelos Cunha Castro Coelho. 2018. “Virtual Reality and Journalism: A Gateway to Conceptualizing Immersive Journalism.” Digital Journalism 6, no. 8: 1090–100. https://doi.org/10.1080/21670811.2018.1502046.
Roberts-Smith, Jennifer, Paul Cegys, William Chesney, Arda Kizilkay, Colin Labadie, Robert Plowman, Tracy Dorrington-Skinner, Gerald Morrison, Tony Smith, Kristina Llewellyn, Jennifer Llewellyn, and the DOHR Team. The Home: The Digital Oral Histories for Reconciliation Virtual Reality Experience. Pilot Release. Auburn Drive High School, Dartmouth, and Hants North Rural High School, Kennetcook, Nova Scotia, Canada, Oct 28–Nov 8, 2019. Beta Release. Nova Scotia Home for Colored Children Restorative Inquiry and Nova Scotia Black Cultural Centre, Dartmouth, Nova Scotia, Canada, Apr 11–12, 2019. Alpha Release. Halifax, Nova Scotia, Canada, July 2018.
Roberts-Smith, Jennifer. 2017. “Théâtre et humanités numériques. Du développement des outils au design expérimental. [Theatre and the Digital Humanities: From Tool Development to Experience Design].” Trans. Ioana Galleron. Revue d’Historiographie du Théâtre 4. Special Issue: Études théâtrales et humanités numériques.
Roberts-Smith, Jennifer, Shawn DeSouza-Coelho, Paul Stoesser, and the SET Research Team. 2016. “Cambridge Revisited? Methodological and Phenomenological Theatricalities in Simulated Environments for Theatre History Research.” In Early Modern Studies and the Digital Turn: New Technologies in Medieval and Renaissance Studies, edited by Laura Estill, Diane Jakacki, and Michael Ullyot, 318–44. Toronto: Iter Press.
Roberts-Smith, Jennifer, and the SET Research Team. 2013. “Visualizing Theatrical Text: From Watching the Script to the Simulated Environment for Theatre (SET).” Digital Humanities Quarterly 7, no. 3.
Sanchez-Vives, Maria V., and Mel Slater. 2005. “From Presence to Consciousness through Virtual Reality.” Nature Reviews Neuroscience 6, no. 4: 332–39. https://doi.org/10.1038/nrn1651.
Schroeder, Ralph. 2002. The Social Life of Avatars: Presence and Interaction in Shared Virtual Environments. Berlin: Springer Verlag.
Schuler, Douglas, and Aki Namioka. 1993. Participatory Design: Principles and Practices. Hillsdale, NJ: L. Erlbaum Associates Inc.
Scoresby, Jon, and Brett E. Shelton. 2011. “Visual Perspectives within Educational Computer Games: Effects on Presence and Flow within Virtual Immersive Learning Environments.” Instructional Science 39, no. 3 (May): 227–54. https://doi.org/10.1007/s11251-010-9126-5.
Simon, Roger I. 2015. A Pedagogy of Witnessing: Curatorial Practice and the Pursuit of Social Justice. Albany: SUNY Press.
Slater, Mel. 2009. “Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments.” Philosophical Transactions of the Royal Society B: Biological Sciences 364 (1535): 3549–57. https://doi.org/10.1098/rstb.2009.0138.
Slater, Mel, and Sylvia Wilbur. 1997. “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments.” Presence: Teleoperators and Virtual Environments 6, no. 6: 603–16. https://doi.org/10.1162/pres.1997.6.6.603.
Slater, Mel, Amela Sadagic, Martin Usoh, and Ralph Schroeder. 2000. “Small-Group Behavior in a Virtual and Real Environment: A Comparative Study.” Presence: Teleoperators and Virtual Environments 9, no. 1: 37–51. https://doi.org/10.1162/105474600566600.
Steen, Marc. 2013. “Co-Design as a Process of Joint Inquiry and Imagination.” Design Issues 29, no. 22: 16–28. https://doi.org/10.1162/DESI_a_00207.
Sullivan, Elaine, Angel David Nieves, and Lisa M. Snyder. 2017. “Making the Model: Scholarship and Rhetoric in 3-D Historical Reconstructions.” In Making Things and Drawing Boundaries: Experiments in the Digital Humanities, edited by Jentery Sayers. Minneapolis: University of Minnesota Press.
Taylor, Nicholas, and Gerald Voorhees, eds. 2018. Masculinities in Play. Cham, Switzerland: Palgrave Macmillan.
Triberti, Stefano, and Giuseppe Riva. 2016. “Being Present in Action: A Theoretical Model About the ‘Interlocking’ Between Intentions and Environmental Affordances.” Frontiers in Psychology 6. https://doi.org/10.3389/fpsyg.2015.02052.
Tuch, Alexandre N., Sandra P. Roth, Kasper Hornbæk, Klaus Opwis, and Javier A. Bargas-Avila. 2012. “Is Beautiful Really Usable? Toward Understanding the Relation between Usability, Aesthetics, and Affect in HCI.” Computers in Human Behavior 28, no. 5: 1596–607. https://doi.org/10.1016/j.chb.2012.03.024.
Weech, Séamas, Sophie Kenny, and Michael Barnett-Cowan. 2019. “Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review.” Frontiers in Psychology 10 (April). https://doi.org/10.3389/fpsyg.2019.00158.
Weibel, David, Bartholomäus Wissmath, Stephan Habegger, Yves Steiner, and Rudolf Groner. 2008. “Playing Online Games against Computer- vs. Human-Controlled Opponents: Effects on Presence, Flow, and Enjoyment.” Computers in Human Behavior 24, no. 5: 2274–91. https://doi.org/10.1016/j.chb.2007.11.002.
Zembylas, Michalinos. 2015. Emotion and Traumatic Conflict: Reclaiming Healing in Education. New York: Oxford University Press.
Zembylas, Michalinos and Zvi Bekerman. 2008. “Education and the Dangerous Memories of Historical Trauma: Narratives of Pain, Narratives of Hope,” Curriculum Inquiry 38, no. 2: 125–54.
About the Authors
Jennifer Roberts-Smith (Associate Professor, Theatre Performance, University of Waterloo) is an award-winning artist-researcher, whose transdisciplinary work in performance, digital media, design, education, and social justice has appeared in theatres, exhibitions, and scholarly publications internationally. She is currently director of the qCollaborative (the intersectional feminist design research lab housed in the University of Waterloo’s Games Institute), and of the Social Sciences and Humanities Research Council-funded Theatre for Relationality and Design for Peace projects. Since 2017, JRS has served as creative director and virtual reality cluster lead for the Digital Oral Histories for Reconciliation project.
Justin Carpenter is a PhD Candidate in English Language and Literature at the University of Waterloo. His current research traces the use of the term “generative” from literary to computational contexts, arguing that an understanding of this term opens up a variety of arguments around concepts such as authorship, agency, and emergence. He argues that such a genealogy can help situate game studies scholarship in dialogue with modernist and postmodernist literary studies, as well as cinema and other media. His other research interests include poetry, philosophy of technology, and aesthetics.
Kristina R. Llewellyn is Associate Professor of Social Development Studies at Renison University College, University of Waterloo. She is an expert in oral history, history education, history of education, and women’s history. Llewellyn has numerous award-winning publications, including The Canadian Oral History Reader (MQUP, 2015), Oral History and Education: Theories, Dilemmas, and Practices (Palgrave, 2017), and Oral History, Education, and Justice: Possibilities and Limitations for Redress and Reconciliation (Routledge, 2019). Llewellyn is a co-investigator on the SSHRC-funded project Thinking Historically for Canada’s Future, which is working to revise history education across Canada. She is the Principal Investigator and Director for Digital Oral Histories for Reconciliation: The Nova Scotia Home for Colored Children History Education Initiative project.
Jennifer J. Llewellyn is a Professor of Law and the Yogis and Keddy Chair in Human Rights Law at the Schulich School of Law, Dalhousie University. She is an expert in relational theory and a restorative approach. She served as a Commissioner on the Restorative Inquiry for the Nova Scotia Home for Colored Children. She directs the Restorative Approach International Learning Community and the Restorative Research, Innovation and Education Lab at Dalhousie University. She is a member of the Steering Committee for the Digital Oral Histories for Reconciliation project.
Tracy Dorrington-Skinner is a member and former co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). She was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team she was one of the three storytellers. Tracy was a member of the UJIMA Design Team for the Restorative Inquiry and a member of the Advisor Group for the Restorative Inquiry.
Gerald “Gerry” Morrison is a co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). He was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team, he was one of the three storytellers. Gerry was also a member of the UJIMA Design Team for the Restorative Inquiry and a Commissioner on the Restorative Inquiry.
Tony Smith is a co-chair of Victims of Institutional Childhood Exploitation Society (VOICES). He was a resident of the Nova Scotia Home for Colored Children. A member of the DOHR Team, he was one of the three storytellers. Tony was also a member of the UJIMA Design Team for the Restorative Inquiry and a Commissioner on the Restorative Inquiry. He served as the co-chair of the Council of Parties (Commissioners) for the Restorative Inquiry.
Issue Editors
Amanda Licastro
Angel David Nieves
Victoria Szabo
Managing Editor
Patrick DeDauw
Copyeditors
Param Ajmera
Elizabeth Alsop
Patrick DeDauw
Kelly Hammond
Jojo Karlin
Michelle McSweeney
Brandon Walsh
Nicole Zeftel
Dominique Zino
Staging Editors
Laura Wildemann Kane
Patrick DeDauw
Kelly Hammond
Anna Alexis Larsson
Krystyna Michael
Benjamin Miller
Teresa Ober
Gregory Palermo
Danica Savonick
sava saheli singh
Inés Vañó García
The irony of writing about extended reality (XR) at this time when so many of us have been thrust into the virtual is not lost on us. The situation is reminiscent of the world depicted in Cline’s Ready Player One (2011), in which the entire population uses virtual reality (VR) to escape the increasingly precarious environment outside their doors. Works such as “The Machine Stops” by E. M. Forster (1909), or He, She, and It by Marge Piercy (1991), provide a dystopic vision of a virtual future, but now, we need—perhaps even crave—optimism. The virtual offers us a way to connect; the possibility of engaging with each other even from a distance. We also need to find ethical and sustainable ways going forward to employ a medium poised for mass adoption. This issue demonstrates the power of XR in pedagogical applications, community partnerships, and future gatherings.
This special issue focused on XR—most often referring to virtual and augmented reality (AR)—emerged from our shared excitement about the potential of immersive media to support innovative pedagogy at all levels of education, but also from our healthy skepticism about the limited circles of people actually empowered to do shape this process. The full reality–virtuality continuum, a concept first introduced by Paul Milgram and his colleagues in 1994, encompasses everything from virtual reality theaters and mixed reality headsets to augmented reality experiences on mobile devices. Until very recently, the higher end of XR was limited to specialized labs and researchers. As both the technology itself and the means of creating content have become more accessible, the field has expanded and diversified.
Some researchers include 360º imaging and 3D video in the XR mix, especially if experienced immersively; others might also extend the concept to include ambient technologies in “smart city” and “smart home” applications. At its core, the XR we are exploring in this issue is about designing an immersive and interactive experience in the service of teaching and learning. While XR in the future may boast the seamless interfaces of science-fiction fantasy, today’s implementations mostly remain awkward, partial, and experimental. This is not a bad thing. As scholars, we still have room to maneuver and to change the terms of use before the technology is naturalized into an invisible, yet costly, necessity for twenty-first–century learning.
Working in the field at this critical moment, we noticed the lack of scholarship engaged with humanistic concerns regarding XR technologies in pedagogical applications. The literature is dominated by rigorous publications on the technical side describing developments in the field of computer science, and by informative case-study examples from museums, journalistic applications, and popular entertainment. Pedagogical experiments in K–12 education, industry, building technologies, city planning, and medicine, meanwhile, have clearly demonstrated the potential of XR applications for teaching and training. Yet the critical conversations around the medium itself, its affordances, challenges, and opportunities for educational use, still take place within small, often isolated pockets of discipline-specific practitioners. We meet up in local working groups, at conferences and workshops, on Twitter—and now in the pages of JITP. One of our objectives in putting together this issue is to bring these different groups into deeper conversation with one another, promoting critical knowledge construction in the field while building out a body of citable literature in humanistic XR studies. We hope this issue helps expand the field to include a greater diversity of voices and experiences.
In the future, the pressing questions of our current circumstances may find answers in XR. Take for instance the growing number of virtual conferences, virtual tours, and virtual open houses already happening in response to the shift to remote learning and working conditions. Although made more urgent by COVID-19, the creation of virtual labs, virtual workspaces, virtual archives, and virtual art studios has long been the dream of XR researchers. Now is the time for scholars to envision and build this future with or without the collaboration of the big tech companies (most recently Apple and Verizon) that have been quietly buying up XR platforms and start-ups in anticipation of a pivotal moment like this. How will we ensure our values are embedded in the XR systems that emerge, or that the resulting models of pedagogy are immersive, interactive, accessible, and collaborative? Even as we go online with our teaching, we realize how much we’re missing from lived experience in physical proximity. How can we leverage the affordances of the real in pursuit of the digital? How does the digital expand access, opportunity, vision, and community? How might XR facilitate lifelong learning applications and the global communities these interventions make possible? The articles in this issue begin to explore these questions in greater depth, and offer potential avenues for further development, especially in terms of community engagement and social justice. This special issue also makes clear that diversity, in all its many forms, is an essential component for XR-based teaching and research, especially as we consider ways of applying intersectional analysis to applied learning.
Several of the articles in this issue focus on using XR for social justice. In “Immersive Pedagogy: Developing a Decolonial and Collaborative Framework for Teaching and Learning in 3D/VR/AR,” Lorena Gauthereau, Jessica Linker, Emma Slayton, and Alex Wermer-Colan draw from a symposium held at Carnegie Mellon University and conversations held with librarians, technologists, developers, and faculty in attendance there. The authors advocate for continued conversations regarding integration, use, and review of 3D/VR/AR teaching and learning technologies. In “Developing Virtual Reality Modules Aimed to Enhance Social Work Students’ Skills and Reinforce Knowledge,” Nicholas Lanzieri, Henry S. Samelson, and Jonathan Bowen describe how multiple approaches to the use of VR in therapist training—360º video and avatar-based game environments—can be embedded into a social work curriculum. Their work demonstrates how prior exposure to environments and potential conversations can enhance live engagement with a diverse set of clients. One of the more revelatory examples of XR technology applications is evidenced in the essay, “‘Relational Presence’: Designing VR-Based Virtual Learning Environments for Oral History–Based Restorative Pedagogy,” by Jennifer Roberts-Smith, et al. This article describes an approach to designing VR that intentionally makes users aware of their virtual environments in order to situate themselves apart from the oral histories they experience in simulations. In this piece, Roberts-Smith et al. introduce the Digital Oral Histories for Reconciliation (DOHR) project (dohr.ca), which worked in partnership with the Nova Scotia Home for Colored Children Restorative Inquiry (restorativeinquiry.ca) to create a VR experience intended to expose the truth of institutionalized racism and to empower the survivors to “build more just relationships for the future.”
Several of our articles also engage with the potential for partnership building and community engagement through historic sites, landscapes, and the university/college campus. In their article “Representing Indigenous Histories Using XR Technologies in the Classroom,” Amy J. Lueck and Lee M. Panich argue for the wider adoption of XR technologies such as annotated 360º video tours at key locations on campus to help undergraduate students understand and intervene in the continued erasure of Indigenous histories from existing commemorative landscapes there. This emphasis on community partnerships is echoed in the issue’s fifth article. In “Blending Disciplines for a Blended Reality: Virtual Guides for a Living History Museum,” Juilee Decker, Amanda Doherty, Joel Geigel, and Gary D. Jacobs demonstrate how an interdisciplinary partnership between a university and a local museum offered the opportunity for students to develop digital storytelling skills and multimodal literacy. In “Barriers to Supporting Accessible VR in Academic Libraries,” Jasmine Clark and Zach Lischer-Katz address both accessibility and the important role of libraries in VR creation, implementation, and support as we scale up from experimentation to broad-based implementation strategies.
Especially exciting and filled with generative possibilities are the kinds of lessons learned from case studies arising out of various pedagogical contexts and disciplines. A diverse team of David Neville, Vanessa Preast, Sarah Purcell, Timothy D. Arner, Justin Thomas, and Christopher French, describes a whole-college approach in “Using Virtual Reality to Expand Teaching and Research in the Liberal Arts.” Their approach to infrastructure development highlights how a smaller institution can make XR happen at a thoughtful, systemic scale—in harmony with existing pedagogical values and practices—in a highly selective teaching-focused undergraduate setting. “Truly Immersive Worlds? The Pedagogical Implications of Extended Reality” by Tamara O’Callaghan and Andrea Harbin provides specific examples of the kind of VR and AR applications an instructor might use in a liberal arts context. Specifically, they investigate how 3D models of historic sites and AR overlays on historic documents can serve as virtual tools to enhance the physical space. Another example, from Alison Burke, Elana Blinder, Leah Potter, and David Langendoen in their article, “Mission US TimeSnap: Developing Historical Thinking Skills through Virtual Reality,” shows that VR is a promising and useful tool for K–12 history education. Their TimeSnap game has helped to increase students’ engagement with historical documents, narratives, and terminology. Thinking critically about such XR engagement is not limited to the humanities classroom. In the sciences, there have been several applications supporting laboratory work and surgery, but in “Virtual Chirality: A Constructivist Approach to a Chemical Education Concept in Virtual Reality,” authors Samuel R. Putnam, Michelle M. Nolan, and Ernie Williams-Roby demonstrate how it is important not only to use XR in teaching, but also to bring students into the process of building applications. Together, these articles provide an interdisciplinary view of how XR technologies are shaping education at all levels through a critical engagement with interdisciplinary applications.
This issue celebrates pedagogical innovation and forward thinking, but we would be remiss not to acknowledge that it will be released at a time of profound loss, reflection, and fear. The Journal of Interactive Technology and Pedagogy was founded and remains housed at the Graduate Center, CUNY, in New York City, a city losing multitudes of souls, including brilliant academics, each day. All of us have felt this loss, and it is only because of the incredible dedication and hard work of everyone involved that this issue was published at this time. The greatest thanks go to our managing editor, Patrick DeDauw, who not only ensured that the process of publishing this issue was streamlined and efficient; he also made it enjoyable despite the most incredible of obstacles. Patrick, you are our hero. We would also like to acknowledge the willingness of our authors to work through revisions and copyedits with grace and professionalism amidst a global crisis. And to our reviewers, please know your service is very deeply appreciated.
One final personal note: this issue is dedicated in memory of Dr. David Greetham, a founding member of the Interactive Technology and Pedagogy program at the Graduate Center, CUNY. Without his foresight and inspiration, many digital innovations cited in this issue would not have been possible.
Bibliography
Cline, Ernest. 2011. Ready Player One. New York: Crown Publishers.
Forster, E. M. 1909. “The Machine Stops.” Oxford and Cambridge Review (November).
Milgram, Paul, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1994. “Augmented Reality: A Class of Displays on the Reality–Virtuality Continuum.” Proceedings of Telemanipulator and Telepresence Technologies 2351: 282–92.
Piercy, Marge. 1991. He, She, and It. New York: Alfred A. Knopf.
About the Editors
Amanda Licastro is the Assistant Professor of Digital Rhetoric at Stevenson University in Maryland, as well serving on the editorial collective of the Journal of Interactive Technology and Pedagogy. Her research explores the intersection of technology and writing, including book history, dystopian literature, and digital humanities. Publications include articles in Kairos, Digital Pedagogy in the Humanities, Hybrid Pedagogy, and Communication Design Quarterly, as well as a recent chapter on social annotation in Digital Reading and Writing in Composition Studies, published by Routledge. Her grant-funded project on virtual reality was awarded the Paul Fortier Prize at the 2017 Digital Humanities conference, and has been featured in the Baltimore Sun and Baltimore Magazine.
Angel David Nieves, BArch, MA, PhD, is Professor of History and Digital Humanities at San Diego State University in the Area of Excellence in Digital Humanities and Global Diversity. He is the author of An Architecture of Education: African American Women Design the New South (University of Rochester Press, 2018). Nieves’s scholarly work and community-based activism critically engage with issues of race and the built environment in cities across the Global South.
Victoria Szabo is a Research Professor of Visual and Media Studies at Duke University. Her teaching and research focus on critical and creative approaches to interactive and computational media in the arts and humanities. She was the founding director of the interdisciplinary PhD program in Computational Media, Arts & Cultures, and currently heads the Digital Humanities Initiative at the John Hope Franklin Humanities Institute at Duke. She is also Chair of the ACM SIGGRAPH Digital Arts Community.
Tamara F. O’Callaghan, Northern Kentucky University
Andrea R. Harbin, State University of New York, Cortland
Abstract
This article provides an overview of the extended reality applications virtual reality (VR) and augmented reality (AR) and examines the affordances and constraints of each with regards to their application in the humanities. The interactive nature of these extended realities engages their audiences in new and compelling ways. VR and AR applications have moved beyond gaming and are proving particularly effective and engaging for historic recreations. However, these technologies also present new challenges, precisely because they create immersive worlds so captivating that these environments may be perceived as “real” rather than as simulacra, especially by students and the general public. Using both VR and AR projects based in medieval Europe (Bologna 3D Open Repository and 3D Paris Saga) as case histories, we discuss some of the issues that these technologies pose to their creators and to their consumers—from how they might be used to make a heritage site more meaningful, to how they pose dangers of an excess of verisimilitude. As these technologies become more ubiquitous in academic settings, these early ventures into extended realities highlight some perhaps hitherto unconsidered pitfalls as well as demonstrate the promise that these new technologies offer in terms of pedagogy and community outreach.
Introduction
In the summer of 2016, the world was introduced to the emerging technology of augmented reality (AR) in the form of Pokémon Go, a location-based, AR-enhanced game that became one of the most popular mobile apps of the year. Many people were already familiar with virtual reality (VR), “a medium composed of interactive computer simulations that sense the participant’s position and actions and replace or augment the feedback to one or more senses, giving the feeling of being mentally immersed in the simulation (a virtual world)” (Sherman and Craig 2003, 13). As a popular gaming environment, VR has four key elements: it is a virtual space for the participant; it is immersive on both a physical and mental level for the participant; it provides sensory feedback directly to the participant; and it is interactive, responding to the participant’s actions (Sherman and Craig 2003, 6–11).[1] VR, in its most effective form, requires the user to be isolated from a conscious awareness of the real world by some sort of head-mounted display, such as Oculus Rift, Microsoft HoloLens, or HTC Vive. Alternatively, the user can experience VR in an enclosed, projection-based or flat-monitor-based environment, such as a CAVE.[2] Typically, the experience must be held in a static, controlled space; otherwise, the user might collide with real-world objects in the effort to participate fully in the virtual world. And, for many individuals, the VR experience results in motion sickness, sometimes known as VR sickness or cybersickness.[3] In contrast, AR is a medium in which digital information is overlaid on the physical world that is in both spatial and temporal registration (i.e., alignment) with the physical world and that is interactive in real time (Craig 2013, 36). Consequently, AR is much more accessible because the required equipment, usually a smart device (iPad, iPhone, Android tablet, or Android phone), is minimal. The fact the user remains cognizant of the real world around them while using the technology reduces the possibility of motion sickness and does not typically limit the user to a static, controlled space for the experience.
Both technologies have applications beyond gaming and are proving particularly effective and engaging for historic recreations. Such recreations can have a significant impact on learning, for they engage viewers—both the general public and students—in an educational immersive experience. Many of these viewers may never visit the actual historic site in their lifetime, so accuracy is important. Consequently, we need to keep in mind that a 3D digital model is a re-creation and not the real place. And as we move forward with VR and AR, we must give serious consideration to the goals we need and/or wish the technologies to meet, particularly with respect to pedagogy. At this point in time, VR and AR are very successful in engaging audiences for both entertainment and educational purposes:
The increasing development of VR technologies, interfaces, interaction techniques and devices has greatly improved the efficacy and usability of VR, providing more natural and obvious modes of interaction and motivational elements. This has helped institutions of informal education, such as museums, media research, and cultural centers to embrace virtual technologies and support their transition from the research laboratory to the public realm. (Rousso 2002, 93)
For the user visiting a virtual heritage site, the experience can be highly engaging and educational as long as expert guidance is provided. VR and AR cannot substitute for pedagogical instruction. It is not so much that the user must be reminded that the virtualization is not real; rather, supporting documentation must be easily accessible within the virtual world to help the participant understand the meaning and significance of the 3D models they encounter. And content builders must take an interdisciplinary, if not transdisciplinary, approach to the creation of the 3D models and their VR- or AR-enhanced worlds if the learning experience of the participant is to be as significant and valuable.
These technologies have the promise not only of engaging students in the history itself, but also of inviting them to consider how the work of history is done. As scholars and experts, we require the 3D models and their environments to be historically accurate, but that accuracy is necessarily limited. All models are inevitably interpretations of available evidence, and making that process more transparent to the student leads not only to a better understanding of the subject matter but of the process as well. As Willard McCarty has noted,
The best model [e.g., digital humanities tool] of something, that is, comes as close as possible to what we think we know about the thing in question yet fails to duplicate perfectly that knowledge. Failure of the model in an engineering sense is its success as an epistemological instrument of research, because skillfully engineered failure shows us where we are ignorant. (McCarty 2003, 1232)
Failing to create the perfect 3D model of an object in terms of historical authenticity is to be expected and appreciated for what it can teach us not just about the technology but about the 3D model itself in terms of our understanding of its historic accuracy. As teaching tools, VR and AR force the historical experts, as content creators, and their students, as content consumers, to think very carefully and intentionally about the recreation. For example, precise verisimilitude of a medieval English village could only be achieved by travelling back in time to the Middle Ages to conduct the kind of fieldwork envisioned by Connie Willis in her 1992 science fiction novel The Doomsday Book—an unlikely prospect by anyone’s standards.[4] However, it is important that we think beyond what VR and AR can do today. Even if we fail to achieve what we want the technology to do, we will learn from our mistakes and, in so doing, improve both the technology and our students’ understanding of the historical method.
Historical Accuracy: A Theoretical Approach
Virtual constructions of historical objects and architecture raise very real concerns about verisimilitude. To what extent are such 3D models accurate representations of the original? In many ways, VR serves to validate Jean Baudrillard’s understanding of simulacra and concerns about the hyperreal. In Simulacra and Simulation, he argues that the loss of distinction between reality and its representation results in the hyperreal—a world “without origin or reality” (Baudrillard 1994, 1–7). It is pure simulation and, as a result, creates an anxiety of origin and authenticity. Virtual worlds, including those associated with VR, can evoke an apprehension about the hyperreal, especially if the 3D model is used to substitute for the original. The current interest by computer graphic experts and enthusiasts in the creation and redistribution of virtual historic sites illustrates the problem. “Archaeological illustration and reconstruction is not new,” as Clifford L. Ogleby notes,
but the advent of high-speed affordable computers and the associated graphics capability gives people the opportunity to create better looking imagery. The imagery, however, is often the result of the technology, not archaeological or historical research. When this imagery is distributed without the accompanying research that explains the decisions made in the reconstruction, it is open to a variety [of] interpretations. This problem is compounded when the imagery is posted on the [world wide web], as the image can be extracted from the surrounding text and interpreted as an artifact rather than as a diagram. (Ogleby 2007)
Ogleby demonstrates this issue using easily obtainable images from the web that purport to portray accurate reconstructions (some computer generated) of the mausoleum at the ancient Greek city of Halicarnassus.[5] The images are imprecise and even erroneous, yet accepted by the general public as real: “Many people will tend to ‘see’ a photo-like image to be more like a photograph, and therefore a record of a real place in time” (Ogleby 2007). Not surprisingly, these online images almost always fail to include provenance, authorship, and veracity—information that would help the viewer to determine the authenticity of each 3D model and would serve as a reminder that the image being viewed is just that, an image, and not the original. The problem is only exacerbated when these models are incorporated into a virtual environment such as Google Earth or Second Life (Ogleby 2007).[6] These immersive and interactive worlds can encourage the non-expert user, such as a student, to accept the computer-generated model as an overly realistic recreation of the original.
Nevertheless, we should not be dissuaded from using the technology for pedagogical purposes both in the classroom and the community at large. Pierre Lévy argues convincingly against viewing the virtual as simply unreal: “The virtual, strictly defined, has little relationship to that which is false, illusory, or imaginary. The virtual is by no means the opposite of the real. On the contrary, it is a fecund and powerful mode of being that expands the process of creation, opens up the future, injects a core of meaning beneath the platitude of immediate physical presence” (Lévy 1998, 16). It is an actualization rather than a realization, one that involves “the production of new qualities, a transformation of ideas, a true becoming which nourishes the virtual in a feedback process” (Lévy 1998, 15).[7] The virtual and the real are not binary opposites. Rather, they exist on a continuum that supports a complete range of realness from the fully real to the fully virtual. Such a reality-virtuality continuum was first proposed by Paul Milgram and his colleagues. They suggest that everything in between is a mixture of reality and virtuality, including AR in which the real world is augmented by virtual enhancements and AV (augmented virtuality) in which the virtual world is augmented by the real (Milgram et al. 1995, 282–92).[8] The more obviously artificial nature of AR/VR visualizations may be used in a classroom setting to illustrate the sorts of choices that historians make in any evaluation/representation of historical data. What becomes important is not the degree of artificiality but rather the transparency of the method. Just as the creator of the virtual representation must make choices about how “real” to make their visualization (what to include and exclude), so the historian makes choices regarding what data to include and how that data is represented. The artificiality of extended reality technologies thus opens the door to conversations about not only the material being studied, but also the means by which it is studied.
The appeal of VR and AR is not new. Humanity has long held a fascination for trying to create a virtual experience of reality. In the nineteenth and early twentieth centuries, panoramic paintings became particularly popular, including the development of 360º murals that were intended to fill the entire field of vision and make the viewer feel as if he or she were in the virtual world depicted by the paintings (Thompson 2015).[9] The nineteenth century also saw the development of the stereoscopic[10] viewer and images, precursors to the View-Master and, more recently, Google Cardboard (Virtual Reality Society 2016). Experimentation in film also contributed to the development of the technology, particularly the widescreen camera lens. French filmmaker Abel Gance introduced “polyvision,” a specialized widescreen film format that involved the simultaneous projection of three reels of film in a lateral montage, in his 1927 silent epic Napoléon (Cuff 2015, 24). Polyvision, as well as the later development of CinemaScope and Panavision using widescreen lenses, gave the audience a panoramic and, subsequently, more immersive film experience. It was not until 1929 and the development of the flight simulator (Virtual Reality Society 2016) that a virtual environment was designed for teaching rather than for entertainment purposes. This focus on the pedagogical potential of virtual environments has become even more important today as VR and AR evolve from game platforms to teaching tools.
Both technologies exemplify the concerns faced by experts building virtual heritage sites.[11] For historians, archaeologists, and other scholars, the photorealism of the 3D models is the primary goal. In general, there are ten principles of 3D photorealism: clutter and chaos; personality and expectations; believability; surface texture; specularity; aging dirt, rust, and rot; flaws, tears, and cracks; rounded edges; object material depth; and radiosity (light reflections off diffused surfaces) (Fleming 1998, 3). To achieve photorealism, the computer-generated object should demonstrate at least seven of these ten principles (Fleming 1998, 3–4). The virtual world should not be pristine and unblemished because reality is messy and dirty. This concern for photorealism does not, however, apply in the same way to human 3D models. In fact, few virtual heritage reconstructions include human figures and for good reason. Firstly, creating realistic human models is time consuming and expensive since it requires a digital artist with considerable skill in drawing and modelling figures from life. Architectural and cultural artifacts are usually less difficult to build as 3D models. Secondly, living models, unlike objects, are expected to move in some way. Animation adds a complex layer of technology that is usually not the primary focus of the recreated physical environment. Thirdly, and most importantly from a pedagogical point of view, human 3D models can complicate the virtual experience by encouraging the user to try and interact with them rather than focus on the physical reconstruction of the heritage site. Finally, there is the consideration of how exactly “real” such human figures should be. The more realistic the 3D model of the living figure, the more likely that it will become an example of the uncanny valley phenomenon described in social robotics: that is, the 3D model will be almost too real so that the minor imperfections of the recreation become disturbing and even repulsive.[12] Thus a caricature of a human figure may be more appealing and effective than a truly realistic and complex representation in VR or AR.
Two Historic Recreations: Modelling Challenges
Bologna 3D Open Repository is the result of a collaborative project between the municipality of the city of Bologna and CINECA Interuniversity Consortium, an academic supercomputing group that offers technological support to education, business, and the community. The project’s primary goal was to build 3D models for the creation of a virtual Bologna that the municipality could use to promote the candidacy of the city’s historic porticoes, or arcades, as a UNESCO World Heritage Site. The repository is now maintained as a site dedicated to the collection and sharing of the 3D models for didactic purposes—namely teaching students about the city and its history. Figures 1 through 3 show some of the 3D models created by the consortium:
Figure 1. Portico of San Luca.
Figure 2. Hilly landscape south of the city.
Figure 3. Scene of a medieval university lecture.
Through these visualizations, students can learn about the architectural history of Bologna from the medieval period through to the 18th century. The computer graphics are high quality and demonstrate a number of the principles of digital photorealism. In particular, the architecture and landscapes exhibit great attention to detail and authenticity. The project includes human figures, not typical of most historic recreations, and these figures are generally caricatures rather than realistic representations of people. Certainly, such a use of humor in a virtual historic re-creation emphasizes the project’s desire to appeal to a broad, public audience (Guidazzoli, Liguori, and Felicori 2013, 58–65).[13] And the less-than-realistic style of the human figures avoids the potential issue of the uncanny valley.
Like the Bologna 3D Open Repository, the 3D Paris Saga project uses AR and VR to tell the narrative of the architectural history of Paris. Their approach, however, differed considerably. Dassault Systèmes, a European software company that specializes in 3D design, built a complex virtual world that traces the history of the city through almost 2,000 years with a special focus on a 3D reconstruction and interactive experience of the fourteenth-century Palais de la Cité and the Sainte-Chapelle (“Voici” 2015). The project originally included a 90-minute television documentary, a CAVE experience of the virtual world using 3D glasses (Vitaliev 2013), a PC-compatible interactive 3D website, and an AR-enhanced print book (Dassault Systèmes 2012). The visual accuracy and detail of the 3D architecture, topography, and atmosphere enrich the photorealism of the virtual world (see Figure 4). The fact that familiar monuments are shown in various stages of construction transforms the virtual experience into a deeper educational one. Considerable attention is also given to the appearance of the skies, reflecting typical Parisian weather rather than an idealized and eternal perfect sunny day (see Figure 5). Again, 3D human models that inhabit the virtual city are not a common feature of such historic recreations. They are merely shadowy figures and remind the viewer that Paris was always inhabited; however, because the figures are so ethereal, they avoid the uncanny valley phenomenon and encourage the viewer to explore the historic constructions rather than try to interact with the animated models themselves.
Figure 4. View of the Grande Cour and Trésor de Chartres with shadowed human figures in the courtyard (Dassault Systèmes).
Despite its initial success, the VR element of the project is no longer easily accessible: the CAVE environment is only available at Dassault’s Paris headquarters by appointment to select visitors.
Figure 5. View of the rose window on the west facade (Dassault Systèmes).
Virtual reconstructions such as these help students understand cultures, histories, and artifacts that are physically, temporally, or culturally distant. While it may be difficult for American students to visit Notre Dame, extended realities can help them experience it in a way that more traditional media cannot.[14]
The AR-Enhanced Text
The most successful component of the 3D Paris Saga has been the AR-enhanced companion print book published by Flammarion. Whereas current AR technology uses a mobile application on a smart device to trigger the digital enhancements embedded in the printed page, Dassault requires the user to hold select pages from the print volume up to the web camera on a PC.[15] Like a virtual pop-up book, the 3D models appear on the page as viewed through the computer screen (see Figure 6).
Figure 6. AR-enhanced print text (Dassault Systèmes).
The user may turn the book in order to see all sides of the 3D model, thereby gaining a greater appreciation of Parisian architecture throughout history, including the Middle Ages. However, interacting with the book and the technology is awkward and lacks the mobility that a smart device offers. It is also counterintuitive to the standard reading process since the user holds the book but looks away from it at the computer screen.
AR-enhanced texts are not new. Mark Billinghurst and his team at HitlabNZ (the Human Interface Technology Lab at the University of Canterbury, New Zealand) created some of the first examples in the early 2000s. Called “MagicBooks,” the texts are designed to encourage children to read:
The computer interface has become invisible and the user can interact with graphical content as easily as reading a book. This is because the MagicBook interface metaphors are consistent with the form of the physical objects used. Turning a book page to change virtual scenes is as natural as rotating the page to see a different side of the virtual models. Holding up the AR display to the face to see an enhanced view is similar to using reading glasses or a magnifying lens. Rather than using a mouse and keyboard based interface users manipulate virtual models using real physical objects and natural motions. Although the graphical content is not real, it looks and behaves like a real object, increasing ease of use. (Billinghurst, Kato, and Poupyrev 2001, 747)
Although early forms of AR used abstract, specifically designed images (often QR codes) to trigger enhancements, the technology has advanced to the point that any complex, informationally dense image may serve as a fiducial marker. The use of mobile apps and smart devices makes interaction with the text easy and intuitive.
A new wave of AR technology seems to be driven by the increased capability and ubiquity of our mobile devices. Jordan Frith notes that early theories about the internet hypothesized that humanity (or at least that bit of it that could afford computers) would become more isolated and private—living their lives at home—we assume spending their time (and money) ordering from Amazon (Frith 2002, 136). Mobile computing has diverted us from this possible future. Instead, we are bringing our private lives into public spaces, attempting to control these spaces through our AirPods or earbuds, our Google maps, and Four Square—all the while curating our experience of the urban environment on social media.
It is to this mobile landscape that AR brings such promise. AR’s ability to overlay the physical world with digital information offers a new kind of experience and understanding of our world. Victoria Szabo argues that AR may be used to make the site of cultural history more meaningful to their visitors through the layering of digital information over the physical space. As she explains, “Mobile AR systems have the potential to help users create situated knowledge by bringing scholarly interpretation and archival resources in dialog with the lived experience of a space or object” (Szabo 2018, 373). In so doing, she argues, the visitors move from comprehension of the site which entails historical distance and critical interpretation—in other words traditional educational materials that might guide visitors through the site—to apprehension. Apprehension is more experiential learning and “relies on the tangible and felt qualities of the immediate experiences” (Martin 2017, 837; quoted in Szabo 2018, 374). The ability of AR to merge the “real” physical world of the historical site with digital material such as reconstructions, interpretive data, etc. facilitates both apprehension and comprehension.
When we consider an AR publication, however, we are moving away from Szabo’s paradigm to its inverse. With the book form, we are beginning not with the physical space—which already brings with it the tangible learning central to apprehension—but with the more traditional way of making meaning within education: the book. AR is still in its infancy in the publishing industry, but interest in its possibilities is growing. According to one 2017 poll, only 9% of Americans have experienced an AR application (Martin 2017, 20). Yet in this same year, five major tech companies, including Apple, launched AR frameworks or apps following the surprising success of the AR game Pokémon Go in 2016 (Tan 2018, 22). According to Digital Capital, an investment group, AR and VR are poised to become major players in technology. They estimate an AR/VR market of $108 billion with AR as the primary force and with predicted revenues of $90 billion by 2022 (Tan 2018, 22). This market data may seem irrelevant to academia, but what it means is that publishers are beginning to move into AR as well, creating new opportunities for academic AR publications. Major news media such as The New York Times, The Guardian, The Wall Street Journal, BBC, CNN, Hulu, and Huffington Post have all experimented with some form of Virtual, Augmented, or Mixed Reality (VAMR) media (Martin 2017, 21). Deniz Ergurel, technology journalist and founder of the media start-up Haptical, asserts that VAMR marks the next major technological shift. According to Ergurel, “Every 10–15 years, the technology landscape is reshaped by a major new cycle. In 1980s, it was the PC. In 1994, it was the Internet. And in 2007, it was the smartphone. By 2020, the next big computing platform will be virtual reality” (Martin 2017, 20).
AR text, because it is multisensory, can bring some of the features of experiential learning to its readers including the visual features of the text, historical contextualization, images, audio, video, data visualizations, supplementary text, and most importantly, 3D AR augmentations. The multimodal possibilities of AR texts make them particularly useful to teachers of literature that is culturally or historically distant because, through such reading environments, students may be more easily introduced to the material culture that surrounds and creates the texts they are studying. Furthermore, this approach allows the students to engage with the material in a multimodal fashion, appealing not only to the language centers of the brain, but to the visual and aural centers as well. The digital environment encourages the reader (and even the author) to “play” with the text in terms of design and interactive engagement (Douglas 2000, 65). The brain’s ability to play is something we, like many animals, are hardwired to do for survival; consequently, the process of reading text, especially digital text, has neurological value precisely because it encourages the brain’s playfulness (Armstrong 2013, 26–53).
Conclusion: The Future of VR and AR
The argument can be made that neither VR nor AR offers a truly immersive experience because not all five primary senses of the participant are engaged. Certainly, computer technology can generate both visual and aural enhancements in the form of 3D models and recorded sound. However, touch, smell, and taste are more challenging. Haptic tools, such as gloves or a stylus device, are becoming more popular and offer both the VR and AR user the ability to touch and sense physical contact with virtual objects. AR actually has the advantage of offering much more real-world haptic information by default than VR can. With AR, the user can feel the actual book because it can be a real-world object, but, in VR, the technology must do something to allow the participant to feel such an object because the entire environment is computer created. Demand has been less so far for smell and taste, although there have been some experiments, largely unsuccessful, in adding odors to virtual worlds. Recent developments in the creation of technological tools to trigger the sensation of taste in an individual, such as the “digital lollipop” (Ramasinghe and Do 2016) and Electronic Food Texture System (Niijima and Ogawa 2016, 48–9), show promise for the eventual incorporation of this primary sense into the VR experience.
If full sensory engagement is required for a virtual world to be completely realized, then perhaps the most immersive and interactive experience of the Middle Ages may be one that is not computer-generated at all: Jorvik Viking Centre. Located in York, England, the museum and tourist attraction was created in 1984 and has long been famous for its appeal to the senses of its visitors, most significantly the sense of smell. A quick glance at such online review sites as Trip Advisor, Virtualtourist.com, etc. makes it clear that the intentional smells associated with the exhibit are not just memorable but also a significant factor in recommending the Jorvik Viking Centre. The exhibit’s use of scents to enhance the Viking experience has even generated scholarship exploring the effectiveness of odor in retrieving the memory of the tourist experience. Apparently, it is very effective (Aggleton and Waskett 1999, 1–7).[16] The Centre, in fact, intentionally engages all the senses of its visitors in order to make the historic re-creation a memorable and educational experience. In 2015, it actively promoted its non-digital exhibit in the language of virtual and augmented technologies, inviting guests to have a 4D Viking encounter rather than a mere 3D one. In this campaign, the Centre emphasized that all five primary senses of its visitors will be fully engaged (Jorkvik Viking Centre 2015):
“Touch: Handling collection of Viking Age artefacts, including bone, antler and pottery, on offer to visitors in the queue—participants will be blindfolded and asked to identify the object/material.”
“Sight: Binoculars are available in the ‘Time Capsules’ that take visitors around the recreated Viking city. These are to be used to spot the various animals that inhabit the scenes of the ride experience. A ‘spotter’s guide’ will be issued, allowing visitors to score themselves against their finds.”
“Taste: A Viking Host will be on hand to explain the Viking diet and offer up tasters of unsalted, dried cod (a Norse delicacy) and for visitors over 18, Mead, a beverage made of fermented honey, will be available.”
“Smell: JORVIK is already famed for its re-creation of the smells of the 10th century York but this will be taken a step further with the introduction of ‘smell boxes’ in the ‘Artefacts Alive’ gallery. A new aroma will be located next to a display of object, with the smell paired to match the contents. [Four] smells will be available: Iron (for the Iron working display), Leather (next to the leather and shoemaking), Beef (for the general living display), and wood (for our wood finds).”
“Sound: A Viking will entertain visitors with period-specific musical instruments (including a recreation of the panpipes found at Coppergate) and retellings of some favourite Viking sagas.”
But as entertaining as the Jorvik Viking Centre clearly is, do we really want, or even need, a fully immersive and interactive experience? From the perspective of pedagogical effectiveness and student engagement, perhaps not. AR may, in fact, be the technology that has greater potential as a pedagogical tool precisely because it allows the user to learn in a digital environment while always keeping a strong foothold in the physical world—a reminder that the 3D world is not, ultimately, a real place.
Notes
[1] For further discussion of these key elements, see Søraker 2011, 44–72.
[2] Cave Automatic Virtual Environment: an immersive video theatre experience in which a participant wearing shuttering glasses views stereoscopic images as they are projected on the walls of a self-contained space in response to the participant’s position and actions.
[3] Such motion sickness may be caused by display and technology issues, sensory conflict, or postural instability; see LaViola, Jr. 2000, 47–56.
[4] Curiously, in 1935, a version of what we consider to be VR glasses was, in fact, envisioned by science fiction writer Stanley Grauman Weinbaum in his short story “Pygmalion’s Spectacles”; see Project Gutenberghttp://www.gutenberg.org/files/22893/22893-h/22893-h.htm.
[5] Given the current interest in VR and AR, it is tempting to turn to 3D model sites, such as TurboSquid, to purchase ready-to-use models; however, evaluating these models for historical accuracy is essential. For example, searching on “medieval castle” brings up a wide selection of 3D models from fairly realistic structures to fantasy, fairytale confections that should be avoided for virtual historic sites; searching on “medieval woman” is even more problematic in terms of the results.
[6] Rousso expresses similar concerns about virtual heritage representation: “First, the issue of validity of information, commonly referred as authenticity. Second, the importance of accuracy in the representation of this information. Authenticity and accuracy are characteristics that archeologists, historians, and museum people strive to achieve and that the general public comes to expect from them. On the other hand, technologists dealing with the visualization of certain content are more concerned with the technical issues that pertain to implementation of the visualization and less concerned with authenticity and accuracy of the content itself” (Rousso 2002, 93).
[7] For a fuller analysis of Lévy’s understanding of actualization, see Ryan 1999, 78–107.
[8] For a detailed analysis of Milgram’s concept, see Craig 2013, 28–35.
[10] Stereoscopic imaging is the technique of creating an illusion of depth by using two offset images, one for the left eye and the other for the right, so that the brain processes both as a single, 3D image.
[11] We are making a distinction here between virtual heritage sites, which are 3D reconstructions of archaeological sites, architecture, or any other type of object, and 3D “real virtual worlds,” which combine 3D with “community, creation, and commerce,” such as World of Warcraft and Second Life; see Sivan 2008, 1–32.
[12] The phenomenon was first described by Masahiro Mori in 1970 and translated as “uncanny valley” by Jasia Reichard (Mori 1978).
[13] The project team has, in fact, used the 3D models to produce an award-winning stereoscopic short film, APA Etruscan (2012), for the Museum of the History of Bologna in which APA, an Etruscan character (see Figure 3), takes the viewer through a virtual history of the city.
[14] It is perhaps worth noting that, even though such virtual reconstructions are typically informed by the real world, the 3D digital exterior model of Notre-Dame de Paris created by Dassault Systèmes for the 3D Paris Saga as well as the 3D interior model created by Unisoft for the game Assassin’s Creed Unity may prove to be valuable resources for the rebuilding of the Cathedral after it was severely damaged by fire on April 15, 2019. Ironically, the real may now be informed by the virtual; see Wong 2019.
[16] Capitalizing on the Centre’s success with odor and its notoriety, York’s tourism board published Britain’s first scented tourist guidebook in 2014 (Gordon 2014).
Bibliography
Aggleton, J.P., and L. Waskett. 1999. “The Ability of Odors to Serve as State-Dependent Cues for Real-World Memories: Can Viking Smells Aid the Recall of Viking Experiences?” British Journal of Psychology 90.
Armstrong, Paul B. 2013. How Literature Plays with the Brain: The Neuroscience of Reading and Art. Baltimore, MD: The Johns Hopkins University Press.
Baudrillard, Jean. 1994. Simulacra and Simulation. Trans. Sheila Faria Glaser. Ann Arbor: University of Michigan Press.
Billinghurst, Mark, Hirokazu Kato, and Ivan Poupyrev. 2001. “The MagicBook: A Transitional AR Interface.” Computers & Graphics 25, no. 5.
Craig, Alan B. 2013. Understanding Augmented Reality: Concepts and Applications. Waltham, MA: Elsevier.
Cuff, Paul. 2015. A Revolution for the Screen: Abel Gance’s Napoleon, Film Culture in Transition. Amsterdam: Amsterdam University Press.
Dieleman, Hans, and Donald Huisingh. 2006. “Games by which to Learn and Teach about Sustainable Development: Exploring the Relevance of Games and Experiential Learning in Sustainability,” Journal of Cleaner Production 14.
Douglas, J. Yellowlees. 2000. The End of Books-Or Books without End: Reading Interactive Narratives. Ann Arbor, MI: The University of Michigan Press.
Fleming, Bill. 1998. 3D Photorealism Toolkit. New York: John Wiley & Sons.
Guidazzoli, Antonella, Maria Chiara Liguori, and Mauro Felicori. 2013. “Open Creative Framework for a Smart Cultural City: Bologna Porticoes and the Involvement of Citizens for a UNESCO Candidacy.” Information Technologies for Performing Arts, Media Access, and Entertainment Lecture Notes in Computer Science, Vol. 7990: 58–65.
McCarty, Willard. 2003. “Humanities Computing.” Encyclopedia of Library and Information Science. New York: Marcel Dekker.
Milgram, Paul, et al. 1995. “Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum.” Proceedings of the SPIE Conference on Telemanipulator and Telepresent Technologies 2351: 282–92.
Niijima, Arinobu and Takefumi Ogawa. 2016. “Virtual Food Texture by Electrical Muscle Stimulation.” Proceedings of the 2016 ACM International Symposium on Wearable Computers. New York: ACM.
Ramasinghe, Nimesha, and Ellen Li-Yuen Do. 2016. “Digital Lollipop: Studying Electrical Stimulation on the Human Tongue to Simulate Taste Sensations.” ACM Transactions on Multimedia Computing, Communications, and Applications 13, no. 1.
Rousso, Maria. 2002. “Virtual Heritage: From the Research Lab to the Broad Public.” Virtual Archaeology: Proceedings of the VAST Euroconference. Ed. Franco Niccolucci (BAR International Series 1075.
Ryan, Marie-Laure. “Cyberspace, Virtuality, and the Text.” Cyberspace Textuality: Computer Technology and Literary Theory. Ed. Marie-Laure Ryan. Bloomington: Indiana University Press. 78–107.
Sherman, William R., and Alan B. Craig. 2003. Understanding Virtual Reality: Interface, Application, and Design. The Morgan Kaufmann Series in Computer Graphics. San Francisco: Morgan Kaufmann Publishing.
Sivan, Yesha. 2008. “3D3C Real Virtual Worlds Defined: The Immense Potential of Merging 3D, Community, Creation, and Commerce.” Journal of Virtual Worlds Research 1, no. 1: 1–32.
Søraker, Johnny Hartz. 2011. “Virtual Entities, Environments, Worlds and Reality: Suggested Definitions and Taxonomy.” Trust and Virtual Worlds: Contemporary Perspectives. Eds. Charles Ess and May Thorseth. New York: Peter Lang Publishing. 44–72.
Szabo, Victoria. 2018. “Apprehending the Past: Augmented Reality, Archives, and Cultural Memory.” The Routledge Companion to Media Studies and Digital Humanities. Ed. Jentery Sayers. New York: Routledge.
Tan, Teri. 2018 “Making AR and VR Work in Publishing.” Publishers Weekly. July 2, 2018.
Thompson, Seth. 2015. “VR Panoramic Photography and Hypermedia: Drawing from the Panorama’s Past.” ISEA 2015: Proceedings from the 2015 International Symposium on Electronic Art. n.p. Accessed December 19, 2019. http://isea2015.org/proceeding/submissions/ISEA2015_submission_46.pdf.
We are extremely grateful to Alan B. Craig for reading and commenting on an earlier version of this article and for drawing our attention to the Bologna 3D Open Repository.
About the Authors
Tamara F. O’Callaghan is a Professor of English at Northern Kentucky University where she teaches medieval literature, history of the English language, and introductory linguistics as well as digital humanities approaches to literature. She received a Ph.D. in medieval studies from the Centre for Medieval Studies, University of Toronto, with a specialization in Middle English and Old French literature and medieval manuscript studies. She is the co-author of the textbook Introducing English Studies (Bloomsbury, 2020) and has published on medieval literature and manuscript studies as well as on the digital humanities and teaching. She also co-directs The Augmented Palimpsest Project, a digital humanities tool which explores how the medium of AR can be used in teaching medieval literature.
Andrea R. Harbin is an Associate Professor of English at the State University of New York, Cortland where she teaches medieval literature, the history of English, and Shakespeare and serves as department chair. She has worked as a digital humanist since 1998 as curator/editor of NetSERF: an Internet Database of Medieval Studies. She received a Ph.D. in Medieval English Literature with a specialization in medieval drama from The Catholic University of America, and has published articles on digital humanities, pedagogy in medieval studies, and medieval drama. She is likewise a co-director of The Augmented Palimpsest Project.
Extended reality (XR) is a growing interest in academia as instructors seek out new ways to engage students beyond traditional learning material. At the University of Florida, a team of library workers at Marston Science Library identify and design virtual reality (VR) learning objects for faculty and staff across campus. In summer 2019, the team at Marston Science Library and faculty in the Department of Chemistry partnered to pilot the implementation of a VR learning object into a general chemistry course. The Library team met with chemistry faculty and teaching assistants and developed a corresponding experience in virtual reality using the Unreal Engine, a game engine used in VR development. The VR learning object was designed for the section of the course related to chirality, an important chemistry concept that requires spatial awareness to understand. This article will explore VR as an approach to constructivist pedagogy and its application in chemistry education, specifically as a tool to positively impact spatial awareness. The results of the pilot implementation of the VR learning object was successful as chemistry faculty anecdotally noted increased student engagement and understanding of the course material. After a successful pilot, the learning object was also deployed in two organic chemistry courses. A survey was used to collect information from the students’ perspectives and demonstrated that the experience was beneficial for users developing spatial awareness of molecules for chemistry education.
Introduction
Extended reality (XR) and its subsets, virtual reality (VR), augmented reality (AR), and mixed reality (MR), have expanded their roles in academia as researchers continue to seek out emerging technologies to solve modern problems. In the realm of teaching and learning, instructors are turning to XR learning objects as potential improvements on traditional learning objects. In response to this interest, universities have grappled with deploying VR learning objects in spite of associated costs and complicated logistics (Kavanagh et al. 2017). At the University of Florida (UF), Marston Science Library (hereafter Library) created MADE@UF, a virtual reality development space run by library staff, with the vision of providing VR technology to all of campus. The Library’s mission for MADE@UF is two-fold: supporting student learning and development of virtual reality, and aiding faculty in identifying, developing, and implementing VR experiences in curriculum.
In maintaining and coordinating a VR development space, the Library collaborates with faculty across campus from various disciplines including English, medieval studies, astronomy, psychology, and tourism, to name a few. These collaborations can involve identifying existing VR experiences to deploy as well as creating VR experiences for specific courses. In summer 2019, faculty from the Department of Chemistry approached the Library with an idea for VR experience to be designed for an Accelerated General Chemistry course in fall 2019. The Library assembled its team of experts, consisting of the Engineering Education Librarian, who is also the director of MADE@UF, the Chemical Sciences Librarian, and the 3D and Emerging Technologies Manager. Each Library team member brought their individual expertise in learning theories and pedagogies, chemistry education, and virtual reality development, respectively, to consider how creating a virtual simulation would benefit teaching and learning in this Accelerated General Chemistry course.
Constructivist Pedagogy in Virtual Reality
Constructivism as a learning theory involves the learner constructing their knowledge based out of their experiences in which the learner is an active participant (Glasersfeld 2003). Most VR experiences discussed in scholarly literature are not built on constructivist pedagogy; rather, most practitioners focus on and research intrinsic factors, such as immersion, motivation, and enjoyment, as essential to using virtual reality applications in teaching and learning (Kavanagh et al. 2017). This initial oversight is understandable, as before establishing the pedagogy of a virtual reality experience, the most fundamental aspects of virtual reality must be established. However, early VR researchers reference the importance of presence, accommodation, and collaboration while advocating for VR as a framework for constructivism (Bricken 1990). Immersion, another intrinsic factor, is fundamental in creating a virtual reality experience that is compatible with constructivism, specifically that “immersion in a virtual world allows us to construct knowledge from direct experience, not from descriptions of experience” (Winn 1993). These fundamental aspects of a VR framework must emphasize the importance of establishing identity, presence, and collaboration within a virtual space, all of which would be self-evident in physical spaces; this would then allow the learner to experience conceptualization, construction, and dialogue, which are staples of constructivist pedagogy (Fowler 2015). These intrinsic factors guide the creation and implementation of VR learning environments as frameworks for pedagogies to build upon (O’Connor and Domingo 2017).
In addition to focusing on intrinsic factors, researchers are also attempting to categorize VR learning objects retroactively under various pedagogies and learning theories such as experiential learning, situated cognition, or constructivism (Johnston et al. 2018). Of all the pedagogies and learning theories used in VR, constructivism is the most referenced pedagogy to accompany virtual reality experiences in education (Kavanagh et al. 2017). Constructivism is not inherent to all virtual reality experiences as it requires more than VR can independently provide. Rather, constructivist VR experiences should aim to provide feedback that results in revision and restructuring of previous knowledge constructs (Aiello et al. 2012). The VR experience also needs to include active learning, a component of constructivism in which learners derive meaning from their sensory inputs, so learners can freely explore and manipulate their environment while receiving sensory feedback (Chen 2009). VR experiences using a constructivist approach can facilitate knowledge construction and reflection as well as social collaboration (Neale et al. 1999). A benefit of a constructivist approach in virtual reality is improving the learners’ perceived usefulness of the learning material, which is the most significant contributor to positive learner attitude (Huang and Liaw 2018). A constructivist approach to VR has also led to gains in knowledge, skills, and personal development in a VR learning environment (Bair 2013). Spatial visualization, an important factor in chemistry education, has proven malleable and positively impacted by VR designed with constructivism pedagogy (Samsudin et al. 2014).
Chemistry Education Background
The molecular properties and chemical reactivity of compounds rely heavily on the way molecules are arranged and oriented in three-dimensional space, which is referred to as the stereochemistry of a molecule (Brown et al. 2018). A fundamental skill for chemistry students is the development of spatial awareness at the molecular level: understanding the structural geometries and relative sizes of molecules, as well as how to mentally translate between different visual representations of molecules, is a prerequisite to understanding and predicting chemical phenomena (Oliver-Hoyo and Babilonia-Rosa 2017). Teaching students how to visualize molecules in space is one of the quintessential challenges in chemistry education, particularly because the nebulous nature of chemistry concepts can be difficult to make tangible. Because there is no way to directly observe a molecule or molecular interactions at the sub-nanometer scale, models are used to represent chemistry concepts in both chemistry education and practice. A particular stereochemistry concept introduced at the undergraduate level is the chirality, or handedness, of organic molecules. Chirality refers to the relationship between objects that are mirror images of one another but cannot be perfectly aligned (or “superimposed”) on top of each other (Brown et al. 2018). This property is visible in all everyday objects that aren’t perfectly symmetric, such as a person’s left and right hands, threaded screws, and headphone earbuds. At the molecular level, organic compounds have a chiral center at any carbon atom with four different groups attached to it. Recognizing chirality and systematically naming chiral molecules are particularly troublesome tasks for undergraduate students due in large part to the difficulty of mental 3D visualization required to “see” these properties (Ayorinde 1983; Beauchamp 1984).
Research has indicated that handling concrete and pseudo-concrete representations of molecules (tactile models and computer-generated graphics) improves students’ spatial understanding of molecular structures in comparison to abstract 2D representations (Ferk et al. 2003). Educators have deployed a variety of visualization tools to help students translate the 2D representations of compounds in the pages of their textbooks into visualized 3D objects, including handheld “ball-and-stick” modeling kits and computer-based modeling programs. Ball-and-stick models were first employed in the mid-nineteenth century (Matthew F. Schlecht 1998) and are still the most widely used method for 3D visualization in undergraduate chemistry curricula. However, these model kits make a number of assumptions about molecular structures that are not accurate, including that bond lengths and atom sizes are all uniform. Commercially available modeling kits vary widely and leave more advanced visualization nuances to the imagination of the students. Computer modeling programs have the ability to represent individual molecules more accurately in terms of bond lengths, bond angles, and atom sizes because they do not rely on fixed physical pieces that the user assembles. Most of these programs are streamlined for ease of use and there are many free and open source software options for students to access (Pirhadi, Sunseri, and Koes 2016), including the popular programs Avogadro, JMol, MolView, and Visual Molecular Dynamics. The largest drawback of computer graphic representations for student learning is that they are not tactile and are typically viewed on a computer screen. A comparison of the 2D structural drawings common in chemistry materials, 3D models built with ball-and-stick model kits, and pseudo-3D digital images generated by computer software are shown in Figure 1.
Figure 1. The chiral molecule bromochlorofluoromethane (CHBrClF) as represented by (a) the typical 2D line-angle formula created in ChemDraw; (b) two different commercial ball-and-stick model kits; and (c) computer modeling generated in MolView.
Now that the costs of developing XR learning objects and obtaining the equipment necessary for students to experience them are becoming more obtainable, chemistry educators are exploring the use of AR, MR, and VR in the classroom. A review on the use of XR in education highlighted that course content being presented in a novel and exciting way, the ability to physically interact with the media, and the direction of students’ attention to the important learning objectives were all positive factors in the success of XR lessons (Radu 2014). Some examples specific to the chemistry domain include laboratory experiments designed in game engines like Second Life (Pence, Williams, and Belford 2015), AR smartphone applications that allow molecules to jump off the pages of lecture notes as 3D structures (Borrel and Fourches 2017), molecule building and structure interactions with AR (Singhal et al. 2012), environmental chemistry fieldwork simulated through VR (Fung et al. 2019), and VR experiences involving interactive computational chemistry (Ferrell et al. 2019). For teaching students about stereochemistry and chirality, the power of VR to bridge the divide between the structural accuracy of computer modeling and the tactile advantage of ball-and-stick model kits seems promising.
Many chemistry-education protocols have proposed that using multiple model types is the most beneficial approach for teaching students who may learn in different ways (Dori and Barak 2001). While there is evidence that viewing instructors manipulate computer models on a screen does improve student understanding in large chemistry lecture courses (Springer 2014), allowing for students to directly manipulate the model themselves has been suggested as the ideal approach to implementing computer modeling whenever feasible (Wu and Shah 2004). Encouraging students to translate between 2D and 3D representations during a facilitated interaction with 3D models has also been suggested to improve students’ ability to reason with chemical formulae, as opposed to students using models on their own with no instructor intervention (Abraham, Varghese, and Tang 2010). Combining these constructivist and chemistry education pedagogical insights, we chose to design and implement a lesson on visualizing, handling, and naming chiral organic molecules using an in-house built VR experience. During this lesson, the following strategies were employed:
Undergraduate chemistry students in the class were previously instructed on the concept of chirality in their lecture course and had been exposed to 2D representations of chiral molecules.
Each student had the opportunity to individually participate in the VR experience.
Students were able to freely handle, rotate, and superimpose the molecules in the 3D virtual space.
Students in groups were asked to make observations and explain the chemical phenomena in the virtual experience.
Design and Implementation of the VR Learning Object
The VR chirality experience was designed for CHM 2047, a one-semester, accelerated undergraduate General Chemistry course designed for students with a strong high school chemistry background who are interested in moving into upper-level chemistry courses. The course met three times a week with two lecture periods and one discussion period. The faculty member led the weekly lectures and split the students into five groups for the weekly discussion periods; each of the discussion groups was led by a peer mentor, an undergraduate student who had recently completed CHM 2047 and finished at the top of the class. Chemistry doctoral students were also involved in the course as teaching assistants (TAs) and participated in some supervised instruction as well as oversaw the undergraduate peer mentors. For the discussion period related to chirality, the faculty member for CHM 2047 solicited the expertise of the Library team to incorporate a virtual reality learning object. The Library team created a virtual reality template for classes to use in an assignment that allows learners to interact with 3D molecular models using virtual tactility and physics. During this interaction, the Library team devised a constructivist approach for the learning object.
Learners would recall knowledge learned in prior and current chemistry courses, specifically knowledge related to chirality and systematically describing chiral geometries. Drawing on this knowledge, students in groups would hypothesize and discuss their observations of the virtual environment and the molecules within it. Students would interact with other group members, testing their ideas about the virtual experience, and constructing an understanding of the learning object. Ultimately the objective for the students is to locate the chiral center of a molecule, describe the geometry of this chiral center, and realize the non-superimposable nature of chiral pairs. Additionally, students may be able to create a mental visualization of the molecules and improve their spatial awareness.
In order to prepare the VR template for use in the course, the instructing professors were asked to compile a list of relevant chiral molecule examples, generate computer models of these molecules using the software of their choice, and provide the models to the Library team in .PBD file type form. Although this activity was focused on small organic molecules, the Library team proposed this workflow because .PBD file types can accommodate small molecules as well as large macromolecules, such as proteins and polymers. This practice would allow for the use of protein structures from the Protein Data Bank (PDB), a global archive of 3D structure data of biological macromolecules (wwPDB consortium 2019), in future VR activities with ease. It is also possible to allow students to directly generate structures and provide them to the Library team, rather than the course instructors, as a part of the chirality lesson. The Library team was then able to import these 3D models into the game engine while retaining all color information provided in the original software. The Library team used an in-browser file converter designed by chemists to rapidly generate XR files from chemical structure files called RealityConvert (Borrel and Fourches 2017) to process the models from their original filetype (.PDB) to .OBJ 3D models with associated .PNG and .MTL files for mapping color to the model’s topography.
The team chose Unreal Engine V 4.20 because it is free to use for educational purposes and boasts pre-built VR interactive tools. Aside from its practicality, Unreal Engine can reproduce a project for Windows, Mac, mobile, HTML5, and other platforms. Once the VR template is set, it is relatively easy to drag and drop a new molecule model into the program and view it in immersive VR. The template was designed to show every loaded model in a museum-style room on a pedestal with the name of the molecule displayed above. The learner can approach each model, walk around it, and see from every angle. They can pick it up using motion controls and rotate the model in their hands. They can also grab a model in each hand to freely move the models around and compare. Once released, the model snaps back to its original position. For increased usability, the team felt it was necessary to design a physics object that had a natural feel when the viewer grabbed the model and rotated it using their own wrist and controller movement; this is notable as the team removed any physics interaction created by overlapping objects as well as the game engine’s own preset “gravity.”
The team chose a very plain room to model, using rectangular topography so as not to distract the learner from the molecules placed throughout the space; ample lighting was generated to create a well-lit space to explore. Additional lights were added below each of the models to highlight the topography and heighten the sense of three-dimensionality. The experience allowed the learner to move around the room by two separate methods depending upon the configuration of the VR experience. Either the learner could physically move through the space if using a VR setup that allows for full-range motion tracking; or the learner could use a trigger on the hand controller to point to a specific point in the virtual space and “jump” to it when releasing the trigger. A simple text document was provided to explain the controls.
The pedestals in the room were arranged according to a grid with three pedestals in each row. The learning objective of the VR experience was for students to compare the two versions of chiral arrangement for each molecule selected by the instructor. Chiral molecules are systematically classified as either R (“Rectus,” right-handed) or S (“Sinister,” left-handed) configurations. In each row of three pedestals, the R and S versions of the molecule were placed on the far left and right sides of the row. On the center pedestal of each row, a side-by-side display of both R and S versions was shown for the students to view. Because students were expected to determine and assign R or S configuration to the molecules they viewed, the R and S structures were intentionally randomized in regard to their positions on the “left” or “right” side of the room so as not to indicate chiral configuration. For example, one molecule might be arranged as R, R and S, S in its row in the room, while another might be arranged as S, R and S, R.
It is worthy of note that because the 3D models were placed in the VR space as non-rigid bodies—meaning that the objects can clip through one another and occupy the same virtual space—students were able to experience the non-superimposability of chiral molecules in a unique way. The defining feature of chiral molecules is that they cannot be perfectly aligned on top of one another, and typically ball-and-stick models of the two versions are held side-by-side as closely as possible to demonstrate this property. However, in this VR environment, students were able to hold one version in the same space as the other version for each chiral pair and see that no matter how they manipulated the models, they could not align all atoms in a way that matched.
Figure 2. Screenshot of the Chirality VR experience displaying two 3D models being manipulated by virtual hands.
Once the template was updated to include the student-created models, the VR learning object was installed on VR-ready computers in the MADE@UF space at Marston Science Library. Library workers set up three Oculus Rifts on VR-ready computers in MADE@UF for five consecutive class periods on the day of a discussion period. Groups of two to four students moved to the VR stations, each with an Oculus Rift headset for the student and a monitor for the supervisor, a role filled by the peer mentor, teaching assistant, faculty member, or chemistry librarian. The role of the supervisor was to explain logistical questions with minimal input about the content of the experience, although supervisors would intervene if the students’ conclusions about the virtual experience were incorrect. The students interacted with the five sets of molecules, each set increasing in complexity as the student progressed through the virtual space. The students were able to manipulate, compare, and superimpose the two models in order to assign R/S configuration.
Further Use and Assessment
The CHM 2047 course instructor was looking to expose students to more advanced chemical concepts beyond the typical first-year general chemistry curricula in an innovative way. Chirality is a concept that may sometimes be introduced at the general chemistry level but is universally taught during the subsequent organic chemistry sequence. After the VR program was created and implemented in CHM 2047 during the fall of 2019, the same program was used for facilitated VR experiences in Fundamentals of Organic Chemistry (CHM 2200) and Organic Chemistry and Biochemistry 1 (CHM 3217) during the spring 2020 semester.
After these VR sessions were completed, a brief survey instrument (see Appendixes) was deployed in order to assess the student’s perceptions of the VR experience’s effectiveness in improving their understanding of chirality, including in comparison with other chemistry model types, and whether the students experienced any accessibility barriers during the process. Responses were collected from twenty-one students in total from the three chemistry courses.
Students were asked which molecular visualization methods they have used while studying chemistry and which they found most valuable to their understanding of chemical concepts. The four methods were VR (used by nineteen), ball-and-stick models (used by sixteen), drawings (used by nineteen), or a non-VR computer model (used by nine).
Figure 3. Student use of visualization methods in chemistry.
In terms of ranking how valuable each visualization method was, VR was ranked as the top choice with ten of twenty-one students, followed by drawings (seven students) and ball-and-stick models (four students). Two students ranked VR as their lowest choice method. Although nine students answered that they have used non-VR computer modeling before, none of the respondents ranked computer modeling as their top preferred visualization tool, which may be related to the intangible nature of computer modeling for novice chemistry learners.
Figure 4. Student ranking of preferred visualization method while studying chemistry.
The majority of students believed that virtual reality was a benefit to their spatial awareness of molecules. Eighteen of twenty-one students believed that manipulating the molecules in the virtual reality experience improved their ability to make R/S assignments. Sixteen of twenty-one students believed that manipulating the molecules in the virtual reality experience improved their understanding of the non-superimposibility of enantiomers. Lastly, seventeen of twenty-one students believed that manipulating the molecules in the virtual reality experience improved their ability to mentally visualize molecules. Students who answered in the affirmative to these questions often referenced that being able to see, visualize, move, hold, and touch the molecules was a benefit. One student described the experience as “incredibly helpful experience for someone like me that isn’t the best at spatial configurations,” while another mentioned that they “can still visualize how the molecules looked in the virtual reality experience and it has helped me to visualize molecules in my head.” A small group of students did not believe the VR experience was helpful. These students indicated that they already had an understanding of the concepts or that ball-and-stick models were superior. One student noted that “ball and stick models do the same without all the fancy equipment.”
The student responses to the survey highlight a need for improved methods for teaching content that requires spatial reasoning. While some students already have the requisite spatial reasoning skills, other students struggle with converting 2D, non-tangible drawings to a 3D mental construction. VR in chemistry can then serve as a tool to create more accessible content for a subset of students who historically have struggled with spatial reasoning. VR could then be used in conjunction with the traditional 2D drawings and ball-and-stick models.
One area of improvement for the VR experience was related to the visual accessibility of the program. Survey responses recorded that one out of the twenty-one respondents experienced “barriers” during the lesson, but this respondent did not disclose specific details of the accessibility issue. However, during one of the sessions hosted in the library, a library facilitator was needed to dictate the colors of specific atoms and indicate their identities to a user with color blindness. This accessibility concern is widespread in chemistry and chemistry education because periodic table elements are typically designated by a common color scheme and visualized molecules usually do not contain textures or patterns in addition to color coding. In future iterations of this VR experience, finding ways to depict atom identities that do not rely on color perception will increase user accessibility.
Reflection and Conclusion
Overall, the VR experience was successful. Chemistry faculty and TAs conducted informal debriefing sessions with the students following the Library VR session. Students provided positive feedback, with several noting an increased understanding of chirality following the VR experience. The faculty and instructors noticed the students were more engaged during the VR session than during other discussion or lecture periods, a feat that was observed to be uncommon for undergraduate chemistry courses. The course instructor mentioned that in previous years, the typical assignment on chirality involved students drawing 2D representations of 3D structures on paper; after the VR experience this semester, students commented on the ease of model manipulation the experience granted and said that they “truly understood” what the concept of chirality meant. The professor also noted that the undergraduate peer mentors (who had previously been students in the course before the VR lesson was implemented) “were particularly content on the new way to look at molecules, describing it as a more direct way to understand the role of 3D in chemistry.” The chemistry faculty are already interested in using the experience again for their Fall 2020 coursework, and several other chemistry faculty have also contacted the Library about deploying a similar VR learning object for their classes. The CHM 2047 professor commented that “it is clear from the success of this assignment that teaming up chemistry instructors with experienced librarians is the best combination to implement new technologies within the chemistry curricula.”
Bibliography
Abraham, Michael, Valsamma Varghese, and Hui Tang. 2010. “Using Molecular Representations To Aid Student Understanding of Stereochemical Concepts.” Journal of Chemical Education 87, no. 12: 1425–29. https://doi.org/10.1021/ed100497f.
Aiello, P., F. D’Elia, S. Di Tore, and M. Sibilio. 2012. “A Constructivist Approach to Virtual Reality for Experiential Learning.” E-Learning and Digital Media 9, no. 3: 317–24. https://doi.org/10.2304/elea.2012.9.3.317.
Ayorinde, F. O. 1983. “A New Gimmick for Assigning Absolute Configuration.” Journal of Chemical Education 60, no. 11: 928. https://doi.org/10.1021/ed060p928.
Beauchamp, Philip S. 1984. “‘Absolutely’ Simple Stereochemistry.” Journal of Chemical Education 61, no. 8: 666–67. https://doi.org/10.1021/ed061p666.
Borrel, Alexandre, and Denis Fourches. 2017. “RealityConvert: A Tool for Preparing 3D Models of Biochemical Structures for Augmented and Virtual Reality.” Bioinformatics 33, no. 23: 3816–18. https://doi.org/10.1093/bioinformatics/btx485.
Dori, Yehudit Judy, and Miri Barak. 2001. “Virtual and Physical Molecular Modeling: Fostering Model Perception and Spatial Understanding.” Educational Technology & Society 4 no. 1: 61–74.
Ferk, Vesna, Margareta Vrtacnik, Andrej Blejec, and Alenka Gril. 2003. “Students’ Understanding of Molecular Structure Representations.” International Journal of Science Education 25, no. 10: 1227–45. https://doi.org/10.1080/0950069022000038231.
Ferrell, Jonathon B., Joseph P. Campbell, Dillon R. McCarthy, Kyle T. McKay, Magenta Hensinger, Ramya Srinivasan, Xiaochuan Zhao, Alexander Wurthmann, Jianing Li, and Severin T. Schneebeli. 2019. “Chemical Exploration with Virtual Reality in Organic Teaching Laboratories.” Journal of Chemical Education 96, no. 9: 1961–66. https://doi.org/10.1021/acs.jchemed.9b00036.
Fowler, Chris. 2015. “Virtual Reality and Learning: Where Is the Pedagogy?” British Journal of Educational Technology 46, no. 2: 412–22. https://doi.org/10.1111/bjet.12135.
Fung, Fun Man, Wen Yi Choo, Alvita Ardisara, Christoph Dominik Zimmermann, Simon Watts, Thierry Koscielniak, Etienne Blanc, Xavier Coumoul, and Rainer Dumke. 2019. “Applying a Virtual Reality Platform in Environmental Chemistry Education To Conduct a Field Trip to an Overseas Site.” Journal of Chemical Education 96, no. 2: 382–86. https://doi.org/10.1021/acs.jchemed.8b00728.
Glasersfeld, Ernst von. 2003. Radical Constructivism: A Way of Knowing and Learning. Vol. 6. Studies in Mathematics Education Series. London: Routledge Falmer. EBSCOhost. https://eric.ed.gov/?id=ED381352.
Huang, Hsiu-Mei, and Shu-Sheng Liaw. 2018. “An Analysis of Learners’ Intentions Toward Virtual Reality Learning Based on Constructivist and Technology Acceptance Approaches.” The International Review of Research in Open and Distributed Learning 19, no. 1. https://doi.org/10.19173/irrodl.v19i1.2503.
Johnston, Elizabeth, Gerald Olivas, Patricia Steele, Cassandra Smith, and Liston Bailey. 2018. “Exploring Pedagogical Foundations of Existing Virtual Reality Educational Applications: A Content Analysis Study.” Journal of Educational Technology Systems 46, no. 4: 414–39. https://doi.org/10.1177/0047239517745560.
Kavanagh, Sam, Andrew Luxton-Reilly, Burkhard Wuensche, and Beryl Plimmer. 2017. “A Systematic Review of Virtual Reality in Education.” Themes in Science and Technology Education 10, no. 2: 85–119. https://eric.ed.gov/?id=EJ1165633.
Matthew F. Schlecht. 1998. “Historical Overview of Molecular Modeling.” In Molecular Modeling on the PC, edited by Matthew E. Schlecht, 3–10. New York: Wiley-VCH.
Neale, H. R., D. J. Brown, S. V. G. Cobb, and J. R. Wilson. 1999. “Structured Evaluation of Virtual Environments for Special-Needs Education.” Presence: Teleoperators and Virtual Environments 8, no. 3: 264–82. https://doi.org/10.1162/105474699566224.
O’Connor, Eileen A., and Jelia Domingo. 2017. “A Practical Guide, With Theoretical Underpinnings, for Creating Effective Virtual Reality Learning Environments:” Journal of Educational Technology Systems 45, no. 3: 343–64. https://doi.org/10.1177/0047239516673361.
Oliver-Hoyo, Maria, and Melissa A. Babilonia-Rosa. 2017. “Promotion of Spatial Skills in Chemistry and Biochemistry Education at the College Level.” Journal of Chemical Education 94, no. 8: 996–1006. https://doi.org/10.1021/acs.jchemed.7b00094.
Pence, Harry E., Antony J. Williams, and Robert E. Belford. 2015. “New Tools and Challenges for Chemical Education: Mobile Learning, Augmented Reality, and Distributed Cognition in the Dawn of the Social and Semantic Web.” In Chemistry Education, edited by Javier García-Martínez and Elena Serrano-Torregrosa, 693–734. Weinheim, Germany: Wiley-VCH Verlag GmbH & Co. KGaA. https://doi.org/10.1002/9783527679300.ch28.
Pirhadi, Somayeh, Jocelyn Sunseri, and David Ryan Koes. 2016. “Open Source Molecular Modeling.” Journal of Molecular Graphics and Modelling 69 (September): 127–43. https://doi.org/10.1016/j.jmgm.2016.07.008.
Radu, Iulian. 2014. “Augmented Reality in Education: A Meta-Review and Cross-Media Analysis.” Personal and Ubiquitous Computing 18, no. 6: 1533–43. https://doi.org/10.1007/s00779-013-0747-y.
Samsudin, Khairulanuar, Ahmad Rafi, Ahmad Zamzuri Mohamad Ali, and Nazre ABD Rashid. 2014. “Enhancing a Low-Cost Virtual Reality Application through Constructivist Approach: The Case of Spatial Training of Middle Graders.” The Turkish Online Journal of Educational Technology 13, no. 3: 8. https://files.eric.ed.gov/fulltext/EJ1034227.pdf.
Singhal, Samarth, Sameer Bagga, Praroop Goyal, and Vikas Saxena. 2012. “Augmented Chemistry: Interactive Education System.” International Journal of Computer Applications 49, no. 15: 1–5. https://doi.org/10.5120/7700-1041.
Springer, Mike T. 2014. “Improving Students’ Understanding of Molecular Structure through Broad-Based Use of Computer Models in the Undergraduate Organic Chemistry Lecture.” Journal of Chemical Education 91, no. 8: 1162–68. https://doi.org/10.1021/ed400054a.
Samuel R. Putnam is the Engineering Education Librarian at the University of Florida, where he is the mechanical and aerospace engineering and engineering education liaison and directs the MADE@UF virtual reality development space. He received his MLIS from Florida State University in 2009, focusing on library management and leadership. Samuel’s current research focuses on multimodal and multimedia instruction as a means to promote information literacy and active learning.
Michelle Nolan is the Chemical Sciences Librarian at the Marston Science Library in the University of Florida, where she serves as the reference and instruction specialist for users pursuing chemical research. She received her PhD in chemistry from the University of Florida in 2018, where her doctoral studies focused on organometallic synthesis and materials deposition, and she transitioned from bench scientist to library employee later that year. Michelle’s current interests include student-centered learning related to chemical information and the promotion of social justice in STEM disciplines.
Ernie Williams-Roby is a visual artist and designer based in Gainesville, Florida. He holds an MFA in Art + Technology from the University of Florida. He has contributed internationally to digital media artmaking and invention in the academic and public spheres for over a decade.
Email us at [email protected] so we can respond to your questions and requests. Please email from your CUNY email address if possible. Or visit our help site for more information:
