Tagged accessibility

A table describing the project timeline during full scale implementation from summer of 2018 to summer of 2019.
0

Implementing OER at LaGuardia Community College: Three Case Studies

Abstract

LaGuardia Community College, City University of New York (CUNY) began receiving funds from a New York State Open Educational Resources (OER) grant in 2017. Since then, several departments have made significant strides in the implementation of OER. We offer three case studies of putting OER into practice, in math, astronomy, and chemistry. Each case study presents the context for OER conversion, the steps taken, early results from OER implementation, and the main takeaways. A common feature across courses and departments is a process-oriented implementation of OER, namely, one that is constantly improving and moving towards the goal of openness and accessibility. The conclusion and future directions provide a critical reflection on the NY State grant, which is understandably focused on outcomes and not processes. The grant largely awards money based on converting courses to Zero Textbook Cost (ZTC), with total savings a primary metric. This approach leaves out features of open educational practice (OEP) that are critical to a process-oriented implementation. One feature highlighted in future directions is accessibility via Universal Design for Learning, which can help students with disabilities better utilize learning materials. Faculty and staff at CUNY have achieved a great deal from the New York State OER grant, despite a perpetual state of austerity. The authors recommend increased support for ongoing, less quantifiable, process-oriented improvements for OEP at LaGuardia and across CUNY.

Keywords: open educational resources (OER); open educational practice; STEM; library; IMathAS/MyOpenMath; LaGuardia Community College; CUNY; accessibility.

Introduction

Community colleges in the United States are among the most critical places to implement open educational resources (OER), here defined by the United Nations Educational, Scientific, and Cultural Organization (UNESCO) as educational and research materials that permit “no-cost access, use, adaptation and redistribution by others with no or limited restrictions” (UNESCO 2002; UNESCO 2012). At LaGuardia Community College, part of the City University of New York (CUNY), faculty and staff working on OER initiatives are part of a larger project that aims to achieve three goals: lowering economic barriers, lowering academic barriers, and improving open educational practices (OEP). The Cape Town Open Education Declaration (2007) frames OEP as follows:

Open education is not limited to just open educational resources. It also draws upon open technologies that facilitate collaborative, flexible learning and the open sharing of teaching practices that empower educators to benefit from the best ideas of their colleagues. It may also grow to include new approaches to assessment, accreditation and collaborative learning.

The three case studies included below present a process-oriented approach to OER implementation and OEP that encompass the above goals. These goals correspond to Lambert’s social justice aligned definition of OER, which is centered on redistributive, recognitive, and representational justice (Lambert 2018, 227–228). As Lambert argues, lowering economic barriers (redistributive justice), while important, does not necessarily lead to recognitive justice, which centers around representing diverse populations in learning materials, or representational justice, which provides “self-determination of marginalised people and groups to speak for themselves, and not have their stories told by others” (Lambert 2018, 227–28). Lowering academic barriers and attaining justice for students is more likely to occur when educators develop and implement learning materials that reflect students’ material and cultural lives (Benjamin and Vaught 2018).

The above factors are critical, since LaGuardia students often come from financially burdened homes and cannot afford costly textbooks; 62% of LaGuardia students living independently have incomes below $25,000 per year, and nearly 50% of students live with families earning less than $25,000 (LaGuardia Community College, Office of Institutional Research & Assessment 2021, 6). In addition to tuition and fees, LaGuardia estimates in its catalog that books and supplies cost each student over $1,500 per academic year. While cost and pedagogical consistency are common motivators for implementing OER, what has emerged after five years of development has been a commitment to the OER process.

A process-oriented approach is central to OER initiatives at LaGuardia and developed organically. Since 2017, the City University of New York and State University of New York have each received an annual $4 million grant from New York State to support OER; many departments at LaGuardia have successfully applied for portions of this grant to fund OER projects. Early efforts included Library Department-hosted one-off workshops, using the Open Education Network model (Cronin and MacLaren 2018), that introduced classroom faculty to OER textbooks. Simultaneously, the Math, Engineering, and Computer Science (MEC) Department piloted OER platforms for several gateway courses. A new professor in the Natural Science Department sought an automated grading system for his classes and began using an open-access online homework and learning management system (LMS) called MyOpenMath, powered by IMathAS software and published by Prof. David Lippman of Pierce College, Washington. The following year, the Natural Science Department initiated work on an OER portal for biology, chemistry, physics, and astronomy courses. Even as the COVID-19 pandemic upended higher education, with LaGuardia being no exception, OER and OEP work continued as the pandemic foregrounded crucial topics like accessibility—both for students with disabilities and for those with limited access to the necessary technologies. These topics are becoming an increasing part of LaGuardia’s OER process, but much work remains to be done. The initiatives described in the case studies are part of a larger community of practice (Kirschner and Lai 2007) at LaGuardia. Each year, LaGuardia’s growing OER community inches closer to Lambert’s three types of educational justice.

The three case studies provide a practical, step-by-step overview of how OER was implemented in courses in the MEC and Natural Science Departments. The studies are organized around four topics: context and problem, steps taken, results, and main takeaways. Each situation is unique, albeit with overlapping problems that needed to be solved through OER implementation. When available, the authors discuss the results of OER implementation, as well as the primary takeaways and future directions. What emerges is a detailed picture of how LaGuardia reached its current level of OER implementation and OEP.

Case Studies

Implementing OER in math: Written by Alioune Khoule

Problem and context

Most LaGuardia students, like those at many community colleges in the United States, are placed in remedial mathematics courses due to the lack of readiness for college mathematics, and they are highly likely to drop out of college due to the financial and academic challenges (Bailey, Jaggars, and Jenkins 2015). To combat the latter fact, MEC’s central focus has been removing barriers and ensuring access to educational resources, especially in its remedial courses.In Spring 2017, MEC at LaGuardia launched its OER initiative with the express intention of eliminating the costs of textbooks and platforms while at the same time expanding access to learning, improving class materials and enhancing our conventional mathematics course content (Khoule, Idrissi, and Sze 2021). The CUNY OER initiative funded by the State of New York contributed to the success of our local initiative by awarding MEC $1,016,250 in 2017–2022, which was used to redevelop courses, align homework and assignments, design videos, create more suitable course materials, and conduct faculty professional development.

Steps taken

Our first pilot phase of ten sections began in Fall 2017, detailed in Figure 1, using three different platforms: MyOpenMath, Webwork, and Khan Academy. One of the three platforms used in phase I, MyOpenMath, was selected to pilot thirty-four sections in Spring 2018 using Intro to Algebra, Fundamental Algebra, College Algebra, Corequisite STEM, and Elementary Statistics courses.

This table describes the OER project timeline for Spring 2017, Fall 2017, Spring 2018, and beyond.

  • Spring 2017: Course Design; Platform Selection.
  • Fall 2017: Pilot #1; Platform Final Selection; Course (Re)Design; Faculty Support Program = Preparation for Pilot #2.
  • Spring 2018: Pilot #2; Collaborative Adaptation of Courses; Assessment of Course Outcomes and Student Experience; Full Scale Faculty Support Program = Preparation for Full Scale Implementation.
  • Fall 2018: Full Scale implementation OER for: Fund of Algebra; Intro to Algebra; Elem Algebra; Co-req STEM; College Algebra; Elem Statistics.
  • Beyond: Pilot #3 for gateway courses; Explore low-cost courses.
Figure 1. MEC Department project timeline from pilot phase to full scale implementation.

Our OER courses were designed to make sure our students are learning with valuable resources needed to succeed in mathematics at zero or low cost.

We designed the course master sections using MyOpenMath by taking the following steps:

  • Examine the copyright license and terms of use to check provisions with respect to retaining, reusing, revising, remixing, or redistributing resources.
  • Ensure that there were sufficient resources available that aligned with established curricula.
  • Structure course items according to the order of the course curricula in consultation with course coordinators.
  • Set up a gradebook using the same weights as in the syllabi.
  • Ensure that each topic contains a detailed lecture from the chosen book as well as homework assignments and video lectures from YouTube.

After three semesters of exploring and piloting OER courses, MEC decided to go full-scale on six major courses: Intro to Algebra, Fundamental Algebra, College Algebra, Corequisite STEM, and Elementary Statistics courses. Figure 2 illustrates how we refined and improved all master sections by adding more supplemental readings as well as exam practices and video assignments.

A table describing the project timeline during full scale implementation from summer of 2018 to summer of 2019.

  • Summer 2018: Launch Copyright Team. Institute Training; Adapting Syllabus; Creating Accounts.
  • Fall 2018: Full Scale OER Expansion—Professional Development. OER Course design for Quantitative Reasoning; Elementary Statistics 2; Pre-Calculus; Calculus Series. Faculty Training. Copyright Team.
  • Winter 2019: Data Collection and Assessment. Professional Development. Faculty Training for the Courses: Quantitative Reasoning; Elementary Statistics 2; Pre-Calculus; Calculus Series.
  • Spring 2019: QR, Gateway & Calculus Course Series Pilot. Faculty/Tutors Training for the Courses: Quantitative Reasoning; Elementary Statistics 2; Pre-Calculus.
  • Summer 2019: Master & Syllabus Creation, QR, Calculus Series. Data Collection & Assessment. Working on Accessibility.
Figure 2. Full scale timeline from Summer 2018 to Summer 2019.

In Spring 2020, the COVID-19 pandemic shifted all courses to distance learning, requiring us to create and host accessible e-learning materials. We used a manual accessibility checklist from Duke University (2017) to be certain our course contents were accessible to all students. This checklist produces a more thorough accessibility review, as popular software like Utah State University’s Institute for Disability Research, Policy, and Practice’s WAVE tool may only find 30%–50% of accessibility issues (Duke University 2017).

At first, we made sure all virtual course materials were clearly labeled; quizzes, homework, exams, and exam practices were organized and easy to access. Chapters were divided into separated content areas, each containing a Portable Document Format (PDF) lecture from the book, a video lecture, and supplemental resources or links that were valuable to students learning. We converted all Word documents to PDF for better compatibility with screen readers—we also acknowledge there is considerable debate whether PDFs or Word documents are better from an accessibility perspective. For example, PDF text can be magnified much more than a Word document and background colors can be changed to improve accessibility. However, math problems often appear as images in PDFs that cannot be read by screen reading software, whereas they appear as screen-readable text in a Word document (Jisc Accessibility and Inclusion 2016). We ensured all our PDF images had descriptions saved into the alt text descriptions. We edited all video lectures and video assignments to make sure that they all contain captions. We created recordings of the two statistics projects and are planning on creating recordings for all our PDF content materials.

Because students were taking exams online from home, course coordinators changed the design and structure of the departmental exams for all our courses to a format in which students had to upload their work for some of the questions. Coordinators rewrote the exam questions which were later uploaded into a Lumen Learning platform (a front-end, low-cost software architecture that uses the same IMathAS backend as MyOpenMath) and MyOpenMath. Faculty members were paid to complete a professional development workshop on how to manually grade answers uploaded by students in MyOpenMath and Lumen.

Results

The total savings for students taking OER courses reached $1,184,329.25 by Fall 2019 and nearly $2,000,000 by Fall 2021. In addition, in Fall 2017 the passing rates for courses using MyOpenMath were slightly lower for fundamental algebra and college algebra and slightly higher for the co-req STEM courses when compared to non-OER. With a full-scale implementation of OER courses in Introduction to Algebra, Fundamental Algebra, College Algebra, Co-req STEM and Elementary Algebra courses (no pilot courses were offered in Fall 2018), we compared the pass rate of OER sections in Fall 2018 to Spring 2018 OER sections and non-OER sections. The pass rates for OER sections in Fall 2018 were the same or better than the pass rates of non-OER and OER courses in Spring 2018. The OER course template refinements and improvements achieved in Fall 2018 might have had an impact on student success: we added more course materials (video lectures and supplemental reading) and assessments (video homework and critical thinking homework and critical thinking exercises).

Takeaways

In addition to the costs of textbooks saving, the MEC OER initiative brought more freedom and flexibility in designing courses to ensure our students have all the learning materials they need to succeed. As of Spring 2022, all mathematics courses (about 200 sections per semester) are fully converted to OER. Our successful collaboration within MEC has made our OER initiative reach this far. We are working, in collaboration with the Natural Sciences Department, on implementing a college-wide, locally-hosted implementation of IMathAS, which would create an Open Learning LMS at LaGuardia. Our OER project still presents some challenges, such as creating more accessible visualizations of math problems for students with disabilities, for which we are still finding the most effective and suitable solutions.

Implementing OER in astronomy: Written by Joshua Tan

Context and problem

Upon arrival at LaGuardia Community College in Fall 2017 as a new faculty member, two major concerns occupied my mind. One was the lack of support for assessment. Full-time instructors at LaGuardia were required to teach twenty-seven credit hours a year, which typically corresponded to nine courses spread over two semesters. Each course was capped at twenty-four students, which potentially meant that 216 students a year would need assessment. Many instructors had adopted plans where homework assignments were assigned but never evaluated, which had the auxiliary effect that examinations would become the main means of assessment. Knowing that formative rather than summative assessments are often the most valuable means to provide student feedback and acknowledging that, at institutions with more resources, there were often teaching assistants and graders who would assist in reviewing students’ work on homework, it seemed reasonable that automated assessment strategies be considered. Additionally, the population at LaGuardia was uniquely vulnerable to the costs of textbooks and course materials that have lately been an increasing burden on student higher education. With these motivations in mind, I immediately began searching for options to address all these needs. OER options were an obvious choice, as they would allow for both open-source development of assessment modules from across all participating institutions, as well as providing access to students in affordable and accessible means.

Steps taken

The first completely free, modular system that I found was MyOpenMath, and even though no other astronomers were yet using the platform, I adopted it immediately to provide a randomized and automatically graded system for my classes, hoping that my example in the future might encourage others to follow suit.

Within a few months, a call for proposals was issued from City University of New York for implementation of Open Educational Resources (OER). I wrote and received a $10,000 grant to create homework questions using the platform along with two other professors in the department in the hopes of gaining a Zero Textbook Cost (ZTC) certification; see Figure 3 for an example of a question created with MyOpenMath. Upon the successful completion of that grant in 2018, the next round expanded implementation of OER to classes across the Natural Science department, with MyOpenMath forming a critical assessment structure for many of the courses with quantitative components (especially astronomy, physics, and chemistry).

An example question from Joshua Tan's library of IMathAS assessments in astronomy.

The question reads as follows: "Some exoplanets are discovered using the radial velocity (RV) method which measures the wobble of a star towards and away from an observer. Shown below is data from the RV curve for HD 149026 b, an exoplanet that was discovered using this method. This is a mock-up of the real data for this object, and you can verify this using the NASA exoplanet archive." A plot is then shown with alt text that indicates a sinusoidal variation of a particular velocity amplitude in meters per second and period in years. The questions are then presented as follows: "On the basis of this observation, what is the maximum radial velocity observed (in meters per second)?", "What is the period of the exoplanet orbit (in days)?", and "Since we know the period of the orbit, we can calculate the semi-major axis of the orbit given that the star that HD 149026 b orbits around has a mass of 1.3 M⊙ (the symbol for the mass of the Sun). What is the semi-major axis of HD 149026 b's orbit in astronomical units (AU)?" A scaled version of Kepler's 3rd Law is given as "Potentially Useful Information".

Figure 3. This question is used in assessments for Joshua Tan’s astronomy and astrobiology classes which uses randomly selected real data from the NASA exoplanet archive to generate unique “mock observations” for every student that illustrate how the exoplanet (that is, a planet that orbits another star) was discovered.

The advantage of the IMathAS system is its modularity, openness, and flexibility in design. As a community-based project, the thousands of questions in its libraries, the customization possible in assignment and assessment design, and the full capabilities of the platform as an LMS make it uniquely suited to provide for innovative, open pedagogy at the level of assessments. Questions can be designed to randomize quantities, pull a random element from a list, or even display a random image for analysis. I use the capabilities of the system to test specialized vocabulary, scaffold assessments (Crippen and Archambault 2012), and guide students through complicated calculations in a piecewise fashion. Such modularity aligns with pedagogical design principles that foreground student understanding in instruction and assessment (Wiggins and McTighe 2005; Lowyck 2002).

At the point where the COVID-19 pandemic hit, assessment materials had been developed and implemented so that distance learning was much more easily accommodated. One of the primary concerns of many instructors in LaGuardia’s Natural Sciences Department was whether and how assessments could be proctored to avoid instances of cheating or sharing answers. The skills used to develop effective OER assessments that addressed this concern were no longer simply best practices. They suddenly were instruments that could allow class to carry on with minimal disruption. Assignments that provided each student with a different question or a randomized element were in high demand as controversies swirled around instructors instituting proctoring software with troubling implications for surveillance and privacy (Logan 2021).

In the Natural Sciences, the biggest hurdle to enforced quarantine was the question of laboratory investigations. While physics, chemistry, and biology laboratory skills are difficult to gain without access to the equipment, physical venues, and hands-on engagement that in-person lab experience provides, a significant number of pedagogical goals including hypothesis testing, data production and analysis, and empirical engagement can be achieved with simulation. The structure provided by the IMathAS system used in tandem with online simulations and applications released under OER licenses (perhaps, most famously, the PhET simulations of the University of Colorado) encouraged me to develop entire virtual laboratory environments that worked through the open licenses, randomized the experimental set-ups, and allowed for experiments that would be impossible to achieve in any other fashion. For example, a thirty-minute investigation into the force of gravity using the application developed by PhET simulates an experiment that uses masses on the order of billions of kilograms separated by kilometers of distance to achieve human-scaled forces. Thus, OER technological innovation allowed pedagogy to extend beyond the traditional limitations of introductory science classes and provided a context for authentic learning experiences in place of what is otherwise conceptual inquiry at the theoretical level.

At the same time as I was developing OER material for introductory science classes, a cross-disciplinary team of educators from LaGuardia Community College’s Library Department, Mathematics, Engineering, and Computer Science (MEC) Department, and Natural Science Department came together to lead a professional development seminar with the hopes of supporting the implementation and development of OER materials across the campus. Running every spring since 2020, this seminar has encouraged a critical examination of extant platforms and opportunities for development and highlighted best practices for use, modification, and production of OER materials involving more than fifty faculty participants from around the college.

Results

Having now created a library of 1,200 questions, the MyOpenMath astronomy community which I initiated, includes instructors at LaGuardia and increasingly at other institutions as well. The two astronomers from LaGuardia (myself and Prof. Allyson Sheffield) along with Dr. Jana Grcevich, an adjunct instructor at City College of New York and Outreach Director at Columbia University, led two Astronomical Society of the Pacific conference workshop sessions in 2020 and 2021 to encourage astronomy educators to use the platform by highlighting the versatility and the compendium of questions already created (Tan, Sheffield, and Grcevich 2021).

The outcomes of the OER seminar have included faculty adopting and adapting OER materials for their own use, documented on our seminar website. The progression from considering OER to lower the financial impact for students toward a model of open pedagogy (Teixeira 2013) encouraged us to incorporate accessibility for students with disabilities as a major consideration in development. This emphasis has become more pronounced as we have continued in this work.

Takeaways

Indeed, emergency distance learning highlighted the way that OER development can promote greater accessibility for course materials (Baran and AlZoubi 2020). While the financial considerations are most often emphasized in evaluating the impact of such work, the modularity afforded by material that is licensed for modification and sharing provides an efficient means to accommodate the accessibility needs of students with a variety of needs from having low bandwidth to those with disabilities (Almeida 2017). To address the goal of open learning and educational equity, the OER framework has encouraged greater consideration of accessibility requirements and the active participation of instructors in ensuring that such ideals are met and maintained.

What does the future hold for OER at my institution? My current interest is to use the modularity and flexibility of the IMathAS platform to expand beyond the STEM fields which have traditionally utilized it. To that end, the LaGuardia Center for Teaching and Learning (CTL) sponsored a pilot professional development seminar to encourage adoption of the system for fifteen faculty members from across the college to examine use cases within their own disciplines. Led by myself and Prof. Alioune Khoule of the MEC Department (author of the first case study in this paper) and technologist Pablo Avila from the CTL, this pilot program engaged STEM faculty as well as professors of accounting, English, and critical thinking. A group of faculty including myself and Prof. Khoule are now interested in implementing our own local fork of the IMathAS project to start an Open Learning LMS that would allow for innovative modification by all at the college with the potential to provide a disruptive technology and pedagogical openness that will change the dynamic of students and faculty from consumers of such technology to developers and users of accessible, open-source learning (cf. Calma and Dickson-Deane 2020).

MyOpenMath in general chemistry classes: Written by Marta Kowalczyk

Context and problem

In Fall 2019, the Natural Sciences Department received a grant funded by New York State as part of the CUNY OER Initiative. In each subsequent year, several courses are in transition to open educational resources using grant funds. The transition involves:

  • Searching for and adapting a textbook
  • Creating and writing a lab manual
  • Adapting or creating online homework assignments as OER

At the start of each semester, we schedule meetings with each course coordinator and their teams to help with the OER search, adjustment, and implementation process.

The goal was to switch from commercial products and allow students to reduce the cost of the chemistry course while maintaining a high level of student learning outcomes.

Here we present guidelines on implementing MyOpenMath as an online homework assignment platform in general chemistry (Gen Chem I and II) classes. In these classes, students use online homework to practice and grasp the learning objectives at their pace. Many commercial platforms are available to ease the burden on the teaching faculty. However, even though highly advanced and very resourceful, a substantial amount of time is required to set up an accurate online homework assessment for students every semester, which comes with its price. The primary questions we asked heading into the process were: how do you use MyOpenMath when you are unfamiliar with HTML code, and can you use the platform for chemistry questions?

Steps taken

We have started using a question bank, already available on the platform. We have created a few fundamental assignments, distributed them to selected sections, and collected students’ feedback. Based on it, we modified the assignments and piloted them in additional sections. In the following semester, we included our questions and learned basic HTML. We asked for help from physics and math professors to help with troubleshooting HTML code. Following students’ feedback, we include links to the OER textbook with the learning objective and hints. Also, in addition to text, we have added links to various OER videos, so students could watch additional explanations in video demonstrations.

Results

MyOpenMath was run as required homework in Gen Chem I classes in the next academic year. As many faculty are teaching this course, we have prepared and distributed all assignments in all sections and provided maximum guidance and support to teaching instructors.

There is a lot of support in the instructor’s forum, and that’s helpful. However, MyOpenMath does not provide any student support. The instructor is responsible for helping their students and resolving their issues. While instructors are learning the setting of MyOpenMath, an assigned faculty provides support to instructors (especially part-time instructors) and their students. After successful implementation in Gen Chem I, we used a similar implementation approach in Gen Chem II.

Takeaways

The learning curve can be steep for an instructor implementing this platform in their class; Figure 4 compares the student view with the professor’s view in MyOpenMath. Anyone planning to use MyOpenMath in general chemistry (especially those with limited HTML experience) would be best served by starting small; learn the differences between assessment, block, and forum. Be sure to check for available questions in the data bank. As with other OER, a lot of material you need may already be covered—look for existing templates and promoted courses. When it comes to existing questions, test them first because they may require modifications. If you plan to write your own questions, modify existing multiple-choice questions, which can help familiarize you with HTML. Modifying code is much easier than starting from scratch, especially for complicated questions and randomizations. Ask your colleagues for help and use the faculty support group on the platform. Finally, first use your questions in a low-stakes assignment and get feedback from students. Their feedback is critical to improving your questions and overall assessments.

An example of an online chemistry question created with the program MyOpenMath. The image shows the question as students see it and as it appears with HTML code as a professor creates it.
Figure 4. Example of chemistry question. Students can utilize PheT simulations to answer questions (insert A). Example of HTML code for this question (insert B). Questions written in pair with the code (insert C).

During the pandemic, faculty started to record more videos, tutorials, and how-to resources. The video demonstrations were initially planned to be used during the pandemic; however, after positive feedback from students, they became a part of their learning process beyond the textbook and lecture slides while doing online homework. The aspect of accessibility of multimedia and visual aids requires more attention. While faculty focus primarily on the content, learning objective, and accurate demonstration, we are heavily relying on the accessibility office to help students with educational needs.

Online homework assignments for Gen Chem classes are continually improving every semester (see Figure 5), including introducing new resources, creating further questions, and increasing students’ learning outcomes while keeping it at zero cost and advancing student learning and success.

Factors that move implementation of MyOpenMath
Three gears are shown from top to bottom, each one interlocking with the next and arrows added to indicate gear movement. The top gear is labeled Students' feedback. The middle gear is labeled Faculty feedback. The lowest gear is labeled Improving questions.
Figure 5. Factors taken into consideration before full implementation of MyOpenMath in general chemistry classes.

Conclusion

These case studies demonstrate the cultivation of OEP at LaGuardia. Each study acknowledges the importance of lowering or eliminating the cost of textbooks, which is often the initial motivation for using OER. Cost is critical, but other factors are just as important for LaGuardia students and instructors: increased access, quality of teaching materials, pedagogical consistency, curricular alignment, and automated grading, among many others. These factors necessitate the process-oriented approach detailed in the above studies. This iterative process requires flexibility, consistent self-examination, and a willingness to develop new skills.

Alioune Khoule describes how once the MEC Department selected MyOpenMath, math instructors meticulously implemented the new platform, with particular attention to copyright, accessibility regarding technological access and for students with disabilities, and organization of teaching materials. To implement OER at scale, MEC needed course materials to be clearly organized for the dozens of professors teaching the courses using the new platform (Lumen Learning). Joshua Tan’s motivation for using MyOpenMath has as much to do with effective and efficient student assessment methods as it does with eliminating the cost of expensive textbooks and learning platforms. His novel approach has pushed MyOpenMath’s boundaries into astronomy, and potentially beyond, as he and Professor Khoule teach others how to use the platform. Marta Kowalczyk describes how the relatively inexperienced instructor can successfully adopt MyOpenMath in general chemistry courses. Moreover, chemistry instructors at LaGuardia are continually improving their open question banks based on student and faculty feedback, as well as increasing their facility with HTML coding. These issues all point to the fact that implementing OER consists of a continual movement toward effective openness and cannot be constrained by a single endpoint or goal.

For the first several years, the CUNY OER Initiative, funded by New York State, focused on outcomes like the number of courses converted to OER and students’ textbook savings as the primary metrics included in annual reporting. To this day, each college’s grant allocation is largely determined by how many courses and sections will be converted. At LaGuardia, funds typically go toward paying the faculty member responsible for a course conversion. Professional development stipends are available to train instructors who will teach sections using OER. For better and for worse, this work, from selecting OER to professional development, is left to the individual colleges. This approach helps foster local communities of practice, but it does not provide system-wide support and coordination for critical issues like accessibility via Universal Design for Learning (UDL). To its credit, the CUNY Office of Library Services is supporting work across the system that does not fit neatly into OER conversion (Fiddler and McKinney 2021, 8–10). But there is little incentive to support the work required to revisit courses after they have been converted to OER. Instead, the current model prioritizes OER implementation, like switching to an OER textbook. In other words, the grant should do more to support ongoing open educational practices (OEP).

Future Directions

Accessibility and UDL demand greater attention. Making learning materials accessible for students with disabilities is not a measurable outcome of the New York State grant. A course may have zero textbook costs (ZTC) and have no accessibility features in its materials, but it will still meet the requirements of the grant. Alternately, a course may be ZTC and have stellar accessibility features and equally meet the requirements of the grant. If openness is about being open to all regardless of ability, accessibility should be central to grant-funded projects. Further equity in open education lies at the intersection of OER and accessibility/UDL. We must center these principles from the beginning of our workflows. Content that is born digital should also be born accessible (Parks 2015). Evaluating accessibility while a resource is being created contributes to a better adoption once the resource is published in a virtual learning environment or in a repository (Avila et al. 2020). The OER program at Bucks County Community College offers a model for consideration (Bornak et al. 2017). Their OER course template process is supported by a team that includes an accessibility advocate. The advocate meets with instructors throughout the OER conversion process to address accessibility questions and issues as they arise. Our pedagogical practices require a fundamental and sustained shift in design to make learning materials fully accessible. For example, students in biology classes are often required to label diagrams, and, as an accessible stopgap, we can tag images with alt-text. But what would it mean to make such an assignment less reliant on visual faculties? In this way, accessibility fits into a process-oriented view of OER, a continuous movement toward openness and accessibility.

Future and ongoing CUNY OER initiatives should explicitly prioritize accessibility in their metrics of success and include budget line items for the additional costs associated with doing so (Parks 2015). A one-size-fits-all approach to compensation may not be suitable with this change. Dimensions to consider are academic disciplines (size of work for disciplines relying on tables and diagrams to communicate critical information), our proficiency with best and effective practices, training and ongoing support, and whether we are creating our own content or adapting open content. There is an even greater level of effort required to convert non-accessible course materials to accessible OER (Affordable Learning Georgia, n.d.; CAST, 2019). This effort includes accessibility reviews and remediation, or technical assistance to meet certain standards. Finally, budgets should include funds to include and compensate students with disabilities in usability testing throughout the design process.

To this end, LaGuardia is implementing OER and foregrounding accessibility in projects beyond the above case studies. One project is creating the first textbook for LaGuardia’s First Year Seminar (FYS) courses. Instead of replacing a high-priced textbook, this project builds upon other, similar OER textbooks to center accessibility and open pedagogy in the FYS (Baldwin 2020; Campagna et al., n.d.). Student voices have been centered as content reviewers for this project. Future plans include employing students with disabilities to review the content and accessibility of the textbook. The OER professional development seminar described in Joshua Tan’s case study dedicated a sizable portion of time to accessibility to help ensure it is included from the outset of OER projects. Librarians and instructional technologists serve as OER liaisons for LaGuardia’s academic departments to highlight accessibility in their OER projects. To date, the liaisons have worked with classroom faculty in Education and Language Acquisition, English, Health Sciences, and Natural Sciences.

Not only does this foreground the process-oriented nature of accessibility, but it also points to the process-oriented nature of open educational practice. Seminars and presentations get participants thinking about OER and accessibility from an early stage, and, while it may not yield a clear deliverable like an OER course conversion right away, it raises awareness. It builds the culture of OEP. The holistic approach underway at LaGuardia can deliver measurable outcomes for the New York State grant, but it can do much more by moving the college community toward a more just and equitable pedagogy.

References

Almeida, Nora. 2017. “Open Educational Resources and Rhetorical Paradox in the Neoliberal Univers(ity).” Journal of Critical Library and Information Studies 1, no. 1. https://doi.org/10.24242/jclis.v1i1.16.

Avila, Cecilia, Silvia Baldiris, Ramon Fabregat, and Sabine Graf. 2020. “Evaluation of a Learning Analytics Tool for Supporting Teachers in the Creation and Evaluation of Accessible and Quality Open Educational Resources.” British Journal of Educational Technology 51, no. 4: 1019–1038. https://doi.org/10.1111/bjet.12940.

Bailey, Thomas R. (Thomas Raymond), Shanna Smith Jaggars, and Davis Jenkins. 2015. Redesigning America’s Community Colleges: a Clearer Path to Student Success. Cambridge, Massachusetts: Harvard University Press.

Baldwin, Amy. 2020. College Success. Houston: OpenStax. https://openstax.org/books/college-success/pages/1-introduction.

Baran, Evrim, and Dana AlZoubi. 2020. “Human-Centered Design as a Frame for Transition to Remote Teaching During the COVID-19 Pandemic.” Journal of Technology and Teacher Education 28, no. 2: 365–372. https://www.learntechlib.org/primary/p/216077/.

Benjamin, Jesse, and Seneca Vaught. 2018. “Pedagogic Paths to Liberation: Personal Struggles, Institution Building and the Rise of the Walter Rodney School of Groundings Praxis in Atlanta.” The Journal of Intersectionality 2, no. 1: 6–30. https://doi.org/10.13169/jinte.2.1.0006.

Bornak, Mary Ellen, Jacqueline Burger, Linda McCann, and Bill Hemmig. 2017. “Open Educational Resources: An Initiative at Bucks County Community College.” TCLC Program Proceedings 10. https://rdw.rowan.edu/tclc_proceedings/2017/All/10.

Calma, Angelito, and Camille Dickson-Deane. 2020. “The Student as Customer and Quality in Higher Education.” International Journal of Educational Management 34, no. 8: 1221–1235. https://doi.org/10.1108/IJEM-03-2019-0093.

Campagna, Michele, Adam Frank, Hector Graciano, Robin Graff, Rachele Hall, Leonore Rodrigues, and Christine Thomas. n.d. Westchester Community College: First-Year Seminar. Accessed November 22, 2022. https://courses.lumenlearning.com/suny-wcc-collegesuccess2/.

“Cape Town Open Education Declaration.” September 2007. https://www.capetowndeclaration.org/read/.

CAST. 2019. “CAST & ISKME Partner to Make OERs More Inclusive, Effective.” https://www.cast.org/news/2019/cast-iskme-partnership-oer-commons-open-authoring-system.

Crippen, Kent J., and Leanna Archambault. 2012. “Scaffolded Inquiry-Based Instruction with Technology: A Signature Pedagogy for STEM Education.” Computers in the Schools 29, nos. 1–2: 7–173. https://doi.org/10.1080/07380569.2012.658733.

Cronin, Catherine, and Iain MacLaren. 2018. “Conceptualising OEP: A Review of Theoretical and Empirical Literature in Open Educational Practices.” Open Praxis 10, no. 2: 127–143. http://doi.org/10.5944/openpraxis.10.2.825.

DeRosa, Robin, and Scott Robison. 2017. “From OER to Open Pedagogy: Harnessing the Power of Open.” In Open: The Philosophy and Practices That Are Revolutionizing Education and Science, edited by Rajiv S. Jhangiani and Robert Biswas-Diener, 115–124. https://doi.org/10.5334/bbc.i.

Duke University. “Manual Accessibility Testing Checklist.” March 7, 2017. https://spotlight.duke.edu/communications/wp-content/uploads/sites/25/2017/11/Accessibility-Testing-Checklist3.pdf.

Fiddler, Ann, and Andrew McKinney. 2021. “Making Open and Affordable Courses: Best Practice Case Study.” In New York State Open Educational Resources Funds: CUNY Year Three Report, 5–10. https://www.cuny.edu/wp-content/uploads/sites/4/page-assets/libraries/open-educational-resources/CUNY_OER_Report3_Web_V32.pdf.

Jisc Accessibility and Inclusion. 2016. “Word or PDF? What’s the Learner Experience Likely to Be?” Accessibility, Assistive Technology and Inclusive Practice, September 13, 2016. https://accessibility.jiscinvolve.org/wp/2016/09/13/word-pdf/.

Khoule, Alioune, Abderrazak Belkharraz Idrissi, and Sandra Sze. 2021. “Large Scale Open Educational Resources (OER) Initiative in Mathematics.” Journal for the Mathematics Education and Teaching Practices 2, no. 2: 55–69. https://dergipark.org.tr/en/pub/jmetp/issue/66397/1012331.

Kirschner, Paul A., and Kwok‐Wing Lai. 2007. “Online Communities of Practice in Education.” Technology, Pedagogy and Education 16, no. 2: 127–131. https://doi.org/10.1080/14759390701406737.

LaGuardia Community College, Office of Institutional Research & Assessment. 2021. LaGuardia Community College: Institutional Profile 2021. https://www.laguardia.edu/uploadedfiles/main_site/content/ir/docs/institutional-profile-2021.pdf.

Lambert, Sarah R. 2018. “Changing our (Dis)ourse: A Distinctive Social Justice Aligned Definition of Open Education.” Journal of Learning for Development 5, no. 3: 225–244. DOI: https://doi.org/10.56059/jl4d.v5i3.290.

Logan, Charles. 2021. “Toward Abolishing Online Proctoring: Counter-Narratives, Deep Change, and Pedagogies of Educational Dignity.” Journal of Interactive Technology and Pedagogy 20. https://jitp.commons.gc.cuny.edu/toward-abolishing-online-proctoring-counter-narratives-deep-change-and-pedagogies-of-educational-dignity/.

Lowyck, Joost. 2002. “Pedagogical Design.” In Handbook on Information Technologies for Education and Training, edited by Heimo H. Adelsberger, Betty Collis, and Jan M. Pawlowski, 199–217. Berlin, Heidelberg: Springer. https://doi.org/10.1007/978-3-662-07682-8_13.

Parks, Kaela. 2018. “Open and Inclusive Education – Connections to Universal Design.” In The Evolution of Affordable Content Efforts in the Higher Education Environment: Programs, Case Studies, and Examples, edited by Kristi Jensen and Shane Nackerud. Minneapolis: University of Minneapolis. https://doi.org/10.24926/86666.0101.

Tan, Joshua, Allyson Sheffield, and Jana Grcevich. 2021. “An Autograded, Open Educational Resource Assessment Tool for Astronomy Using MyOpenMath.” arXiv preprint. https://doi.org/10.48550/arXiv.2111.15604.

Teixeira, António, Carlos João Correia, Filipa Afonso, Antonio García Cabot, Eva García López, Salvador Otón Tortosa, Nelson Piedra, Luciana Canuti, Jacqueline Guzmán, and Miguel Ángel Córdova Sol. 2013. “Inclusive Open Educational Practices: How the Use and Reuse of OER Can Support Virtual Higher Education for All.” European Journal of Open, Distance and E-Learning 16, no. 2. https://old.eurodl.org/index.php?p=special&sp=articles&inum=5&article=632.

Tijerina, Tiffani. 2021. “OER Accessibility Series and Rubric: Document Design.” Affordable Learning Georgia. https://alg.manifoldapp.org/projects/oer-accessibility-series-and-rubric.

UNESCO. 2002. “Forum on the Impact of Open Courseware for Higher Education in Developing Countries: Final Report.” Last modified July 2002. https://unesdoc.unesco.org/ark:/48223/pf0000128515.

UNESCO. 2012. “2012 Paris OER Declaration.” World Open Educational Resources (OER) Conference. June 22, 2012. https://unesdoc.unesco.org/ark:/48223/pf0000246687.locale=en.

Wiggins, Grant P., and Jay McTighe. 2005. Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development.

About the Authors

Ian McDermott is an associate professor and Coordinator of Library Instruction in the Library Department at LaGuardia Community College, City University of New York. He serves as LaGuardia’s OER Liaison, and his research examines the intersection of open educational practice and critical pedagogy.

Alioune Khoule is a Professor of Mathematics in the Department of Mathematics and Computer Science at LaGuardia Community College, City University of New York. His research interests are in mathematics modeling and mathematics education. His current research focuses on the impact of teaching-based concepts in developmental courses and statistics.

Joshua Tan is an assistant professor of astronomy and physics in the Natural Science Department at LaGuardia Community College, part of the City University of New York. He is also a research associate at the Astrophysics Department of the American Museum of Natural History, New York. In addition to pedagogical innovation, Prof. Tan’s research interests include high-energy astrophysics, remote astronomical observations using small telescopes, and the theory and observations of binary millisecond pulsars.

Marta Kowalczyk is an associate professor of chemistry in the Natural Science Department at LaGuardia Community College, City University of New York. Her research focuses on the experimental and computational photochemistry of inorganic and organic compounds. Her expertise is in charge transfer mechanisms and conjugated polymers. She is currently helping the chemistry and biology faculty in the OER conversion of their classes.

Rena Grossman is an adjunct OER Librarian at LaGuardia Community College and Hostos Community College, City University of New York. She is currently helping faculty in the OER conversion of their classes, and is particularly interested in helping them find media sources for multi-sensory types of learning. Her expertise is at the intersection of accessibility and critical disability studies, all well as urban agriculture.

Emma Handte is an OER Instructional Technologist at LaGuardia Community College, City University of New York. She is also the Project Coordinator for Global Scholars Achieving Career Success, a program that connects CUNY students to students in the Middle East and North Africa through virtual exchanges. She holds a master’s degree in Middle Eastern Studies, and her interests lie at the intersection of open education and global learning.

0

Immersive Pedagogy: Developing a Decolonial and Collaborative Framework for Teaching and Learning in 3D/VR/AR

Abstract

In June 2019, a cohort of CLIR postdoctoral fellows convened Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality at Carnegie Mellon University. The symposium sought to bring together a multidisciplinary group of collaborators to think through pedagogical issues related to using 3D/VR/AR technologies, as well as to produce and disseminate materials for teaching and learning. This essay presents the Immersive Pedagogy symposium as a model for interrogating and developing pedagogical practices and standards for 3D/VR/AR; we offer a decolonial, anti-ableist, and feminist pedagogical framework for collaboratively developing and curating humanities content for this emerging technology by summarizing the symposium’s keynotes, workshops, as well as its goals and outcomes. Workshops, keynotes, and participant conversations engaged with decolonial and feminist methodologies, practiced accessible design for universal learning, offered templates for humanistic teaching, and illustrated the possibilities of using 3D/VR/AR to extend critical thinking. While 3D/VR/AR technologies demonstrate real possibilities for collaborative, multidisciplinary learning, they are also fraught with broader concerns prevalent today about digital technologies, as well as complex issues specific to 3D/VR/AR. There is a clear need to assemble academic practitioners on a regular basis in order to facilitate an ongoing discussion about 3D/VR/AR technology and its responsible, meaningful use in teaching and learning.

Introduction

As access to three-dimensional (3D) technologies has become increasingly available in academic venues, the desire to teach with these emerging technologies, particularly augmented reality (AR) and virtual reality (VR), has outpaced digital humanists’ abilities to provide meaningful support for immersive projects. There is a growing and ongoing need to produce shared and open pedagogical materials adaptable to the needs of teachers in various professions and disciplines and are accessible to students without significant coding experience. This need is partially driven by the contingencies of relatively new and rapidly updating technologies, as well as the fact that support for commercially-available immersive tools are tailored for industry purposes. Game-driven tutorials, for example, do not always take into consideration the needs of humanities practitioners seeking to integrate critical thinking with technical mastery. Contemporary contexts for emerging technologies can structure our interactions with 3D/VR/AR. Though not always visible to users, these can have the effect of naturalizing problematic historical and political narratives through selective access to resources and functionality.

Nonetheless, game engines that offer free educational licenses have been repurposed for academic inquiry and teaching over the past decade. For example, Unity Technologies’ Unity 3D game engine is utilized by over 4.5 million users and has been at the forefront of historical and archaeological 3D visualizations in scholarly research. First available in 2005, the Unity 3D game engine has been used to make approximately 60% of all AR/VR applications and is used by 90% of AR/VR companies (“Public Relations” 2019, np). Educational licenses are available for students and educators seeking to use the engine for scholarly or creative use. Its main competitor, the Unreal Engine, while initially inaccessible beyond professional and academic institutions with licenses, dropped its paywall for educational use in September 2014. VR headsets, once a hypothetical fantasy or niche short-lived technology, are now commercially viable and relatively inexpensive for institutions to purchase. In a few years, the financial barrier for individuals may diminish; in the meantime, Google Cardboards and other stereoscopic viewers with fewer interactive features currently provide alternatives for students with access to smartphones. However, students are also increasingly able to make use of interactive 3D/VR/AR technology within dedicated spaces in academic libraries, maker spaces, media studios, and community outreach centers. Yet, we would be remiss not to point out that access is still mediated by other social hierarchies; 3D/VR/AR technology is still not accessible in much of the Global South, or in marginalized communities across the world. These aforementioned developments still privilege students at institutions that dedicated staff or faculty to maintain and encourage use of 3D/VR/AR technologies and facilities.

This is all to say that in our current 3D/VR/AR moment, digital humanists have a lot to navigate. Current 3D/VR/AR pedagogy and projects can pose problems related to accessibility and long-term preservation of projects and assets, and often run afoul of minimal computing recommendations. Yet the technology offers rich possibilities for multidisciplinary research and collaboration; many virtual reality projects combine art production, computing, archival research, network theory, and data visualization, among other practices. Given its potential for scholarship and teaching, understanding how to use the technology responsibly necessitates engaging with active practitioners to identify what is now possible and what still needs to be done to facilitate productive use of 3D/VR/AR. As many key problems are likely to persist through subsequent permutations of the technology and its use in educational settings, this conversation needs to be ongoing and open. What humanists within and beyond the academy have to say about 3D/VR/AR will probably not be unique to humanistic inquiry. This dialogue will provide crucial critical approaches to the emerging technologies’ advantages and limitations that will be of use to industry professionals as well as the casual creative user. A vocal contingent of humanists seeking to think and learn with 3D/VR/AR may, in fact, fill a wider sociocultural need by addressing these issues.

This is the context in which a small cohort of 2017–2019 Council on Library and Information Resources (CLIR) Postdoctoral Fellows organized Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality at Carnegie Mellon University on June 26 and 27, 2019. The CLIR cohort included Lorena Gauthereau (University of Houston), Jessica Linker (Bryn Mawr College), Eric Kaltman (Carnegie Mellon University), Emma Slayton (Carnegie Mellon University), Neil Weijer (Johns Hopkins University), Alex Wermer-Colan (Temple University), and Chris Young (University of Toronto). The goal of this symposium was to assemble a wide range of stakeholders to develop teaching materials and strategies that considered problems inherent and specific to immersive technologies, as well as to address problems that affect but are not unique to 3D/VR/AR. It is for this reason the symposium was so attentive to decolonial and feminist methodologies in thinking about appropriate pedagogical applications. Building on the previous work of scholars such as María Cotera, Elizabeth Losh, Tara McPherson, Angel Nieves, Roopika Risam, and Jacqueline Wernimont, we have advocated for an intersectional digital humanities that interrogates a wide range of technologies through the critical methods developed by the fields of ethnic and feminist studies. Such methods, we argue, can highlight the ways that technologies often leave out marginalized people by replicating colonial hierarchical structures including race, ethnicity, class, gender, and disability.

The Immersive Pedagogy symposium offered an early—if not first-of-its-kind—opportunity to have productive conversations about what critical approaches to 3D/VR/AR could look like from a multidisciplinary and multi-professional perspective. Additionally, the symposium sought to seed collaborations within and beyond academic institutions and stand as a model for future conversations on these topics. In recounting our experiences with different applications of 3D/VR/AR technology in pedagogical spaces, the group tackled a number of thorny issues, such as accessibility in hardware and bias in asset stores, while acknowledging that we would need to continue the dialogue by reconvening. We sought to develop teaching materials collaboratively with the long-term plan of sharing these resources through a variety of means, including via open-access publications. In the remainder of this essay, the Immersive Pedagogy organizers describe the symposium’s theoretical foundation and methodological approaches as a model for structuring communities around 3D/VR/AR, summarize some of our group’s findings, and invite digital humanities practitioners to help us to continue this work.

Structuring a Symposium on Decolonial Models of Immersive Pedagogy

Because the initiative was organized by CLIR postdoctoral fellows, the symposium emphasized diverse ways that libraries participate in creating, curating, and preserving 3D/VR/AR pedagogical materials. We considered faculty, staff, and students as equal partners in 3D/VR/AR projects, and aimed to include early career researchers at the table. Overarching goals for the symposium included teaching faculty and librarians how to support and enable learning for students using 3D technologies, and to help students to disseminate skills within their own communities. By bringing together scholars from a wide range of disciplines and professions, we addressed problems while identifying new ones. Participants had the opportunity to share links and descriptions to their projects (current and in progress) with each other prior to the symposium via a Slack channel and Google Docs. They also shared information on their work during a lightning talk round as examples of the kinds of humanistic projects 3D/VR/AR could cultivate. The symposium began and ended with keynotes from experienced practitioners whose work modeled creative and responsible uses of the technologies.

Our opening keynote speaker, Angel Nieves (Associate Professor of History and Digital Humanities at San Diego State University), presented “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities.” Nieves’s talk outlined strategies for achieving social justice through digital-supported inquiry, highlighting his own work on Mapping Soweto, a 3D reconstruction of apartheid South Africa. Nieves emphasized the need to ground digital work in women of color theory and argued that fields such as ethnic studies have developed a foundational structure that would benefit the field of digital humanities as a whole:

If we brought the sorts of methodological and practice-based questions about power, privilege, and access from ethnic studies to our work in immersive technologies, we might begin to see new ways of harnessing these tools–that originated as part of the military industrial complex–to serve our social justice needs. (Nieves 2019)

Mapping Soweto draws from Belinda Robtnett’s (1997) work on social movement theory, revealing the often messy, multilayered narratives of social movements by visualizing a map of spatial liberation. This 3D representation shows what Nieves terms an “intersectional cartography,” or a network of social activists—especially networks of women and young girls—across townships “and how those activist networks were embedded into the physical geography and vernacular architecture of individual houses, streets, and neighborhoods” (Nieves 2019). Attention to intersectionality further reveals the ways multiple identities—township, gender, sexual orientation, class, and race—came together to form a cohesive activist movement, whose complexities are often lost in the official retelling of history. In particular, Nieves identified immersive technologies as one way to “re-establish coalition-building potential” (2019) with local communities and reminded us that the important work of recovering marginalized histories for social justice is often messy.

Two image composite. Top image is of Angel Nieves standing behind a podium delivering his keynote speech. Bottom is a slide showing a Unity 3D model of Winnie and Nelson Mandela House, in Soweto South Africa (generated September 2018).
Figure 1. Angel Nieves presents “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities” at the Immersive Pedagogy symposium.

Our closing keynote speaker, Juliette Levy (Associate Professor of History at the University of California, Riverside), presented “How Not to be a Replicant: Working Towards a Useful VR.” Working with a team of women programmers, Levy has developed VR simulations for teaching abstract concepts related to historical thinking, interpretation, and writing. Levy’s keynote presentation focused on the question of gaming and interactivity; and she traced the origin of her experimentations in VR from teaching large lecture classes numbering in the hundreds in hybrid and online courses. Rather than approach VR in the mode of cultural heritage projects, reproducing a historical location, to deal with pedagogical problems commonly experienced in online learning, Levy’s team built Digital Zombies (see Levy 2017), an abstract simulation meant to introduce students to the hierarchy of library information and assessment of primary and secondary resources through game-based learning. Levy envisioned a VR environment for her historical research methods class that not only encouraged students to follow a written outline of research steps, but to extend their library experience in a more immersive, playful way by completing a series of game-like missions related to research that students would be more likely to remember. Levy argued that the cognitive effect of a VR experience has a lasting impact on users: “What matters about doing something in VR isn’t about what happens in VR, but what happens outside of VR, after the VR experience” (Levy 2019). Yet, despite the advantages of VR, Levy warned that a lack of critical conversation and pedagogy around digital literacy can have dire consequences, as increasingly ubiquitous immersive technologies become exploited to misrepresent historical events. The stakes for fomenting critical conversations between technology creators, consumers, and scholars, therefore, are quite high, as they could have lasting effects on how people choose to build and interpret virtual representations of historical events and people.

Juliette Levy stands at the podium while presenting a slide reading “fake news, fake history, alternative facts, virtual reality or fake reality” in front of an image of John Lennon and Che Guevara playing the guitar.
Figure 2. Juliette Levy presents “How Not to be a Replicant: Working Towards a Useful VR” at the Immersive Pedagogy symposium.

The symposium included five workshops that centered on theory, methods, and practices significant to and capable of incubating pedagogy related to US Latinx, Latin American, and Caribbean studies, which we prioritized when considering applicants. The workshop topics were: (a) Decolonial Methodology and Theory; (b) Accessible Immersive Pedagogy; (c) Integrating Immersive Technology in the Classroom; (d) Critical Writing for Immersive Tech; and (e) Collaboratively Designing 3D/VR Experiences. The Immersive Pedagogy organizers, joined by Jasmine Clark (Temple University) and Juliette Levy, led the participants through these interactive workshops (“Program” 2019). Pedagogical content crafted by participants before, during, and after the symposium included a bibliography of 3D/VR/AR-related readings, an archive of workshop slides, video recordings of keynote presentations, adaptable templates for pedagogical activities, and working models of 3D/VR/AR pedagogical applications. For example, Kat Hayes and Samantha Porter submitted a video walkthrough of their IOS app Virtual MISLS that explores historic buildings at Fort Snelling, while Meaghan Moody and Carol Salmon submitted a description of their work with students using a virtual map of historic Paris to better understand life under German occupation during World War II.

Carnegie Mellon University (CMU) Libraries hosts the symposium’s materials on its institutional repository, KiltHub. KiltHub provides stable, long-term global open access storage for 3D/VR/AR assets, and functional applications, as well as pedagogical and technical documentation. Materials in this repository are held for a minimum of ten years, ensuring that what is submitted will remain available past typical terms of software updates. The teaching materials produced during and following the symposium will also be published in the Digital Library Federation’s Pedagogy Working Groups open-access series, the DLF Teach Toolkit. The materials will be revised and tested, including during a pre-conference workshop at DLF’s Annual Forum 2020, pending acceptance.

Group of twenty-one symposium attendees sitting in a circle actively engaged in a workshop discussion session.
Figure 3. Immersive Pedagogy symposium participants in discussion.

The following essay sections explore the key components of the symposium, which outlined the theoretical foundations to decolonizing development and curation of 3D/VR/AR tech, before guiding participants through workshops on decolonial critique and accessible design, on integrating immersive technology into the classroom and beyond, and on collaboratively designing 3D/VR projects.

Decolonial Foundations: Critical Approaches to the Development and Curation of 3D/VR/AR Technologies

To practically introduce the decolonial methodologies and theories crucial to our workshops on developing and curating 3D/VR/AR materials, the Immersive Pedagogy symposium opened with a workshop, led by Gauthereau and Young on the “walkthrough method” (Light, et. al. 2018, 881–900), a critical analysis of technology using the Unity Asset Store as an example. This exercise was contextualized through a theory of decolonial pedagogy and a discussion on the critical analysis of the game platforms that curate content for 3D modeling and representation.

The application of decolonial theory and methods to digital pedagogy allows students to interrogate and resist colonial, hierarchical epistemologies, especially the privileging of Western European and Anglocentric knowledge structures. Such an approach is increasingly necessary as 3D/VR/AR technologies become integral to Western education systems and overwhelmingly applied to cultural heritage projects by and for Western consumers. While colonialism refers to the “political and economic relation in which the sovereignty of a nation or people rests on the power of another nation,” making that nation an “empire,” coloniality “refers to long-standing patterns of power that emerged as a result of colonialism, but that define culture, labor, intersubjective relations, and knowledge production well beyond the strict limits of colonial administration” (Maldonado-Torres 2007, 243). Thus, coloniality denotes the ways in which colonial hierarchies of power continue to structure our everyday lives (i.e. racialized class hierarchies, labor hierarchies, gender hierarchies, the gender binary, racism, etc.). Decolonialism urges us to actively de-link from colonial epistemologies and ontologies in order to avoid re-creating colonial worldviews and hierarchies.

Considering the ways that 3D/VR/AR technologies allow users to create immersive worlds and environments, the symposium sought to stress the need to avoid replicating the colonial gaze. Representing marginalized people through this gaze continues to enforce racialized and gendered hierarchies of power. Colonial epistemologies continue to control knowledge production, not only through institutional archives, but also through academic research, digital projects, and 3D/VR/AR environments. Jacqui Alexander and Chandra Talpede Mohanty argue that decoloniality has a “pedagogical dimension” as it obligates us “to understand, to reflect on, and to transform relations of objectification and dehumanization, and to pass this knowledge along to future generations” (1997, xxviii-xxix). For this reason, the symposium’s first workshop exercise involved guiding participants through a decolonial walkthrough of the Unity Assets Store. The walkthrough method requires researchers to directly engage with “an app’s interface to examine its technological mechanisms and embedded cultural references to understand how it guides users and shapes their experiences” (Light, et. al. 2018, 882). We asked participants to browse and search the Unity Assets Store for 2D, 3D, audio, and animation assets and interrogate them using a decolonial approach, as well as to document their walkthrough by taking notes, taking screenshots, and recording audio-visual content.

To guide the decolonial inquiry, we asked participants to consider a set of questions adapted from Roopika Risam’s discussion of the stakes of postcolonial and decolonial digital humanities (2019, 35–46):

  • What are compulsory activities within the Unity Asset Store?
  • What are the social hierarchies within the menu system?
  • To whom and which types of users is this knowledge accessible?
  • What is considered a “legitimate” asset within the Asset Store?
  • Whose epistemologies, such as histories, languages and memories, are considered important enough to archive in the Asset Store?
  • What knowledge or assets are privileged within the Asset Store?
  • Does the asset avoid the exoticization or fetishization of a people/cultures?

This inquiry resulted in participants recognizing the disproportionate representation of a Eurocentric worldview. For example, they noticed that the search term “Viking” yielded twice as many results as “Native American,” whereas the term “Indigenous” yielded zero. Among results for the search term “Mexican,” participants discovered a Mexican Restaurant Pack that reflected generalized stereotypes of Mexican aesthetics and cuisine, reduced to bottled hot sauce, chips and salsa, and a decorative green parrot. Assets also reduced the multiple and varied cultures, nations, flora, and fauna of the entire African continent to the myopic colonial imaginary of only the Serengeti, populated by wild animals. During group break-out sessions exploring the Asset Store, participants discovered a potential intervention through editing crowdsourced user tags. Like during Wikipedia Edit-a-Thons, users could challenge the authenticity of colonial representations of people, cultures, and nations by tagging or reviewing assets as not authentic, representing stereotypes, reproducing colonial views, etc. Since the symposium, unfortunately, Unity has removed the user tagging option and currently limits metadata generation to the individual uploading the asset.

This workshop stressed that engaging in decolonial work requires a constant questioning of how knowledge (3D/VR/AR environments, research, stories, syllabi, etc.) is being produced, who is producing it, whose stories are being told, and how these stories are being told. Not only should we consider what histories are told in the digital world, but we must also attend to the ways in which they are produced. As a result, the participants learned that generating and interacting with 3D/VR/AR environments they must use decolonial methods to acknowledge their role as world-creators and reflect on the ways that these technologies often replicate colonialism.

In the following workshop, Clark foregrounded the ableism endemic to technological innovation in the West, introducing participants to accessible user design for virtual reality. This involved a tutorial on developing alternate access plans for disabled students in classrooms. Clark’s work with Temple Libraries’ colleagues Jordan Hample and Wermer-Colan has prioritized research into and creation of accessible features for VR during their development of the Virtual Blockson: A Primary Source Teaching Tool for Secondary Education (Clark 2018, np). Clark’s workshop overviewed the standards of the World Wide Web Consortium’s Web Content Accessibility Guidelines in order to showcase the problems with applying standards created for web-based screens to virtual reality environments and experiences. She related an overview of key advancements that can be made to enable universal design for this emerging technology ranging from innovations in haptic feedback to caption legibility. Clark’s talk focused on guiding participants through strategies for accessing resources for disabled students at their universities. She led participants through an exercise with a template she created for developing “alternate access plans” that enable teachers to offer comparable options for students who cannot use the available VR and/or AR hardware and software. This approach to accommodating students with different learning styles provides a realistic way for teachers to work with emerging technologies in academic institutions, most of which still lack sufficient resources to support disabled students in the use of analog technologies.

Virtual Lessons: Integrating Immersive Technology in the Traditional Classroom and Beyond

After the symposium’s opening workshops on decoloniality and anti-ableism in immersive pedagogy, Levy’s workshop put to practice the principles she laid out in her closing keynote address on the idiosyncratic game mechanics for simulating virtually interactive dialogue and exam questions involving classification. VR offers, Levy argued, a unique pedagogical opportunity, functioning as a distraction-free zone where her students were able to recollect experiences at a much higher rate compared to other learning activities. During the workshop, Levy asked symposium participants to select several library books from various library collections and work in groups to think about how to put the texts in conversation with each other based on titles, subject headings, table of contents listing, and a quick skim of their contents. Levy demonstrated how and why she constructed a VR environment that simulated this activity, as her students had to physically place boxes with various titles onto empty shelves in an order that reflected connections. The application of VR to this type of historiographical exercise, Levy maintained, left a lasting impression on the students that they were able to put into practice for essay assignments. Levy’s emphasis throughout her workshop on the pedagogical significance of “what happens before and after” the virtual experience, furthermore, offered a valuable foundation for the subsequent workshop on integrating writing exercises to guide student learning during virtual and augmented reality experiences.

Wermer-Colan’s workshop modeled how to guide undergraduate students across the disciplines through a structured composition exercise for reflection, in particular, by guiding the participants through a reflection on what they hoped to learn and do in the coming school year as they sought to develop their immersive pedagogy projects. To provide a context and model for students before their writing reflections, Wermer-Colan summarized his current projects employing 3D technologies for Temple University Libraries’ Digital Scholarship Center (now the Loretta C. Duckworth Scholars Studio). Temple Libraries has experimented with transforming the purposes of library collections, development, and reference work to enhance its learning and technology outreach, including through its Innovative Teaching with Makerspace Technology Grant and its newly constructed VR Lab in the new Charles Library. Wermer-Colan’s past experiences working in the Medgar Evers College Writing Center in the City University of New York (CUNY) system helped him to think about ways the Digital Scholarship Center can use 3D/AR/VR technology to enhance learning across the disciplines.

As an example of Temple Libraries’ supporting the use of immersive technologies in class-room projects, Wermer-Colan detailed a collaborative project with Ajima Olaghere, Assistant Professor of Criminology working with her ethnography students to do “systematic social observation” of Philadelphia neighborhoods. This project used 360 cameras to record neighborhoods affected by Temple’s gentrification of North Philadelphia. The recordings were later viewed on twenty-dollar Desktek smartphone headsets that allowed students to remotely examine environments to understand what contributes to disorder and crime, while the instructor facilitated ways to maintain a critical understanding of what they were viewing. The accompanying writing exercises guided students to reflect on their mediated experiences of urban space and call into question the “broken windows theory,” common assumptions that visible signs of public disorder exacerbate criminal behaviors. The use of phone-based headsets also invited an opportunity for students to consider the physical processes that enable virtual technology. Instructors were faced with the problem of scaling pedagogical uses of VR; as this project used relatively inexpensive headsets, workshop participants considered how to create immersive experiences similar in quality to those offered by state-of-the-art VR headsets like the HTC VIVE or the Oculus Rift that, as of 2020, cost hundreds of dollars.

To illustrate the role libraries and digital scholarship centers can play in the curation of 3D content for teaching and learning, Wermer-Colan overviewed a complementary use of immersive technologies. His collaboration on the Virtual Blockson project with Digital Scholarship Librarian Jasmine Clark, Academic Technician and Developer, Jordan Hample, and Blockson Archivist Leslie Willis-Lowry aims to recreate Temple’s Charles L. Blockson Afro-American Collection as a virtual reality game for innovating the teaching of primary source literacy in high schools across Philadelphia. The project at its heart allows a small, fixed collection and its reading room to be available to students remotely, lowering the intimidation factor and physical limitations of these spaces, while enabling interactive explorations of historical artefacts. The Virtual Blockson offered an opportunity to discuss how libraries can help curate interactive gaming environments for remediating archival collections and cultural heritage sites to foreground previously marginalized histories. In these contexts, virtual reality offers affordances for lowering the barrier for students to use archival sources and spaces, facilitating access and accessibility, and offering students a novel medium through which to conceptualize analog and digital literacies necessary to navigate the changing new media world today.

3D-rendering of the Charles L. Blockson Afro-American Collection, a few sculptures and a painting on display in the reading room.
Figure 4. Screenshot of the Virtual Blockson designed by Jordan Hample using Unity 3D. For more, see Jasmine Clark’s “Progressing Towards an Accessible VR Experience”: https://sites.temple.edu/tudsc/2018/11/07/progressing-towards-an-accessible-vr-experience/.

Wermer-Colan foregrounded in both these projects the use of writing exercises to help students reflect on their virtual experiences in meaningful ways. The 360 SSO writing exercise encouraged humanistic thinking about the technology by asking students to compare their field work exercises with the virtual experience, as well as writing reflections that asked the students to identify various ways the 360/VR technology mediated said experience. Similarly, humanistic writing exercises were designed to guide students before and after their experience of the Virtual Blockson’s introduction to archival spaces, etiquette, and practices through game-based, interactive experiences. Drawing upon the Society of American Archivists’ Standards for Primary Source Literacy and the Common Core Standards for historical understanding, digital literacy, and critical thinking, these critical writing questions ensure students reflect upon the virtual experience of library collections’ historical artifacts from the African diaspora. After offering these models to the Immersive Pedagogy participants, Wermer-Colan guided the group through a critical writing exercise to reflect on their own plans to implement the 3D/VR/AR technology for various pedagogical purposes. Wermer-Colan encouraged participants to think of resources at their local institutions, pedagogical standards in their disciplines, and affordances in the spatialized medium of VR for enhancing their approaches to teaching. The writing exercise simulated the kind of exercise participants could implement in their own pedagogy, while offering an opportunity for the symposium participants to reflect on what they had learned during the workshops.

Feminist Reconstructions: Collaboratively Designing 3D/VR Experiences

The concluding workshop, run by Linker and Young, offered a sustainable model for including students as partners in the creation of 3D/VR pedagogical materials, through an overview of Linker’s time creating the Bryn Mawr Women in Science project with her various undergraduate partners: Elia Anagnostou, Courtney Dalton, Jocelyn Dunkley, Tanjuma Haque, Arianna Li, and Linda Zhu. From 2017 to 2019, Linker taught undergraduate students how to integrate historical inquiry with 3D technology in order to think about women’s invisible scientific labor, the spaces they occupied, the tools they used, and their everyday lived experiences. The project considers Margaret Rossiter’s “The Matthew Matilda Effect in Science,” which articulates a systematic disparity in affording women scientists credit for sophisticated and important discoveries, which in turn necessitates that historians find ways to tell stories in order to make their labor visible. It likewise adapts aspects of Pamela Smith’s Making and Knowing Project by taking seriously the need to consider scientific processes. However, rather than engaging in physical reenactment, students offered up women’s processes in a modern, digital format, contextualized by a recreation of spaces that were no longer intact or available for historical analysis.

3D rendering of a biology lab created for the Bryn Mawr Women in Science Project. Rendering contains depictions of glassware, scientific artifacts, equipment, and laboratory furniture.
Figure 5. Screenshot of the 3D-rendered Advanced Biology Lab c. 1900, from Bryn Mawr Women in Science.

Linker and her students recreated two laboratory spaces that had once existed at Bryn Mawr College in the late nineteenth and early twentieth centuries, the Major Chemistry Lab and the Advanced Biology Lab.[1] Students learned a variety of 3D skills, including 3D modeling, photogrammetry, various mechanics of the Unity 3D game engine, and the Oculus Rift. Interactive WebGL versions of the project are available online, and a VR demo of Bryn Mawr’s Advanced Biology Lab was available at the conference. The Advanced Biology Lab was the site of early genetic research and a place once utilized by Nettie M. Stevens, the subject of Stephen Brush’s Nettie M. Stevens and the Discovery of Sex Determination by Chromosomes. Years before Margaret Rossiter coined the phrase “the Matilda Effect,” Brush identified that Stevens’ discoveries had been overshadowed by male collaborators or individuals working concurrently on the same subject. Her contributions had likely been diminished because she was a woman. Students researched each space by spending time in Bryn Mawr’s Special Collections. Through building each laboratory, the students became aware of how to put historical materials in conversation, as no resource could tell them everything they needed to know to build and contextualize the 3D models. Pedagogically, the two-year process of building was designed to seed humanistic deployment of 3D technologies by undergraduate collaborators. Afterward these students participated in professional presentations of the digital and historical work, and served as ambassadors to various communities in order to disseminate the skills the project cultivated to a wider audience.

Linker enabled her students to accomplish a lot in a short period of time; no student was an expert in the technology or in historical research prior to their tenure on the project. This was intentional, as she sought to teach rather than to employ experts. Students represented a diverse range of interests and majors, and all students participated in each phase of production (rather than assigning humanities majors to research and STEM majors to coding) so that afterwards, they could create projects similar to this on their own. Part of what facilitated their success is that she treated them as equal partners in the project, making decisions with them throughout the two years they worked together.

To prepare her students to participate as equals, she devised a plan that would serve as an introduction to using 3D technology to address social and pedagogical problems, and would also serve as a diagnostic tool for assessing student strengths and interests. Essentially, students were asked to propose and implement a 35-hour project (which could be run over the course of days or weeks, depending on individual need) that used an aspect of the Unity 3D game engine’s functionality to teach users about something the students cared about. Students drafted plans that identified what they knew, what they needed to learn, and were prompted to think about modularity, such that students could scale the project if they were running out of time. Students who were not familiar with coding at first could use Unity’s GUI interfaces to produce fully functional scenes, allowing for students with varying levels of proficiency with computer science to produce something useful by the end of the exercise. By the end of the 35-hour period, students not only had a small project they could put in professional portfolios, but had become proficient in a particular aspect of Unity, thought about the technology as a means to serve others, and in implementing their projects, had a better sense of what they would need to do going forward. It also convinced them that they were capable of using the technology in a way the Unity tutorials did not engender. Linker and Young guided the participants in thinking through how symposium participants might adapt this exercise for their own project teams.

Conclusion

Through the symposium and the workshops described above, participants engaged in conversations around designing socially-conscious pedagogy for 3D/VR/AR. Building a framework for teaching and learning with 3D/VR/AR technologies founded in decolonial theory and practices resonated with our participants. This enabled the group to evaluate how projects and assignments fit into an ethical model for cultural heritage pedagogies. The symposium closed with a productive discussion about what the participants learned, with a focus on planning for future steps.[2] Several participants suggested the importance of backward design, which would specifically place the learning outcomes as the first step in creating 3D/VR/AR and related assignments.

Conversations among group members brought up multiple questions, such as: how do we anticipate student use? How do we adjust our use of 3D/VR/AR in response to unexpected circumstances? How do we introduce emerging technologies in the classroom while accommodating individuals unable to take advantage of the intended purposes of ready-made hardware and software? How can these technologies enhance hybrid and online learning? Are students (or faculty) distracted by the freedom of immersive environments? Can we create bilingual metadata in a VR environment? If one could, where would you display subtitles or transcriptions in a virtual or augmented environment?

These conversations confirmed that digital humanists would benefit from future cross-institution discussions of 3D/VR/AR, as well as from shared access to teaching materials, which are often siloed within institutions and departments. Students engage differently with course concepts and each other, depending on the application of the technology within that course. Student learning is dependent on the skills and interests of individual instructors; collaboration is necessary for producing robust materials and responsible projects. Perhaps the most challenging task is creating accessible and sustainable materials applicable to multiple modes of disciplinary learning outcomes at a time of rapid technological and institutional transformation.

In an effort to increase the reach of the conversations that arose out of Immersive Pedagogy, the symposium organizers are working to produce an open-access, peer-reviewed publication containing lesson plans and educational material to facilitate disciplinary and interdisciplinary work that engages 3D/VR/AR technologies. This project aims to extend the work of the Digital Library Federation (DLF) Pedagogy Working Group’s Teach Toolkit that provides lesson plans for digital library instruction.[3] To guide educators to adapt immersive technologies to the needs of diverse disciplines, the Immersive Pedagogy teaching materials will introduce a range of 3D hardware and software, including asset or game repositories. The teaching materials will include diverse lesson plans with tailored learning outcomes, introducing a representative sample of available immersive technologies and resources while addressing humanistic pedagogical goals. Because this project was born out of the CLIR postdoctoral fellowship program, it aims to contribute to the growing field of scholarship on the crucial role that academic libraries or research and teaching centers can play in the integration of immersive technologies across the curriculum.

The Immersive Pedagogy symposium’s prioritization of decolonialism, feminism, and accessibility speak to a radical and critical perspective that can apply to a range of 3D/VR/AR applications and instruction methods. Indeed, in starting conversations on how to promote making immersive experiences accessible and inclusive, there is an opportunity to move beyond operational concerns to lasting pedagogical practices. For decades, contingencies have transformed education and cultural heritage, requiring us to rethink the potential of emerging communication technologies through a critical lens. More evident in the midst of the COVID-19 global pandemic, which has spurred the need for digital ways of teaching and learning, is the critical pedagogical use of virtual surrogates. These include 360° museum spaces and objects, 3D virtual meeting spaces, photogrammetry models, and interactive exhibits. By addressing upfront, rather than through remediation, the issues of social justice, accessibility, and decolonial pedagogies in immersive technology, educators can leverage these tools to respond to a transformative period in the education system.

Notes

[1] For a discussion of problems and considerations specifically related to the construction of historical 3D spaces, see Sullivan, Nieves, and Snyder 2017.

[2] For more detail, see the Immersive Pedagogy collaborative notes: “Shared Notes Wrap Up Session.” 2019. https://drive.google.com/drive/folders/1TSv8jrQlOlbPwi-TyvyOfV1_ZvA9I4y8.

[3] See the #DLFteach Toolkit 1.0: Lesson Plans for Digital Library Instruction.

Bibliography

Alexander, Jacqui M. and Chandra Talpede Mohanty, eds. 1997. “Introduction.” In Feminist Geneologies, Colonial Legacies, Democratic Futures: xxvii-xxiv. New York and London: Routledge.

Brush, Stephen G. 1978. “Nettie M. Stevens and the Discovery of Sex Determination by Chromosomes.” Isis 69, no. 2: 163–172. https://www.jstor.org/stable/230427.

Cotera, María. 2018. “Nuestra Autohistoria: Toward a Chicana Digital Praxis.” American Quarterly 70, no. 3: 483–504. https://muse.jhu.edu/article/704334.

Clark, Jasmine. 2018. “Progressing Towards an Accessible VR Experience.” Loretta C. Duckworth Scholars Studio. https://sites.temple.edu/tudsc/2018/11/07/progressing-towards-an-accessible-vr-experience/.

Digital Zombies. n.d. http://digitalzombies.ucr.edu.

“Guidelines for Primary Source Literacy.” 2018. Society of American Archivists. https://www2.archivists.org/standards/guidelines-for-primary-source-literacy.

Grayburn, Jennifer, Zach Lischer-Katz, Kristina Golubiewski-Davis, and Veronica Ikeshoji-Orlati, eds. 2019. 3D/VR in the Academic Library: Emerging Practices and Trends. Arlington, VA: Council on Library and Information Resources.

KiltHub Repository. n.d. Carnegie Mellon University. https://www.library.cmu.edu/kilthub/about.

Levy, Juliette. 2017. “‘Digital Zombies’– A Learner-Centered Game: Social Knowledge Creation at the Intersection of Digital Humanites and Digital Pedagogy.” In Social Knowledge Creation in the Humanities: An Open Anthology, edited by Alyssa Arbuckle, Aaron Mauro, and Daniel Powell. ITER Press. https://ntmrs-skc.itercommunity.org/.

———. 2019. “How Not to be a Replicant: Working Towards a Useful VR.” Keynote address, Carnegie Mellon University Symposium, Pittsburgh, PA, June 28.

Light, Ben, Jean Burgess, and Stefanie Duguay. 2018. “The walkthrough method: An approach to the study of apps.” New Media & Society 20, no. 3: 881–900.

Linker, Jessica, Elia Anagnostou, Courtney Dalton, Jocelyn Dunkley, Stella Fritzell, Tanjuma Haque, Arianna Li, Rachel Starry, and Linda Zhu. 2019. Bryn Mawr Women in Science. https://digitalscholarship.brynmawr.edu/howis/.

Losh, Elizabeth and Jacqueline Wernimont, eds. 2019. Bodies of Information: Intersectional Feminism and Digital Humanities. Minneapolis, MN: University of Minnesota Press.

Maldonado-Torres, Nelson. 2007. “On the Coloniality of Being.” Cultural Studies 21, no. 2: 240–270.

McPherson, Tara. 2012. “Why Are the Digital Humanities So White? or Thinking the Histories of Race and Computation.” In Debates in Digital Humanities, edited by Matthew K. Gold. Minneapolis: University of Minnesota Press.

Nieves, Angel David. 2019. “Developing a Social Justice Framework for Immersive Technologies in Digital Humanities.” Keynote address, Carnegie Mellon University Symposium, Pittsburgh, PA, June 27.

“Program.” 2019. Immersive Pedagogy: A Symposium on Teaching and Learning with 3D, Augmented and Virtual Reality. https://events.library.cmu.edu/immersive-pedagogy/program/.

“Public Relations.” 2019. Unity. https://unity3d.com/public-relations.

Risam, Roopika. 2019. New Digital Worlds: Postcolonial Digital Humanities in Theory, Praxis, and Pedagogy. Evanston, IL: Northwestern University Press.

Robnett, Belinda. 1997. How Long? How Long?: African American Women in the Struggle for Civil Rights. Oxford: Oxford University Press.

Rossiter, Margaret. 1993. “The Matthew Matilda Effect in Science.” Social Studies of Science, 23, no. 2: 325–341. http://www.jstor.org/stable/285482.

Sayers, Jentery, ed. 2017. Making Things and Drawing Boundaries: Experiments in the Digital Humanities. Minneapolis: University of Minnesota Press.

Smith, Pamela. n.d. The Making and Knowing Project: Intersections of Craft Making and Scientific Knowing. https://www.makingandknowing.org/. Accessed Dec. 13, 2019.

Sullivan, Elaine, Angel David Nieves, and Lisa M. Snyder. 2017. “Making the Model: Scholarship and Rhetoric in 3D Historical Reconstructions.” In Making Things and Drawing Boundaries: Experiments in the Digital Humanities, edited by Jentery Sayers, 301–316. Minneapolis: University of Minnesota Press.

Unity Assets Store. 2020. https://assetstore.unity.com/.

Acknowledgments

We would like to acknowledge The Andrew W. Mellon Foundation and the Council on Library and Information Resources (CLIR) for the microgrant that funded the Immersive Pedagogy symposium, as well as Carnegie Mellon University Libraries for hosting the event. Thank you to the entire Immersive Pedagogy team, including Eric Kaltman, Neil Weijer, and Chris Young for making the symposium possible. Last, but certainly not least, thank you to all the Immersive Pedagogy participants and keynote speakers, who created a positive, productive community of practice: Andy Anderson, DB Bauer, Katie Chapman, Elena Foulis, Kat Hayes, Juliette Levy, Juan Llamas-Rodriguez, Meaghan Moody, Angel Nieves, Samantha Porter, Coral Salomón, Julia Troche, Jordan Tynes, and Christa Williford.

About the Authors

Lorena Gauthereau is the Digital Programs Manager for the US Latino Digital Humanities program at the University of Houston’s Recovering the US Hispanic Literary Heritage. She received her Ph.D. in English and her M.A. in Hispanic Studies, both from Rice University. Her research interests include US Latinx studies, digital humanities, and decolonial theory. Orcid ID: orcid.org/0000-0002-7185-8982.

Jessica Linker is an Assistant Professor of History at Northeastern University. She was previously a Postdoctoral Fellow and Program Coordinator at the Consortium for History of Science, Technology and Medicine, a Visiting Assistant Professor at Bryn Mawr College, and the Director of Bryn Mawr Women in Science. She researches women’s scientific practices in early America.

Emma Slayton is the Data Curation, Visualization, and GIS specialist at Carnegie Mellon University Libraries. She obtained an MPhil from the University of Oxford in 2013 and completed her Ph.D. at the Faculty of Archaeology, Leiden University in 2018. Her current work centers around improving and supporting digital literacy efforts. Orcid ID: https://orcid.org/0000-0003-2230-3101.

Alex Wermer-Colan is a postdoctoral fellow in Temple University Libraries’ Loretta C. Duckworth Scholars Studio, where he coordinates research and pedagogical projects in cultural analytics and digital media arts. His editorial and scholarly criticism have appeared in PAJ: A Journal of Performance and Art, Twentieth Century Literature, The Yearbook of Comparative Literature, Lost & Found, Indiana University Press, and The Los Angeles Review of Books. Orcid ID: https://orcid.org/0000-0001-7030-6070.

Screen capture from computer-generated virtual reality software showing the user's virtual hand reaching for controls in a simulated space. In the middle of the screen are multi-colored, three-dimensional models of spiraling biochemical proteins and floating controls with various labels "uploader, ON, Sun Position, Model, Position, Rotation, Skybox."
0

Barriers to Supporting Accessible VR in Academic Libraries

Abstract

Virtual reality (VR) shows great promise for enhancing the learning experience of students in higher education and academic libraries are at the forefront of efforts to bring VR into the curriculum as an innovative learning tool. This paper reviews some of the growing applications and benefits of VR technologies for supporting pedagogy in academic libraries and outlines the challenges of making VR accessible for disabled students. It defines existing regulations and guidelines for designing accessible digital technologies and offers two case studies drawn from each of the authors’ own academic libraries, at Temple University and at the University of Oklahoma, in order to provide insight into the challenges and benefits of making VR more accessible for students. The paper argues that to continue to serve their mission of equitable access to information for the entire student population, academic libraries that implement VR programs need to balance innovation with inclusion by allocating sufficient staff time and technical resources and bringing accessibility thinking into VR projects from the beginning. To accomplish this, libraries will need the assistance of software developers and accessibility experts, and librarians will need to act as strong advocates for better support from commercial software and hardware vendors and to promote change in their institutions.

Introduction

Virtual reality (VR) and other extended reality (XR) technologies show great promise for supporting pedagogy in higher education. VR gives students the chance to immerse themselves in virtual worlds and engage with rich three-dimensional (3D) models of learning content, ranging from biochemical models of complex protein structures to cultural heritage sites and artifacts. Research shows that VR can increase student engagement, support the development of spatial cognitive skills, and enhance the outcomes of design-based activities in fields such as architecture and engineering. With these benefits, however, come the risks that VR will exacerbate inequality and exclusions for disabled students.[1] Disability is typically defined as a combination of physical (e.g., not having use of one’s legs) and participation (e.g., not having a ramp so that a wheelchair user can access services) barriers. According to the Center for Disease Control, 26% of adults in the United States have a disability. These include cognitive, mobility, hearing, visual, and other types of disability.

As a class of technologies that engage multiple senses, VR has the capacity to engage users’ bodies and senses in a holistic, immersive experience. This suggests that VR holds great potential for supporting users with a diverse range of sensory, motor, or cognitive capabilities; however, there is no guarantee that the affordances of VR will be deployed in accessible ways. In fact, the cultural tendency to ignore disability coupled with the rapid pace of technological innovation have led to VR programs that exclude a variety of users. Within higher education, the exclusion of disabled students from the benefits of these new technologies being deployed risks leaving behind a significant portion of the student population. The U.S. Department of Education, National Center for Education Statistics (2019) has found that 19.4% of undergraduates and 11.9% of graduate students have some form of disability. Libraries have long been leaders in supporting accessibility (Jaeger 2018) and the rise of immersive technologies presents an opportunity for them to continue to be leaders in making information available to all users. Academic libraries, the focus of this paper, are particularly well positioned to address the challenges of VR accessibility given their leadership in innovative information services and existing close relationships with the research and pedagogy communities at their institutions.

In what follows, we present a brief outline of the recent emergence of VR technologies in academic libraries, introduce recent research on VR accessibility, and conclude with a discussion of two brief case studies drawn from the authors’ institutions that illustrate the benefits and barriers associated with implementing accessibility programs for VR in academic libraries.

VR in Higher Education

“Virtual reality” or “VR” refers to a class of technologies that enable interactive and immersive experiences of computer-generated worlds, produced through a mixture of visual, auditory, haptic, and/or olfactory stimuli that engage with the human sensory system and provide the user with an experience of being present in a virtual world. In most VR systems, visual and auditory senses are primarily engaged, with increasing research being done on integrating haptics and other stimuli. Different levels of immersion and interaction are possible depending on the specific configuration of devices, from relatively low immersion and low interaction provided by inexpensive 3D cardboard viewers for use with mobile devices (e.g., Google Cardboard) to expensive head-mounted displays (HMDs) such as the HTC Vive and Oculus Rift systems that use headsets and head and body tracking sensors to capture users’ movements along “six degrees of freedom” (three dimensions of translational movement along x, y, and z axes, plus three dimensions of rotational movement, roll, pitch, and yaw). At present, HMDs are more commonly used than CAVEs, or “Cave Automatic Virtual Environment,” room-sized VR environments that use 3D video projectors, head and body tracking, and 3D glasses to provide multi-user VR experiences (Cruz-Neira et al. 1992), which have been used in academic contexts since the 1990s. This interest in new information technologies that provide library users with access to computer-generated worlds is not new for librarians. The current interest in VR follows experimentation conducted in libraries beginning earlier in the 2000s on “virtual worlds,” 3D computer-generated social spaces, such as Second Life, that users interacted with through a typical configuration of 2D computer monitor, mouse, and keyboard. Libraries envisioned these technologies as potential tools for expanding library services and enhancing support for student learning and research evaluated the pedagogical efficacy of these new tools (e.g., Bronack et al. 2008; Carr, Oliver, and Burn 2010; Deutschmann, Panichi, and Molka-Danielsen 2009; Holmberg and Huvila 2008; Praslova, Sourin, and Sourina 2006).

Since the commercial release of affordable VR systems such as the HTC Vive and Oculus Rift in 2016 (and now cheaper, lower-resolution variants such as Oculus Go and Oculus Quest), academic libraries have started seriously exploring the possibility of VR to support research and pedagogy. They have begun to conceptualize VR as a platform for immersive user engagement with high-resolution 3D models that support existing curricular activities, such as the use of archaeological, architectural, or scientific models in classroom exercises. Cook and Lischer-Katz (2019) argue

the realistic nature of immersive virtual reality learning environments supports scholarship in new ways that are impossible with traditional two-dimensional displays (e.g., textbook illustrations, computer screens, etc.). … Virtual reality succeeds (or fails), then, insofar as it places the user in a learning environment within which the object of study can be analyzed as if that object were physically present and fully interactive in the user’s near visual field. (70)

VR has been used to support student learning in a variety of fields, such as anthropology and biochemistry (Lischer-Katz, Cook, and Boulden 2018), architecture (Milovanovic 2017; Pober and Cook, 2016; Schneider et al. 2013), and anatomy (Jang et al. 2017). Patterson et al. (2019) describe how the librarians at the University of Utah have been incorporating VR technologies into a wide variety of classes, supporting architecture students, geography students, dental students, fine arts students, and nursing students. From this perspective, VR is envisioned as a tool for accessing digital proxies of physical artifacts or locations that students would ordinarily engage with as physical models (for instance, casts of hominid skull specimens), artifacts, or locations, but which are often too expensive or difficult to access directly.

In addition to providing enhanced modes of access to learning materials, using VR can also enhance student engagement and self-efficacy if implemented in close consultation with faculty (Lischer-Katz, Cook, and Boulden 2018). The technical affordances of VR, when deployed with care, are able to support a range of pedagogical objectives. Dalgarno and Lee (2010) identified representational fidelity (i.e., realistic display of objects, realistic motion, etc.) and learner interaction (i.e., student interaction with educational content) as key affordances of VR technologies, which they suggest can support a range of learning benefits they identified, including spatial knowledge representation, experiential learning, engagement, contextual learning, and collaborative learning. Chavez and Bayona (2018) surveyed the research on literature on VR and identified interaction and immersion as the two aspects of VR that should be considered when designing VR learning applications. Similarly, Johnson-Glenberg (2018) identified a set of design principles for using VR in education based on related affordances of VR—“the sense of presence and the embodied affordances of gesture and manipulation in the third dimension” (1) and found that “active and embodied learning in mediated educational environments results in significantly higher learning gains” (9). Research also suggests that the special visual aspects of VR, such as depth perception and motion cues (Ware and Mitchell 2005), head tracking (Ragan et al. 2013), and immersive displays (Ni, Bowman and Chen 2006) are able to enhance the analytic capabilities of human perception. VR has been shown to enhance human abilities of visual pattern-recognition and decision-making, particularly when working with big data (Donalek et al. 2014), prototyping (Abhishek, Vance, and Oliver 2011), or understanding complex spatial relationships and structures in data sets (Prabhat et al. 2008; Kersten-Oertel, Chen and Collins 2014; Laha, Bowman and Socha 2014).

Immersion is often identified by researchers as a key characteristic of VR technologies that is applicable to enhancing the learning experiences of students. Fowler (2015) identified three types of VR immersion relevant to pedagogy: Conceptual immersion, which supports development of abstract knowledge through students’ self-directed exploration of learning materials, for instance, molecular models; task immersion, in which students begin to engage with and manipulate learning materials; and social immersion, in which students engage in dialogue with others to test and expand upon their understanding. One critique of the applications of VR-based pedagogy is that instructional designers and instructors rarely indicate their underlying learning models or theories (Johnston et al. 2018). For instance, Lund and Wang (2019) found that VR can improve student engagement in library instruction, but do not specify which pedagogical models are effective, instead comparing a particular classroom activity with traditional classroom methods versus the same activity using VR, measuring impact on academic performance and motivation. Radianti et al. (2020), in their review of 38 recent empirical studies on VR pedagogy, acknowledge that while immersion is a critical component of the pedagogical affordances of VR, different studies define the term differently. They also found that only 32% of the studies reviewed indicated which learning theories or models underpin research studies, which makes it difficult to generalize approaches and apply them to other contexts. Radianti et al. (2020) point out that “in some domains such as engineering and computer science, certain VR applications have been used on a regular basis to teach certain skills, especially those that require declarative knowledge and procedural–practical knowledge. However, in most domains, VR is still experimental and its usage is not systematic or based on best practices” (26).

What these trends suggest is that VR shows great potential for use in supporting classroom instruction in higher education institutions, even though pedagogical models and methods of evaluation are still being developed and most projects are in the experimental phase of development. Some fields have already been adopting VR into their departments, such as computer science, engineering, and health science programs, but academic libraries are leading the way in promoting VR for their wider campus communities (Cook and Lischer-Katz 2019). Since many libraries are emerging as leaders in supporting VR, it is essential for them to have policies and support services in place to ensure that these new technologies are usable by all potential users at their institution.

As librarians consider adopting these innovative technologies, discourses of innovation can sometimes lead to oversights that may exclude some users. VR technologies enter libraries alongside other emerging technologies and innovative library services. The current discourse of transformational change promoted by the corporate information technology sector are often at odds with critical approaches to librarianship that stress inclusion and social justice (Nicholson 2015). These conceptions of radical innovation and disruption construct institutions, their policies, and regulations as structures that only function to slow down and constrain innovation. The assumption is that innovative technology is inherently neutral in terms of its ethics and politics, and that it does not require institutional processes to constrain or limit its negative effects; however, by decoupling technological change from institutionalized processes that protect the rights of historically marginalized groups of library patrons, technological change inevitably reinscribes exclusion into the infrastructures of learning. As Mirza and Seale (2017) argue

technocratic visions of the future of libraries aspire to a world outside of politics and ideology, to the unmarked space of white masculinity, but such visions are embedded in multiple layers and axes of privilege. They elide the fact that technology is not benevolently impartial but is subject to the same inequities inherent to the social world. (187)

The idea that technologies embed biases and cultural assumptions is not a new idea—scholars in the field of Science and Technology Studies have argued for decades that technologies are never neutral (e.g., Winner 1986)—but librarians, library administrators, and library science researchers often forget to examine their own “tunnel vision and blind spots” (Wiegand, 1999), or more precisely, their unreflected implicit biases that shape decision making about which technologies to adapt and how to deploy them in libraries. On the other hand, this also means that it is possible to balance innovation with inclusivity by foregrounding library values at the start of the process of innovation, rather than by retrofitting designs, which can yield results that are less equitable and more costly (Wentz, Jaeger and Lazar 2011). Clearly, the learning affordances of VR (Dalgarno and Lee 2010), as they are currently designed, need to be reimagined for disabled users.

VR and Accessibility

Aside from these ethical considerations, as VR becomes increasingly common in education, business, and other disciplines, it becomes answerable to legal guidelines. Federal guidelines for more established information and communication technology can be found in Section 508 of the Rehab Act (see U.S. General Services Administration n.d.), which utilizes Web Content Accessibility Guidelines (WCAG) 2.0 as a standard for web technology (W3C Web Accessibility Initiative 2019). WCAG provide guidance on how to make web content accessible to disabled people and they are overseen by the Web Accessibility Initiative (WAI), part of the World Wide Web Consortium (W3C) (see W3C Web Accessibility Initiative 2019). While they provide a valuable framework, WCAG do not directly apply to immersive technologies and there are currently no accessibility guidelines that do so. Work has been done to develop individual accessibility extensions, hardware, and features, but measurable guidelines that would aid in accessible design are still needed. Only in the last few years have accessibility specialists started adapting existing guidelines by examining existing initiatives and mapping them to the success criteria in WCAG. This includes the XR Access Symposium that was held in the summer of 2019 (see Azenkot, Goldberg, Taft, and Soloway 2019), as well as W3C’s Inclusive Design for Immersive Web Standards Workshop held in the fall of 2019 (see W3C 2019). There are also more specific guidelines that can contribute to design considerations, such as the Game Accessibility Guidelines that are more focused on game design (see Ellis et al. n.d.). Increasing the urgency of this matter, as of December 31, 2018, any video game communication functionality released in 2019 or later must be accessible to disabled people under the 21st Century Communications and Video Accessibility Act (Enamorado 2019), which expands the group of industries mandated to meet accessibility guidelines to include the video game industry.

Those interested in learning more about the accessible design of VR and other immersive technologies should consider reading “Accessible by Design: An Opportunity for Virtual Reality” (Mott et al. 2019), which provides general guidelines for designing accessible VR. For an example of designing accessible tools for a specific user group, see Zhao et al. (2019), which details the developments of a VR toolkit for supporting low-vision users.

Before going any further, it is important to distinguish between VR in its current, popularized form vs. the affordances of VR as a medium. The initiatives, guidelines, and research projects referred to in this section are still largely focused on analyzing the design of the former. However, in order for the technology to become truly accessible, critical inquiry must continue to progress in its understanding of the broader capabilities, limitations, and levels of interaction that construct the latter. The design practices and recommendations that have been developed to support the accessibility of VR are largely individualized and prototypical, which means that each institution’s particular experiences tackling the challenges of accessible VR will vary based on a number of factors. These factors include their individual histories supporting VR, staffing levels and development support, resources, and institutional commitments to accessibility. As librarians at Temple University and University of Oklahoma, we are now in the process of developing guidelines and tools to meet these challenges.

VR at Temple University’s Loretta C. Duckworth Scholars Studio

Temple University’s Loretta C. Duckworth Scholars Studio (LCDSS) “serves as a space for student and faculty consultations, workshops, and collaborative research in digital humanities, digital arts, cultural analytics, and critical making” (Temple University Libraries n.d.). Before the main library’s relocation to its new building, the LCDSS, formerly known as the Digital Scholarship Center (DSC), was located in the basement of Paley Library. Upon its 2015 opening, the DSC had two Oculus Rift DK2 headsets available for interested users. Its space in the new Charles Library includes an Immersive Visualization Studio designed for up to 10 people to simultaneously participate in immersive experiences, and as of 2019 has twelve headsets from a variety of manufacturers, in addition to mobile based headsets with an eye towards continuous acquisition of newer technologies. There are six full-time staff members, one of whom is responsible for the upkeep and management of the Immersive Studio among their other duties.

In August of 2017, I (Jasmine Clark) began researching the accessibility of VR as part of a project I was developing during my library residency.[2] Upon reviewing existing literature, it was apparent that research on the usability of VR for disabled users was in its early stages. Most notable was a report, “VR Accessibility: Survey for People with Disabilities,” resulting from a survey of disabled VR users produced in partnership by ILMxLab and the Disability Visibility Project (see Wong, Gillis, and Peck 2018). However, the majority of research and resources exploring the applications of VR to disabled people were composed of one-off solutions and extensions. This included cases of VR being used as an assistive technology (e.g., spatial training for blind individuals), unique hardware solutions (e.g., the haptic cane), and known issues for specific types of users (e.g., assumed standing position in games being disorienting for wheelchair users). These developments, while valuable, were not design standards or solutions broadly adopted by the game industry. Another concern was the fact that, in the context of the DSC, VR was not just a technology, but also a service that included training and assistance in its use for library patrons. This added an additional layer of complexity because, while there have been discussions on disability in the context of making and makerspaces, there was no literature on accessible service policies, best practices, and documentation for digital scholarship as a whole. In response to these challenges, I began examining existing guidelines and assessing their applicability to emerging technologies. Because WCAG is the federal standard, I joined a working group that guided me through reading the supporting documents and success criteria of WCAG, as well as examining the major legislative changes that were happening around accessibility at that time. I also began working with Jordan Hample, the DSC’s (now LCDSS’s) main technical support staff member, to understand whether or not these guidelines were applicable to immersive technologies.

Because we also needed to address service practices and policies, I decided that user testing would be necessary. User testing would consist of three phases that would take place during a single visit: a pre-interview (to ensure safety and gain an understanding of a user’s disability and previous technical experience), a use test (where users would use VR headsets), and a post-interview (to solicit feedback). I coordinated with Temple’s Disability Resources and Services (DRS) and DSC staff to bring in disabled stakeholders (students, alumni, and other members of the Temple community) in an attempt to 1) determine whether or not they would be able to utilize the equipment, and 2) determine if there were barriers to providing them with the same level of service as other patrons. As Wong, Gillis, and Peck (2018) point out in their report, “people with disabilities are not a monolith—accessibility and inclusion is different for everyone” (1). In order to scope the research to a manageable scale, I decided we would begin with visually impaired, deaf/Hard-of-Hearing (HOH), and hearing impaired users (hearing impairment would include individuals with tinnitus, or other auditory conditions not included under the umbrella of deaf/HOH). Working with Jordan, as well as Alex Wermer-Colan, a Council on Library and Information Resources (CLIR) postdoctoral fellow, I proceeded to draft a research protocol that consisted of interview questions and an explanation for participants of what VR is and the purpose of the research being conducted. These were all sent out via DRS listservs to solicit participants. VR services in the DSC involved a lot of hands-on onboarding and orientation from staff. Often, patrons would drop in and simply want to get acquainted with the technology. As a result, the goal of the research project was for disabled participants in our user testing to be able to navigate to our space and successfully work with the staff members responsible for providing VR assistance to identify experiences that would be as usable as possible for them. There was also a need to better understand staff preparedness in providing assistance to disabled patrons. In the months leading up to the testing, I had preliminary discussions with staff, and also inquired into staff training on accessibility and disability more generally at the library and university level. I found that training was not formalized, so I gathered and shared resources with my colleagues to ensure the safety and dignity of participants. This included referring to the Gallaudet University’s guide on working with American Sign Language (ASL) interpreters (see Laurent Clerc National Deaf Education Center 2015) and various video tutorials on acting as a sighted guide for blind/low-vision people, and maintaining active discussions and explanations around ableism and disability. The discussions also allowed for better understanding of gaps in training and norms.

Once staff were sufficiently prepared, user testing commenced in the summer of 2018. Four participants were invited to the center, three of whom had various visual impairments and one of whom was deaf. On the days of their visits, I would go to the library entrance to greet and guide anyone who needed assistance. Upon arrival, they were brought into a meeting room for a pre-interview that would reintroduce the purpose of user testing, gauge any previous experience with the technology, and identify safety concerns by asking if they had other sensitivities that they felt would be a problem in VR (e.g., sensory sensitivities, sensitivity to flashing lights, etc.). We also asked about level of hearing/vision to get a better idea of which types of experiences worked for different types of hearing/vision. Some immediate questions brought up by participants were around accuracy of sound, depth perception, and similarity to real-world visual experience. Once the initial interview was completed, they were guided out to work with Jordan to identify potential experiences, similar to the way he typically worked with students. I took notes on the interactions, and Alex assisted as needed. Alex’s presence became particularly important when it came to the deaf user. It was brought to our attention that 1) due to variations in inner ear formation, those who were deaf/HOH were at higher risk for vertigo and, 2) a user reliant upon an ASL interpreter would not be able to see the interpreter while in the headset, complicating human assistance. In response, Alex took on the role of surrogate for this participant while they watched his activity on a monitor and gave instructions and feedback. Jordan took on the role of listening to the participants’ verbal feedback on each experience and, utilizing his knowledge of the DSC’s licenses for different VR programs, selected experiences that would be more accommodating to their specific hearing/visual needs.

Upon completion of this phase, participants were then brought back into the meeting room for a post interview. Responses to both interviews, as well as observations made during the interactions, were compiled and summarized into an internal report for our team. We had initially planned to have more users come in, but found that feedback on the limitations of the technology was consistent and addressable enough for us to make adjustments that would allow us to improve services and collect more nuanced data moving forward. For example, it was clear that the software varied so drastically that, in order to provide safe and effective services, it would be necessary to index the features and capabilities of various VR experiences.

The timing of this work was crucial, as we were a year away from the move to our new space, and the findings from the study helped us plan for it. The LCDSS is significantly larger than the DSC, and much more visible. However, while it has required that we re-envision our service policies and programming, it has also given us the opportunity to integrate accessibility into our work from the beginning. One way we are doing this is by developing an auditing workflow that would allow any staff member or student worker to examine newly-licensed VR experiences and produce an accessibility report, as there is a glaring lack of Voluntary Product Accessibility Templates (VPAT) for VR products. These reports would detail accessibility concerns and limitations at the beginning, allowing us to better serve disabled patrons. We are also working with the university’s central Information Technology Services to look at how this can be incorporated into broader LCDSS purchasing practices and documentation workflows.

Once this workflow is finalized, it will be used to support LCDSS staff in aiding faculty and researchers in the development of Equally Effective Alternative Access Plans (EEAAP) for their research and teaching. An EEAAP documents how a technology will be used in a class or program, its accessibility barriers, the plan to ensure equitable participation for disabled people, and the parties responsible for ensuring the plan is carried out. LCDSS staff frequently consult with faculty who wish to integrate LCDSS resources into their pedagogical practices. This can include feedback on assignment structure and design, recommended technologies, and other vital information required for pedagogical efficacy. By generating accessibility reports that identify technical limitations, LCDSS staff can aid faculty in developing multimodal approaches to integrating these technologies into their teaching. This means that, not only are we bringing accessibility to their attention early, but that we are also able to guide them and reduce intimidation, making buy-in more successful. Moving forward, Jordan Hample and I will be making all materials involved in this workflow publicly available, as well as continuing and expanding user testing to include other disabilities.

VR at the University of Oklahoma Libraries, Emerging Technologies Program

Accessibility initiatives for VR at the University of Oklahoma have followed a slightly different trajectory than the one outlined by Jasmine in the previous section. The VR program at OU Libraries was officially launched in 2016 in the Innovation @ the EDGE Makerspace, which began hosting classes and integrating VR content into the course curriculum, including initial integrations within biology, architecture, and fine arts courses (Cook and Lischer-Katz 2019). We use custom-built VR software that enables users “to manipulate their 3D content, modify environmental conditions (such as lighting), annotate 3D models, and take accurate measurements, side-by-side with other students or instructors” and support networked, multiuser VR sessions, which forms “a distributed virtual classroom in which faculty and students in different campus locations [are able to] teach and collaborate” (Cook and Lischer-Katz 2019, 73). Librarians provide VR learning opportunities in three main ways: 1) deployment in the library-managed makerspace; 2) facilitated course integrations; 3) special VR events. Each approach requires different levels of support and planning from librarians. In the case of deployment in our makerspaces, students are able to learn about the technology in a self-directed manner, with guidance from trained student workers who staff the space. Workshops and orientation sessions are available, and students, faculty, and community members typically drop in when they want and engage with technology in a self-directed manner. Since the focus of this space is on self-directed learning and experimentation, the training of student support staff is essential for ensuring that the space feels welcoming and inclusive to visitors and that staff are able to adjust the level of support they provide based on the needs of the visitors to the space.

In the case of course integrations, students are typically brought to our makerspace during regularly scheduled class time. We have portable VR kits that use high-powered gaming laptops and Oculus Rift headsets, which makes it possible to bring the learning experiences directly into the classroom if the faculty member prefers. Examples of VR-based classroom activities include interacting with 3D models that simulate learning objects, such as examining the morphology of different hominid skull casts in an anthropology class or analyzing complex protein structures and processes in a biochemistry class. VR is also used in other classes as a creative tool, such as in a sculpture course in which the students created sculptures in VR and then printed them using the 3D printers in the makerspace. In planning VR course integrations, librarians work directly with faculty members to design activities that will support their course learning objectives.

VR is also used frequently at OU Libraries for special events in which experts lead participants on guided tours through scholarly, high-resolution 3D models. Participants can join the VR tour on campus or from other institutions, since our custom-built VR software supports networked, multi-user sessions. Examples include inviting an archaeologist to lead a group through a 3D scan of a cave filled with ancient rock carvings that is located in the Southwestern United States (Schaffhauser 2017), as well as a tour led by a professor of Middle Eastern History through a 3D model of the Arches of Palmyra, located in Syria.

From the start of the emerging technologies initiative at OU Libraries, rapid innovation was a guiding principle, with the hope that the benefits of emerging technologies could be demonstrated to the broader campus community and that the library could become a hub for supporting emerging technologies across campus. It was important to quickly develop a base of VR technologies and librarian skills in order to promote the potential benefits of the technologies to faculty and students across campus. Starting in January 2016, students and faculty began using our VR spaces for research, learning, experimentation, and entertainment, and by 2018 we had faculty from over 15 different academic departments across campus using VR as a component in their classes (Cook and Lischer-Katz 2019), along with over 2000 individual uses of our VR workstations. By 2019, the emerging technology librarians (ETL) unit had grown to five full-time staff members who worked together to “rapidly prototype and deploy educational technology for the benefit of a range of University stakeholders” (Cook and Van der Veer Martens 2019, 614). At this time, concerns were raised by one of our ETLs about the accessibility of existing VR services and the ETL team brought in an accessibility specialist to advise them. One of the key challenges the team identified through the process of reviewing their existing VR capabilities was the fact that most commercially produced VR software lacks accessibility options, particularly in terms of compatibility with assistive devices. In reviewing users’ experiences in our makerspace, ETLs found that users with dexterity, coordination, or mobility disabilities often request passive VR experiences that provide immersive experiences without the need for use of the VR controller inputs. For programs such as the popular Google Earth VR program, it is not currently possible to provide users with passive experiences, rather the user needs to be able to actively control the two VR controllers themselves to engage with the VR experience. To the team’s surprise, some of the lower-resolution, untethered VR systems, such as the Oculus Go have shown more capabilities for providing passive experiences that rely only on head tracking and the use of target circles for movement through the VR space. Making narrated and guided tours for a VR experience available is essential for providing access to some groups of disabled users. Ensuring that VR controllers are accessible has also been a challenge and ETLs have begun experimenting with 3D printing add-on components to make the VR controllers more usable for users with limited hand function. In response to the lack of accessibility options for commercial software releases, modifications were made to OU’s custom-built VR software to provide accessibility capabilities, including: 1) controls for changing the sensitivity of VR interface controls; and 2) options for user interface text resizing. These modest modifications were made in consultation with VR users. Technical solutions alone are not sufficient, of course, and the ETL team has also found it very important to continue to improve training for student staff so that they are prepared to properly assist disabled users in a sensitive and respectful way. Communicating clearly to the wider university community about what accessible software and hardware capabilities are available is also a challenge that the team is tackling. These activities are still ad hoc in many ways, and we have found that additional work is needed to develop procedures for addressing VR accessibility in a more systematic way in the library and across campus.

The ETL team is taking several approaches to improving our support for accessible VR, looking outward to resources beyond the walls of OU Libraries and looking inward to resources at the university to support improvements to accessibility. ETLs are expanding their knowledge base through involvement in accessibility conferences and working groups and looking to our colleagues at other institutions, such as Temple University Libraries, for guidance on policies and procedures for evaluating and implementing VR software and hardware. The ETL team is planning on conducting future usability testing and focus groups with a range of disabled users from the OU community in order to further refine the feature set of our custom software, which we plan to package and distribute for other institutions to use and build upon.

The experiences of ETLs at OU Libraries point to the importance of working with accessibility experts and bringing disabled users into the design process to develop technologies and policies. Librarians should not be expected to take on accessible design by themselves, rather they should look to experts in this field for assistance. Working with our University’s disability coordinator has been essential for helping us to identify areas where we need to improve our accessibility capabilities, as well as providing us with a network of disabled users on campus who could provide us with user feedback on our technologies. The types of issues we are looking into include techniques for auditing VR software for accessibility issues, providing clearer signage and information on websites to provide students and faculty with a clear understanding of which emerging technology tools are accessible and what accommodations are possible, and ways in which we can continue to improve staff training so that the student workers who staff our makerspace can better support disabled users. The process of developing policies and establishing processes and documentation to support those policies does take time; however, this work has been essential for training staff and establishing best practices at our makerspace in order to address the challenges of VR accessibility. Additional work is necessary to codify this ongoing and still experimental work into institutional policy documents and continue to seek out adaptive tools to make VR accessible for a greater range of library patrons.

Conclusion

The current wave of immersive technologies was not initially designed for users with varying levels of visual, auditory, mobility, and neurological capabilities. Even for libraries and centers that do have development support there is no way to remediate the inaccessibility of every experience used and, even if there was, there would be no way to keep up with the regular updates of hardware and software. One-off, localized solutions cannot replace structural change. In order for VR to become an accessible medium, developers, hardware manufacturers, distribution platforms, and other stakeholders involved in its creation and distribution need to ensure accessibility within their respective roles. The current lack of support from these stakeholders makes it crucial that library staff and the educators that they support understand disability and accessibility, develop appropriate documentation, and advocate for software and hardware vendors to provide better accessibility support in their products. In the meantime, libraries supporting different tiers of VR use and investment will have to consider different approaches to accessibility.

The preceding examples drawn from our experiences at Temple University and the University of Oklahoma (OU) show the range of issues facing accessible VR, but also show the differences in approach for different service models and pedagogical objectives. Temple University includes VR in a very broad suite of technical offerings and its faculty are not currently at the phase of “buy-in” where regular VR development is a priority. As a result, Temple’s focus is on indexing experiences and integrating alternative access plans, with accessible development occurring on a smaller scale. In comparison, OU has much more of a demand for custom-developed software solutions. This demand is due to the fact that one of the main VR applications that OU promotes for course integrations is its own flexible, custom software, which supports a variety of disciplines, including courses in biochemistry, anthropology, architecture, and English. OU is beginning to investigate the accessibility challenges of working with commercial software and is looking to Temple for guidance on how to properly evaluate different software titles and provide adequate documentation. For libraries without developer support, we can expect that the focus will more likely follow Temple’s approach. For libraries with regular development efforts, supporting home-grown accessible design practices, such as those at OU, will be more of a central activity. Some libraries will be a mixture of the two, working to blend commercial and homegrown solutions. Regardless of a library’s approach, the major takeaways for other institutions to consider as they bring accessibility thinking into their VR programs include:

  • Plan for Accessibility from the Beginning: Libraries can save time and resources by thinking about accessibility issues at the start of a program or project.
  • Lack of Standards: As of 2020, there are no standards for accessible VR design, but there are related standards that could lay the groundwork for their development.
  • Developer Support is Essential: Libraries that intend to develop VR experiences need to have sufficient developer support with accessibility expertise.
  • Importance of Auditing and Reporting: Out-of-the-box VR experiences will pose different accessibility challenges from one person to the next and should be audited to better understand these barriers to access. If a library lacks a developer to modify software or create new software, at the very least, available software needs to be audited and have a corresponding accessibility report produced.
  • VR is Not the Pedagogy: VR should be another tool in an educator’s arsenal, not the sole focus of a class (unless VR is the course subject). As Fabris et al. (2019) suggest “Having VR for the sake of having VR won’t fly; the VR learning resources need to be built with learning outcomes in mind and the appropriate scaffolds in place to support the learning experience” (74).
  • Acknowledge the Limits of VR Accessibility: There are limits to making VR accessible. The reality is that there will be students who are unable to use VR for a variety of reasons. Therefore, there should always be an alternative access plan developed so that students have access to non-VR learning methods as well.

Considering these best practices will better enable libraries to approach the challenges of making VR accessible. Putting them into action will directly benefit disabled users, improve librarians’ abilities to make their innovative technology spaces more inclusive, and will help administrators to better plan and allocate resources for supporting the missions of their institutions. While these guidelines are focused on supporting academic libraries, they will likely benefit higher education applications outside of the library, too.

Additionally, while it is true that there is extensive work to be done, there are existing inclusive instructional approaches that can be integrated into VR based coursework by individuals. Multimodal course design and Universal Design for Learning (http://udloncampus.cast.org/page/udl_about) are frameworks that can be applied to VR coursework with approaches like collaborative assignments and activities. It is also worth reviewing a 2015 special issue of Journal of Interactive Technology and Pedagogy that considers the benefits of introducing perspectives from disability studies into the context of designing innovative pedagogies. One of the important takeaways from this collection is that embracing disability and the alternative perspectives that it can provide, presents the potential for new learning opportunities (Lucchesi 2015).

Regardless of whichever pedagogical approach educators adopt, it is imperative that, unless VR is the subject of the course, they remember it is not the pedagogy. Instead, faculty should keep a diverse array of tools in their pedagogical toolkit that will support an equally diverse set of learners. As librarians, faculty, and instructional designers become familiar with inclusive learning frameworks, they are better positioned for more targeted, meaningful advocacy within their institutions. Because, while it is true that there is a lot of work to be done, it is equally true that it can only be done together through active involvement in institutional committees and task forces and by ensuring that discussions about accessibility occur in strategic planning and budgeting meetings with administrators. Accessibility awareness needs to be raised throughout libraries and other academic institutions so that the accessibility challenges of emerging technologies are addressed at the design stage and built into pedagogical implementations from the beginning. This will help to ensure that pedagogies founded on emerging technologies will be “born accessible,” for the benefit of learners and educators throughout the academic world.

Notes

[1] The use of identity-first (“disabled person”) vs. person-first (“person with disabilities”) language is debated. Disability is a complex set of identities and the language used should take into account the preferences of disabled people and other contextual factors. Our choice to use identity-first language is a conscious one.

[2] A library residency is a term position during which residents may rotate through different functional areas of the library or focus on one subject area, and often contribute to projects and initiatives at their host library to gain professional (vs. paraprofessional) experience.

Bibliography

Abhishek, Seth, Judy M. Vance, and James H. Oliver. 2011. “Virtual Reality for Assembly Methods Prototyping: A Review.” Virtual Reality 15, no. 1: 5–20.

Azenkot, Shiri, Larry Goldberg, Jessie Taft, and Sam Soloway. 2019. XR Symposium Report. https://docs.google.com/document/d/131eLNGES3_2M5_roJacWlLhX-nHZqghNhwUgBF5lJaE/edit?usp=sharing.

Bronack, Stephen, Amy L. Cheney, Richard Reidl, and Johan Tashner. 2008. “Designing Virtual Worlds to Facilitate Meaningful Communication: Issues, Considerations, and Lessons Learned.” Technical Communication 55, no. 3: 261–69.

Carr, Diane, Martin Oliver, and Andrew Burn. 2010. “Learning, Teaching and Ambiguity in Virtual Worlds.” In Researching Learning in Virtual Worlds, edited by Anna Peachey, Julia Gillen, and Daniel Livingstone, 17–31. London: Springer.

Chavez, Bayron and Sussy Bayona. 2018. “Virtual Reality in the Learning Process.” In Trends and Advances in Information Systems and Technologies, edited by Álvaro Rocha, Hojjat Adeli, Luís Paulo Reis and Sandra Costanzo, 1345–56. Cham, Switzerland: Springer International Publishing, https://doi.org/10.1007/978-3-319-77712-2_129.

Cook, Matt and Betsy Van der Veer Martens. 2019. “Managing Exploratory Units in Academic Libraries.” Journal of Library Administration, 59, no. 6: 606–28. http://doi.org/10.1080/01930826.2019.1626647.

Cook, Matt and Zack Lischer-Katz. 2019. “Integrating 3D and VR into Research and Pedagogy in Higher Education.” In Beyond Reality: Augmented, Virtual, and Mixed Reality in the Library, edited by Kenneth Varnum, 69–85. Chicago: ALA Editions.

Cruz-Neira, Carolina, Daniel J. Sandin, Thomas A. DeFanti, Robert V. Kenyon, and John C. Hart. 1992. “The CAVE: Audio Visual Experience Automatic Virtual Environment.” Communications of the ACM 35, no. 6: 64–73.

Deutschmann, Mats, Luisa Panichi, and Judith Molka-Danielsen. 2009. “Designing Oral Participation in Second Life: A Comparative Study of Two Language Proficiency Courses.” ReCALL 21, no. 2 (May): 206–26.

Donalek, Ciro, George Djorgovski, A. Cioc, A. Wang, J. Zhang, E. Lawler, S. Yeh, et al. 2014. “Immersive and Collaborative Data Visualization Using Virtual Reality Platforms.” In Proceedings of 2014 IEEE International Conference on Big Data, Washington, DC, Oct. 27–30, 609–14.

Ellis, Barrie, Gareth Ford-Williams, Lynsey Graham, Dimitris Grammenos, Ian Hamilton, Headstrong Games, Ed Lee, Jake Manion, and Thomas Westin. n.d. “Game accessibility guidelines.” Game accessibility guidelines. Accessed Dec. 13, 2019. http://gameaccessibilityguidelines.com/.

Enamorado, Sofia. 2019. “The CVAA & Video Game Accessibility.” 3Play Media. https://www.3playmedia.com/2019/03/18/the-cvaa-video-game-accessibility/.

Fabris, Christian, Joseph Rathner, Angelina Fong, and Charles Sevigny. 2019. “Virtual Reality in Higher Education.” International Journal of Innovation in Science and Mathematics Education 27: 69–80.

Holmberg, Kim and Isto Huvila. 2008. “Learning Together Apart: Distance Education in a Virtual World.” First Monday 13, no. 10 (October). https://firstmonday.org/article/view/2178/2033.

Jaeger, Paul T. 2018. “Designing for Diversity and Designing for Disability: New Opportunities for Libraries to Expand Their Support and Advocacy for People with Disabilities.” The International Journal of Information, Diversity, & Inclusion 2, no. 1–2: 52–66.

Jang, Susan, Jonathan M. Vitale, Robert W. Jyung, and John B. Black. 2017. “Direct Manipulation is Better than Passive Viewing for Learning Anatomy in a Three-dimensional Virtual Reality Environment.” Computers & Education 106: 150–65.

Johnson-Glenberg, Mina C. 2018. “Immersive VR and Education: Embodied Design Principles that Include Gesture and Hand Controls.” Frontiers in Robotics and AI 5, art. 81 (July): 1–19, http://doi.org/10.3389/frobt.2018.00081.

Johnston, Elizabeth, Gerald Olivas, Patricia Steele, Cassandra Smith and Liston Bailey. 2018. “Exploring Pedagogical Foundations of Existing Virtual Reality Educational Applications: A Content Analysis Study.” Journal of Educational Technology Systems 46, no. 4: 414–39. https://doi.org/10.1177/0047239517745560.

Kersten-Oertel, Marta, Sean Jy-Shyang Chen, and D. Louis Collins. 2014. “An Evaluation of Depth Enhancing Perceptual Cues for Vascular Volume Visualization in Neurosurgery.” IEEE Transactions on Visualization and Computer Graphics 20, no. 3: 391–403.

Laha, Bireswar, Doug A. Bowman, and John J. Socha. 2014. “Effects of VR System Fidelity on Analyzing Isosurface Visualization of Volume Datasets.” IEEE Transactions on Visualization & Computer Graphics 4: 513–22.

Laurent Clerc National Deaf Education Center. 2015. “Working with Interpreters.” Gallaudet University. https://www3.gallaudet.edu/clerc-center/info-to-go/interpreting/working-with-interpreters.html.

Lischer-Katz, Zack, Matt Cook, and Kristal Boulden. 2018. “Evaluating the Impact of a Virtual Reality Workstation in an Academic Library: Methodology and Preliminary Findings.” In Proceedings of the Association for Information Science and Technology Annual Conference, Vancouver, Canada, Nov. 9–14, 300–8. https://hdl.handle.net/11244/317112.

Lucchesi, Andres. 2015. “Introduction to Special Issue: Disability Studies Approaches to Pedagogy, Research, and Design.” Journal of Interactive Technology & Pedagogy 8. https://jitp.commons.gc.cuny.edu/category/issues/issue-eight/.

Lund, Brady D. and Ting Wang. 2019. “Effect of Virtual Reality on Learning Motivation and Academic Performance: What Value May VR Have for Library Instruction?” Kansas Library Association College and University Libraries Section Proceedings 9, no. 1: 1–7. https://doi.org/10.4148/2160-942X.1073.

Milovanovic, J. 2017. “Virtual and Augmented Reality in Architectural Design and Education.” In Proceedings of the 17th International Conference, CAAD Futures, Istanbul, Turkey, July. https://www.researchgate.net/publication/319665970_Virtual_and_Augmented_Reality_in_Architectural_Design_and_Education_An_Immersive_Multimodal_Platform_to_Support_Architectural_Pedagogy.

Mirza, Rafia and Maura Seale. 2017. “Who Killed the World? White Masculinity and the Technocratic Library of the Future.” In Topographies of Whiteness: Mapping Whiteness in Library and Information Science, edited by Gina Schlesselman-Tarango, 171–97. Sacramento, CA: Library Juice Press. http://mauraseale.org/wp-content/uploads/2016/03/Mirza-Seale-Technocratic-Library.pdf

Mott, Martez, Ed Cutrell, Mar Gonzalez Franco, Christian Holz, Eyal Ofek, Richard Stoakley, and Meredith Ringel Morris. 2019. “Accessible by Design: An Opportunity for Virtual Reality.” ISMAR 2019 Workshop on Mixed Reality and Accessibility. https://www.microsoft.com/en-us/research/publication/accessible-by-design-an-opportunity-for-virtual-reality/.

Ni, Tao, Doug A. Bowman, and Jian Chen. 2006. “Increased Display Size and Resolution Improve Task Performance in Information-rich Virtual Environments.” In Proceedings of Graphics Interface, Quebec City, Canada, June 7–9 139–46.

Nicholson, Karen P. 2015. “The McDonaldization of Academic Libraries and the Values of Transformational Change.” College & Research Libraries 76, no. 3: 328–338. http://doi.org/10.5860/crl.76.3.328.

Patterson, Brandon, Tallie Casucci, Thomas Ferrill and Greg Hatch. 2019. “Play, Education, and Research: Exploring Virtual Reality through Libraries.” In Beyond Reality: Augmented, Virtual, and Mixed Reality in the Library, edited by Kenneth J. Varnum, 47–56. Chicago: ALA Editions.

Pober, Elizabeth E. and Matt Cook. 2016. “The Design and Development of an Immersive Learning System for Spatial Analysis and Visual Cognition.” In Proceedings of 2016 Conference of the Design Communication Association, Bozeman, MT, https://static1.squarespace.com/static/532b70b6e4b0dca092974dbe/t/5755e2df20c647f04c95598a/1465246433366/pobercook_text+%281%29.pdf.

Prabhat, Andrew Forsberg, Michael Katzourin, Kristi Wharton, and Mel Slater. 2008. “A Comparative Study of Desktop, Fishtank, and CAVE Systems for the Exploration of Volume Rendered Confocal Data Sets.” In IEEE Transactions on Visualization and Computer Graphics 14, no. 3 (May-June): 551–63.

Praslova–Førland, Ekaterina, Alexei Sourin, and Olga Sourina. 2006. “Cybercampuses: Design Issues and Future Directions.” Visual Computer 22, no. 12: 1015–28.

Radianti, Jaziar, Tim A. Majchrzak, Jennifer Fromm, and Isabell Wohlgenannt. 2020. “A Systematic Review of Immersive Virtual Reality Applications for Higher Education: Design Elements, Lessons Learned, and Research Agenda.” Computers & Education 147: 1–29. https://doi.org/10.1016/j.compedu.2019.103778.

Ragan, Eric D., Regis Kopper, Philip Schuchardt, and Doug A. Bowman. 2013. “Studying the Effects of Stereo, Head Tracking, and Field of Regard on a Small-scale Spatial Judgment Task.” IEEE Transactions on Visualization and Computer Graphics 19, no. 5: 886–96.

Schaffhauser, Dian. 2017. “Multi-campus VR Session Tours Remote Cave Art.” Campus Technology, (October 9). https://campustechnology.com/articles/2017/10/09/multi-campus-vr-session-tours-remote-cave-art.aspx

Schneider, Sven, Saskia Kuliga, Christoph Hölscher, Ruth Conroy-Dalton, André Kunert, Alexander Kulik, and Dirk Donath. 2013. “Educating Architecture Students to Design Buildings from the Inside Out.” In Proceedings of the 9th International Space Syntax Symposium, edited by Y.O. Kim, H.T. Park and K.W. Seo, Seoul, Korea. https://www.researchgate.net/profile/Saskia_Kuliga/publication/281839210_Educating_Architecture_Students_to_Design_Buildings_from_the_Inside_Out_Experiences_from_a_Research-based_Design_Studio/links/55faa73c08aeba1d9f36bb64/Educating-Architecture-Students-to-Design-Buildings-from-the-Inside-Out-Experiences-from-a-Research-based-Design-Studio.pdf.

Temple University Libraries. n.d. “Loretta C. Duckworth Scholars Studio.” Temple University Libraries. Accessed Dec. 13, 2019. https://library.temple.edu/lcdss.

U.S. Department of Education, National Center for Education Statistics. 2019. Digest of Education Statistics, 2017 (2018–070), https://nces.ed.gov/fastfacts/display.asp?id=60.

U.S. General Services Administration. n.d. “About Us.” Section508.gov, GSA Government-wide IT Accessibility Program. Accessed Dec. 13, 2019. https://www.section508.gov/about-us.

Ware, Colin and Peter Mitchell. 2005. “Reevaluating Stereo and Motion Cues for Visualizing Graphs in Three Dimensions.” In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 51–8. http://doi.org/10.1145/1080402.1080411.

W3C. 2019. “Inclusive Design for Immersive Web standards.” W3C. https://www.w3.org/2019/08/inclusive-xr-workshop/.

W3C Web Accessibility Initiative. 2019. “Making the Web Accessible.” Web Accessibility Initiative. https://www.w3.org/WAI/.

W3C Web Accessibility Initiative. 2019. “Web Content Accessibility Guidelines (WCAG) Overview.” Web Accessibility Initiative. https://www.w3.org/WAI/standards-guidelines/wcag/.

Wentz, Brian, Paul T. Jaeger, and Jonathan Lazar. 2011. “Retrofitting Accessibility: The Legal Inequality of After–the–fact Online Access for Persons with Disabilities in the United States.” First Monday 16, no. 11 (November). https://firstmonday.org/ojs/index.php/fm/article/view/3666/3077.

Wiegand, Wayne A. 1999. “Tunnel Vision and Blind Spots: What the Past Tells Us about the Present; Reflections on the Twentieth-Century History of American Librarianship.” The Library Quarterly 69, no. 1: 1–32.

Winner, Langdon. 1986. The Whale and the Reactor: A Search for Limits in an Age of Technology. Chicago: University of Chicago Press.

Wong, Alice, Hannah Gillis, and Ben Peck. 2018. “VR Accessibility: Survey for People with Disabilities.” Disability Visibility Project & ILMxLAB. https://drive.google.com/file/d/0B0VwTVwReMqLMFIzdzVVaVdaTFk/view.

Zhao, Yuhang, Edward Cutrell, Christian Holz, Meredith Ringel Morris, Eyal Ofek, and Andrew D. Wilson. 2019. “SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision.” In Proceedings of CHI 2019, Glasgow, Scotland, May 4–9. http://doi.org/10.1145/3290605.3300341.

About the Authors

Jasmine Clark is the Digital Scholarship Librarian at Temple University. Her primary areas of research are accessibility and metadata in emerging technology and emerging technology centers. Currently, she is co-leading The Virtual Blockson, a project to recreate the Charles L. Blockson Afro-American Collection in virtual reality, while also doing research on 3D metadata and the development of Section 508 compliant guidelines for virtual reality experiences. Jasmine has experience in a variety of functional areas and departments, including metadata, archives, digital scholarship, and communications and development. She is interested in the ways information organizations can integrate accessible, inclusive practices into their services, hiring, and management practices.

Zack Lischer-Katz is a postdoctoral research fellow at University of Oklahoma Libraries. From 2016 to 2018 he was a Council on Library and Information Resources (CLIR) Postdoctoral Fellow. He employs qualitative-interpretive methodologies to examine visual information preservation and curation in information institutions, with a focus on complex data types, such as virtual reality, 3D, and audiovisual formats. His research has appeared in Library Trends, International Journal of Digital Curation, Information Technology and Libraries, and First Monday. He received his PhD in Communication, Information, & Library Studies from Rutgers University and his MA in Cinema Studies from New York University.

Images are for demo purposes only and are properties of their respective owners. ROMA by ThunderThemes.net

Skip to toolbar