Daily Archives: November 11, 2021

Black headphones on black woodgrain surface
0

Poetry in Your Pocket: Streaming Playlists and the Pedagogy of Poetic Interpretation

Abstract

Teaching students to interpret poetry remains one of the most challenging aspects of humanities instruction due to students’ anxiety about interpretation and skepticism of poetry’s relevance to their lives. Accordingly, this article outlines a new model for using interactive streaming media playlists as a means of increasing student confidence and active engagement with poetry. It draws from my structured approach to a general-studies poetry seminar in which I required students to engage consistently with traditional print text alongside streaming recitations and musical adaptations of poetry. Thereafter, students created their own multimodal adaptations of poems to solidify their perception of poetry as an adaptable living tradition with social significance. Student responses to this strategy demonstrate a meaningful increase in their self-reported confidence in reading poetry. Moreover, the students expressed how the playlist made the study of poetry feel more relevant in a contemporary digital context while appealing to their multiple learning styles. In our current social context in which streaming media dominates many students’ reception of culture, I argue that shifting our instruction into these spaces can be an effective tool to leverage in the pedagogy of poetic interpretation.

Introduction

The interpretation of poetry is notoriously among the most challenging literary skill sets to teach undergraduate students. At a minimum, it requires sustained attention to linguistic and structural detail, as well as knowledge of a few poetic forms. As a result, students often feel that they are on the outside looking in, trying to understand poetic conventions in which they have no stake. The challenge for the poetry instructor thus becomes encouraging students to engage intellectually with a genre they, at best, fear is inaccessible, and, at worst, feel is irrelevant to them.

This paper will present one solution to this conundrum: leveraging streaming platforms that students are already using to make poetry more accessible and to explore the rich multivalence of poetic adaptation. To implement this solution, I designed a course in which students would engage with streaming audio recordings of poetry alongside traditional print texts. My hope was that this approach would overcome the obstacles associated with teaching poetry in three key ways. First, it would encourage students to explore poetry through an interactive and already-popular medium on their smartphones: Spotify. Second, it would highlight the mutability of poetry over time, demonstrating how the poetic tradition is in a constant state of self-refashioning through performance. In this way, students would become more empowered in the interactive process of reading, interpreting, and adapting poetry. Third, it would, by requiring students to read poems while also listening to them on a digital platform, make poetry feel more accessible rather than guarded behind the walls of high culture.

The results of this semester-long experience, outlined below, demonstrate a practical and effective tool for the teaching of poetry interpretation. The assessment data suggest a self-reported increase in active engagement with poetry through the streaming of Spotify playlists, as well as self-reported improvement in the ability to interpret poetry. Yet perhaps more significantly, the students in this course (consisting of non-English majors) became excited about the possibility of poetic adaptation—both in the analysis of interpretations encountered through the Spotify playlist and in their own performances. By the end of the course, they expressed a new level of interest and comfort in interpreting poetry. In describing the multimodal processes of engaging with streaming media and poetic adaptation that led to such an outcome, this paper will underscore the usefulness of shifting our pedagogy of poetry interpretation into interactive platforms our students are already widely using as a way to improve their skills and confidence as active shapers of an accessible poetic tradition.

Approach

My aim was to make a 300-level poetry seminar (part of the general studies requirements at a STEM and business university) feel more interesting, accessible, and contemporary to a student population likely not inclined to the self-initiated study of poetry. My course design was inspired by the explosion of streaming media options over the last decade, which I believed would provide new possibilities of active learning engagement for students of poetry. This study also builds on and extends previous efforts to rethink the pedagogy of poetry interpretation within the framework of multimodality. In an early example of this process, Mary McVee, Lynn Shanahan, and Nancy Bailey (2008) describe using PowerPoint projects in the pre-streaming era to combat student antipathy and anxiety surrounding poetry interpretation. More recently, Hessa A. Alghadeer (2014) provides a foundation for the pedagogical effectiveness of adapting poetry with digital platforms like Twitter, Instagram, and Prezi. Meanwhile, Violeta Janulevičienė and Deimantė Veličkienė (2015) similarly note how using digital adaptations to teach Shakespeare’s sonnets will “shift from monomodality to multimodality,” wherein “utilizing several modes of meaning making create new meanings” (2015, 210, 212). In engaging such multimodalities, working with playlists provides students with plentiful opportunities to ask critical questions about the typical ways genres are categorized by companies like Spotify (Ball, Sheppard, and Arola 2018, 76–77)—topics especially pertinent to poetry seminars like mine mixing print text, spoken word recitation, and digital adaptation. This paper continues this focus on multimodality, aiming to bring poetry into the digital spaces already inhabited by students as a means of increasing interpretive and adaptive engagement.

Finally, this study expands the body of scholarship focused on playlist curation as a pedagogical aid by applying existing models to the teaching of poetry. Anne Burdick, Johanna Drucker, Peter Lunenfeld, Todd Presner, and Jeffrey Schnapp (2012) suggest that playlist curation can serve as a type of content modeling in which decisions like sorting by genre, artist, or composer can open essential questions for students within the digital humanities (19). From the perspective of music education, Scott Jeppesen (2017) writes how “Online listening also empowers teachers to use technology to add additional interactive possibilities to their classes” (60). Both of these studies are indicative of how playlists could significantly revise the pedagogy of the poetry classroom by requiring students to consider critically the process of content curation through interactive streaming media.

Selection of Platform

I selected Spotify as the platform through which to disseminate streaming poetry performances and adaptations for this class because of its free account option and its ability to provide a combination of material essential to the course: poetry read by original authors, poetry read by interpreters, music incorporating poetry or poetic allusions, and the capability to build and share playlists. In respect to the first and second points, Spotify allows for the streaming of the entire catalog of the Smithsonian Folkways label, a nonprofit entity that houses the recordings of the original Folkways Records label. The long-playing vinyl records, and later cassettes and CD-Rs, of this label were once a staple of American libraries; however, with the decline of physical media, many institutions have eliminated such collections. As a result, these vital recordings have become underutilized in the era of digital media. The Smithsonian Folkways holdings include numerous albums of poetry readings by original authors, such as the seminal Anthology of Negro Poetry (1954) featuring recitations by poetry course staples such as Langston Hughes, Countee Cullen, and Gwendolyn Brooks. Likewise, Smithsonian Folkways hosts albums of readings of canonical poetry such as English Romantic Poetry by John S. Martin (1962) and Early English Poetry by Charles W. Dunn (1958), to name only two among many.

Spotify’s vast catalog represents its most significant advantage as a streaming platform when teaching poetic adaptation. Instructors are able to curate playlists containing poetry readings side-by-side with musical recordings that either directly adapt a poem or expand its themes. In respect to direct adaptation, Spotify’s plentiful offerings within the ballad tradition provide a meaningful illustration for students of ballads’ adaptability and mutability over time. More broadly, Spotify’s access to many popular recordings since the advent of recorded sound provides a foundation for demonstrating to students the continued relevance of poetry. For instance, listening to Richard Burton’s reading of Samuel Taylor Coleridge’s Rime of the Ancient Mariner alongside Iron Maiden’s metal adaptation “Rime of the Ancient Mariner” (1984), which weaves lines from Coleridge’s original text with new expansions of the plot, offers students an unexpected and productively challenging example of the continued resonance of poetry in a popular music context.

Spotify is also already used to a significant extent by people who are the age of the traditional university student and provides a free option for those unable to afford a premium account. As of October 2021, Spotify has a userbase of 365 million users (Spotify 2021). The demographics of this userbase tend toward the age of traditional undergraduate students, with a recent study finding that people between the ages of 18 and 35 are significantly more likely to use Spotify than people over 35 (Gomes, Pereira, Soares, Antunes, and Au-Yong-Oliveira 2021, 348). In fact, in the survey forming the basis of the study, 89.1% of respondents between the ages of 18 and 25 indicated that they are Spotify users (Gomes, Pereira, Soares, Antunes, and Au-Yong-Oliveira 2021, 348). As such, many students using Spotify for class will not need to download or learn the mechanics of a new platform, removing a potential impediment to learning engagement. Because of this pervasive use, Spotify represents the most accessible option for student audio streaming absent larger institutional support for a non-commercial option. Anecdotally, of the students enrolled in my poetry seminar, 11 of the 22 had a premium Spotify account (at discounted student rates) upon entering the course, six had a free account, and five did not have an account. I guided the students who did not have a Spotify membership through the steps of creating a free account, which would be accessible on their phones, laptops, or tablets. (In the event students do not own such devices, they would be able to listen to Spotify within web browsers at the university computer lab using their free accounts.) Yet because students typically prefer to listen to music on smartphones, the Spotify app is particularly useful for inserting poetry into their daily listening habits, putting vital course content directly into their pockets.

As is widely acknowledged, Spotify’s compensation model for artists and songwriters is problematic. Like many digital media platforms, Spotify’s initial promise for the democratization of music distribution has been replaced by “a consolidation of long-established power structures” in which record labels profit at the expense of the artist (Marshall 2015, 185). At best, streaming has been a double-edged sword that has lowered digital piracy while also depressing music sales (Aguiar and Waldfogel 2018). More unfortunate still, for much of Spotify’s existence artists would “receive reduced benefits because their royalty rates are lower” (Lesser 2018, 291), though this issue may be somewhat ameliorated with the passage of the Music Modernization Act in 2018. In this way, Daniel S. Hess (2019) argues the MMA will at least provide independent artists “an approachable means to collect royalties” (200–201), although the royalty rate per stream is only $0.004 as of early 2021 (Owsinski 2021).

At the same time, Spotify provides the most accessible option for students at the present moment due to its free account tier and massive user base, making it a pragmatic—if not ideal—choice for the poetry instructor. Unlike other major music streaming platforms like Apple Music or Tidal, Spotify offers an advertisement-supported free account option, thereby allowing students without the resources to pay for a premium account full access to the course playlist. As such, these free accounts facilitate the easy exchange of playlists when building a required listening list for a course. Additionally, free Spotify accounts allow students to participate in the creation and sharing of their individual poetry playlists. More than that, free Spotify accounts, like paid accounts, provide the ability to make collaborative crowdsourced playlists for group projects. For these reasons, Spotify functions as a particularly effective classroom tool even in its free version, setting it apart from other current options. In the absence of a noncommercial educational streaming platform with the full functionality and catalogs of commercial options, instructors can more responsibly integrate Spotify into their courses by making students aware of the ethical tradeoffs of using the platform in the class. Editorials by recording artists like Damon Krukowski’s (of Galaxie 500 and Damon and Naomi) “How to Be a Responsible Music Fan in the Age of Streaming” (2018) would serve as an excellent starting point for students. In particular, Krukowski’s emphasis on Bandcamp as a medium for listeners to support artists through direct purchases of digital files and physical media could help students become advocates for artist compensation, as well as more mindful consumers of sound recordings.

Some scholars have also expressed concern with how Spotify, particularly its algorithms for playlists generated by the service rather than users, might undermine the value of art. Ekberg and Schwieler (2020) argue how Spotify’s structure, particularly these algorithmically-generated playlists, can turn art and people into ephemeral commodities (12). In the context of the poetry course, however, asking students to listen to a course playlist, or even create their own playlists, can work against this dehumanizing possibility by reemphasizing the power of individual interpretation and curation. For instance, Ignacio Siles, Andrés Segura-Castillo, Mónica Sancho, and Ricardo Solís-Quesada (2019) contend that Spotify “playlists can become the basis of a shared affective experience,” suggesting how playlists can harness social power for students (7). Indeed, I often overheard students discussing the playlist before class sessions in terms of affective experience, suggesting one way by which streaming poetry playlists foster not only a pedagogical but also a deeply social collaborative experience.

Method

The syllabus communicated to students the aim of the class related to poetry interpretation and adaptation: “the course will encourage students to approach poetry from a performative perspective—both in exposure to others’ performances and in students’ own original articulations.” As such, students knew from the outset that they would engage with a shifting poetic tradition through streaming audio of poetry and the performance of their own adaptations. Thereafter, the syllabus required print readings alongside listening assignments for each session (see Figure 1).

Poetry course Spotify playlist featuring image of Emily Dickinson and recordings of ballads by Joan Baez, Ween, and Jean Ritchie
Figure 1. Playlist of poetry performances and adaptations incorporated in the class.

The first reading consisted of two foundational English-language ballads: “The Unquiet Grave” and “Bonny Barbara Allan.” In addition to reading them in print form, however, students would also be required to listen to multiple recorded adaptations. For “The Unquiet Grave,” they would hear Joan Baez’s somber 1964 performance, steeped in the acoustic traditions reignited by the folk revival of the late 1950s and early 1960s, alongside Ween’s “Cold Blows the Wind,” a 1997 alternative rock song that expands the ballad in postmodern fashion by shifting the gender dynamic. This side-by-side comparison of two recordings of a traditional ballad would show students how the poetic tradition is constantly remaking itself through adaptation, performance, and thematic revision. It was my hope that students would spend the semester developing an awareness of the elasticity of poetry within this living tradition to counter their anxiety that they would never be able to discover the “right” meaning. Instead, this process would heighten their sense of how subtle changes in performance—in lyrics, melody, tempo, vocal modulations, etc.—can dramatically reshape the meaning of a poetic text.

I would begin most course sessions by streaming one of the required audio recordings to generate critical discussion, thereby encouraging students to think of how poetic performance communicates new meaning. For instance, we started our exploration of the poetry of Gwendolyn Brooks by listening to the author’s reading of “kitchenette building” (retitled “Kitchenette” upon inclusion of the aforementioned Anthology of Negro Poetry). The poem, complete with a pointed rhetorical question (“But could a dream send up through onion fumes”), ironic feminist appropriations of quotations related to gendered behavior (“‘Dream’ makes a giddy sound, not strong / Like ‘rent,’ ‘feeding a wife,’ ‘satisfying a man’”), and multiple exclamation points (“We wonder. But not well! not for a minute!”) (Brooks 2005, 998), suggests a passionate, even angry, response to the frantic confines of domestic female roles. Yet Brooks’ performance of her poem plays with this expectation by reciting in calm, measured tones to highlight yet another way in which the speaker is constrained by the conventions of female propriety. One student noted,

“Kitchenette Building” by Gwendolyn Brooks was a performance that was not quite like I expected it to be, and since the performance by the original author of the poem I was able to change my view of the poem to the way that she had originally intended. Viewing the poem like this allowed a more in-depth understanding of the political battles that she was actually fighting with her words.

This student’s response reinforces the pedagogical power of using streaming audio alongside print poetry: by challenging the authority of the student’s initial interpretations of the print text, audio recordings of authors force the reconsideration of themes within a specific historical and cultural context.

We would also sometimes begin class sessions by listening to a musical work that recirculates the words or themes of a poem to gain a deeper understanding of the ways that poetry’s adaptability allows for contemporary engagement. In one example, we followed the reading of selected poems by Emily Dickinson by listening to Wilco’s “Born Alone” (2011). In this recording, lyricist Jeff Tweedy notes the direct influence of Dickinson’s poetry,

I opened up a book of American poetry and randomly turned to the Emily Dickinson pages, no one poem in particular. I took a lot of words, most of them verbs, and put them against words that looked appealing to me from Whittier and other 1800s poetry. (quoted in Hoyt 2011)

Students then looked for specific allusions to the Dickinson poems within “Born Alone” before exploring the ways in which this adaptation had remade Dickinson’s themes. Our discussion broadened to consider how Dickinson’s nineteenth-century poetry lives on through performance and adaptation in our digital age. Through this process of comparative analysis, I hoped students would gain an appreciation for the ways English-language poetry forms an elastic lineage constantly being shaped, challenged, and remade—even in modes of artistic expression not usually associated with the reinterpretation of nineteenth-century American poetry.

The emphasis on poetic performance was punctuated by each student adapting a self-selected poem in class, either live or by digital recording. This component of the course builds on the work of Daniel Anderson and Emily Shepherd (2016) on e-Poetry, which suggests the rich multimodal possibilities of students adapting poems into media projects in order to “learn new digital writing skills and enjoy extended engagement with the poems.” In this assignment, I communicated to students the ways by which their subtle shifts in tone, pace, and volume could affect the meaning of the poem for their audience of classmates. Additionally, I asked them to consider how the process of digital recording could be transformative, requiring nuanced attention to the multimodal experience of crafting a visual recording of a written text. At the end, students would also articulate the ways by which their performance and digital framing were designed to emphasize specific themes of the original poem. Finally, each student would lead a discussion probing the meaning of the poem via the adaptation. In each of these ways, this assignment would encourage students to take ownership of the interactive process of adaptation, as well as make the genre feel more accessible, relevant, and genuinely meaningful in a contemporary context.

Analysis

I asked the students to complete an anonymous survey about their experiences in the class as a means of gauging the effectiveness of my approach in meeting the course’s goals. Absent a university- or department-wide student survey that would sometimes function as the basis for evaluating specific activities within general education courses (Walvoord 2010), I composed a series of multiple choice and open-answer questions to assess how the Spotify playlist sequence may have increased confidence in poetry interpretation and improved engagement in reading poetry through interactive digital processes.

Notably, the students self-reported low confidence in their ability to read poetry before enrolling in the course. In the survey, 20 of the 22 students reported either “Not Proficient” or “Somewhat Proficient” as their initial skill level in reading poetry, while only two reported “Proficient” and zero reported “Highly Proficient.” My initial conversations with students, as well as our discussions early in the semester of ballads, confirmed this self-reported lack of confidence. As is often the case, these non-humanities majors exhibited substantial anxiety about ever being able to “get” poetry. By the end of the term, however, the students’ confidence (as reflected by their self-reported proficiency upon exiting the course) had significantly improved. Indeed, 21 of the 22 students reported either “Highly Proficient” or “Proficient” as their skill level in reading poetry after taking the course. Clearly, these students had a much greater degree of confidence in their interpretive ability, thereby breaking through their initial fear of never being able to “get” poetry.

Next, I prompted students to reflect on whether the analysis of our required listening contributed to a shift in interpretive confidence by asking, “Did the playlist help make deciphering poetry a more accessible process?” In response to this open-ended question, many students’ viewpoints overlapped with this sentiment expressed by one of their classmates: “spoken word poetry is usually less challenging or daunting than written poetry.” Another student pointed toward how listening facilitated understanding beyond the readings: “With most poems that I was confused with while reading, the recitations on the playlist were able to help me figure out what the meaning was by emphasizing certain words/lines.” This feeling was echoed by a classmate who wrote, “For the hard to follow poems, the adaptations helped me follow and understand them better.” Overall, the students in the course repeatedly emphasized how the consideration of our streaming playlist facilitated the confidence to assert understanding of the texts.

Likewise, many of the students conveyed that listening to adaptations from our course playlist authorized them to identify new meaning and formal techniques in the required readings. As one student noted, “Sometimes you will hear things that you did not pick up when reading or hear it in a way that changes your perspective on the poem.” Several students echoed this reaction that listening to poetic adaptations acted as a conduit for identifying nuances in the poems that, in turn, shifted their interpretation of the print text. One student wrote, “Each adaptation got me to think critically about what the text was saying, how it was saying it, and what elements of that were brought in the recitation.” Another student noted that, although not all adaptations were appealing, the process of analyzing why a particular performance did not work was instructive: “I didn’t like all of the adaptations, but hearing them and being able to describe why I didn’t like them and how they related to the original poem helped me to understand the art of performance poetry a lot better.”

Several students also explained how the incorporation of interactive streaming technology made the readings feel more contemporary and, therefore, accessible. For instance, one student reflected:

it brought the process into the modern technological age. I kind of got stuck in this class and wasn’t really looking forward to it, but the playlist allowed me to get so much more out of the course than I was expecting. I thought the class would be really dry and we’d just be counting syllables for 10 weeks, but the addition of the recitations livened it up.

In this way, streaming media had helped me overcome a central hurdle in teaching poetry to the general studies student: making the texts seem relevant, accessible, and more than only exercises in technical analysis. One student emphasized how the listening contributed to seeing the readings as more than a purely isolated academic exercise: “they gave me an idea of how these poems were used and performed in the real world.” One student even volunteered that, absent the assigned playlist, “I probably would have gone to YouTube and looked up adaptations to help with my comprehension of the poem.” Formalizing this impulse through the creation of a shared playlist both directs students to thought-provoking adaptations and aligns the classroom experience with their typical processes as learners in the digital age. Students did not have to search through a variety of streaming platforms for digital adaptations of the readings but rather could reach into their pockets and shuffle the class playlist any time they wanted to engage with poetry.

The students also expressed several ways in which the multimodality of reading and listening to poetry helped them glean new meanings from the poems. One student noted how interacting with various instantiations of the poems aided in the interpretive process: “I feel that in tandem with reading the poetry first this was an effective way to better decipher a poem.” Several other students reiterated this reaction, with one student asserting that “multiple perspectives create(d) a well-rounded interpretation,” and another writing that “This method provided multiple mediums to capture the information.” Seemingly, this consistent appeal to multimodality, performance, and adaptation through digital media had allowed students to gain confidence in their interpretations.

Still three other students emphasized the importance of our playlist for appealing to their learning styles: “I am an auditory learner,” “Hearing words while reading them gives me two ways to decipher poems,” and “My comprehension has always been better when I listen to something instead of read it, so being able to do both was something that helped me considerably in my understanding.” Although most poetry courses feature reading aloud of texts during the course session, and many courses in the era of physical media included intermittent listening, the consistent integration of a digital playlist also appealed in a powerful way to these learners. One student succinctly summarized this appeal to auditory learners: the playlist “made the homework assignments more enjoyable to either get a review of what you read or hear a new way of how the poem could be interpreted.”

Many students also demonstrated a positive learning experience in creating their own adaptation of a poem. In one particularly successful digital adaptation, a student selected “Boots of Spanish Leather” by Bob Dylan as his source text—which we already read in print form in The Norton Anthology of Poetry, as well as listened to two drastically divergent performances by Dylan (his initial studio recording, a spare and earnest folk rendition, from 1963) and by hard-rock band Nazareth’s lead singer, Dan McCafferty (a mainstream rock rendition from 1975). Building on our discussions of the shifting themes of the various instantiations of the poem, the student recorded, edited, and shared a digital video of himself singing while playing guitar—featuring phrasing, tempo, and emphasis radically divergent from Dylan’s original take with the same instrumentation. Thereafter, the student led a discussion in which he invited his classmates to reflect on how his filmed visual performance shifted the meaning of Dylan’s words. This culminating project thus engaged in valuable “critical examinations of literary texts” via “mediating across sign systems,” a process explored by Heidi Höglund’s study of student video interpretations of poetry at the secondary level (2017, 43). In keeping with recent work in composition studies, however, students also had the option to perform their works in person depending on the goals of their interpretive recitation. As Jody Shipka (2013) notes, multimodality should not be viewed as synonymous with “digitally based or screen-mediated texts”; instead, students should “leave our courses exhibiting a more nuanced awareness of the various choices they make throughout the process of accomplishing that work and the effect those choices might have on others” (76). In this spirit, students were able to take ownership of their performances by defining their process of interpretation and adaptation—therein demonstrating their power to engage actively with the tradition of English-language poetry in the genre of their choice rather than acting only as passive readers.

Future

Since the implementation of the Spotify playlist in this poetry seminar, I have expanded this approach in subsequent courses to include students curating their own public playlists on Spotify. As Anja Nylund Hagen (2015) argues, although playlist curation is not wholly removed from the processes of collecting “rare gems” of physical media, “playlist collecting involves imposing one’s will (and oneself) upon an intangible realm of endless abundance” (643). This narrowing process, in which students select a particular theme, issue, or timeframe and create a succinct one-to-two-hour playlist from Spotify’s overwhelming amount of recorded material allows them to take ownership of research processes for public outreach. In my projects, after creating an overview of relevant recordings (both spoken word and musical) students select a playlist image and write a brief description to draw in listeners, as well as submit a “curator’s statement” in which they outline key aspects of the theme, issue, or period while explaining the inclusion of these particular recordings. Such public-facing acts of criticism and curation have provided a meaningful context in which my students have forged unexpected intellectual connections while also serving as a training ground for more traditional argumentative research-based essays later in the semester. Kelly J. Hunnings (2019) also suggests how the curation of Spotify playlists from the perspective of a fictional character can provide a meaningful space for students to engage with pre-twentieth-century literature in the digital era. In all of these ways, a Spotify curation project carries through the themes of the original reading and listening assignments by asking students to become informed content-creators of streaming media in a real-world setting.

My students’ experiences in the original seminar and subsequent courses demonstrate how streaming media is a valuable tool at the disposal of the poetry instructor. By overcoming the all-too-frequent student intimidation or resistance at the prospect of poetry interpretation, interactive streaming media helps make poetry feel comprehensible and approachable to students. Moreover, asking them to create a unique adaptation or public playlist enables them to take ownership of their reading practice through active interpretation. As streaming media platforms continue to evolve at a rapid pace, we, as poetry instructors, can continue this work to rethink our practice within the context of technologies already shaping the cultural context of our students’ lives.

References

Aguiar, Luis, and Joel Waldfogel. 2018. “As Streaming Reaches Flood Stage, Does It Stimulate or Depress Music Sales?” International Journal of Industrial Organization 57: 278–307. https://doi.org/10.1016/j.ijindorg.2017.06.004.

Alghadeer, Hessa A. 2014. “Digital Landscapes: Rethinking Poetry Interpretation in Multimodal Texts.” Journal of Arts and Humanities 3, no. 2: 87–96. https://www.theartsjournal.org/index.php/site/article/view/354/237.

Anderson, Daniel and Emily Shepherd. 2016. “I Lit: An E-Poetry, E-Portfolio Exhibit.” The Journal of Interactive Technology and Pedagogy 10. https://jitp.commons.gc.cuny.edu/i-lit.

Ball, Cheryl E., Jennifer Sheppard, and Kristin L. Arola. 2018. Writer/Designer: A Guide to Making Multimodal Projects. Boston: Bedford/St. Martin’s.

Block, Friedrich W. 2007. “Digital Poetics or on the Evolution of Experimental Media Poetry.” In Media Poetry: An International Anthology, edited by Eduardo Kac, 229–43. Chicago: University of Chicago Press.

Brooks, Gwendolyn. 2005. “kitchenette building.” In The Norton Anthology of Poetry: Shorter Fifth Edition, edited by Margaret Ferguson, Mary Jo Salter, and Jon Stallworthy, 998. New York: W. W. Norton & Company.

Burdick, Anne, Johanna Drucker, Peter Lunenfeld, Todd Presner, and Jeffrey Schnapp. 2012. Digital_Humanities. Cambridge: The MIT Press. https://mitpress.mit.edu/books/digitalhumanities.

Ekberg, Nicals and Elias Schwieler. 2020. “Evolving Bildung, Technology and Streaming Art.” Popular Communication: 1–16. https://doi.org/10.1080/15405702.2020.1744608.

Gomes, Inês, Inês Pereira, Inês Soares, Mariana Antunes, and Manuel Au-Yong-Oliveira. 2021. “Keeping the Beat on: A Case Study of Spotify.” In Trends and Applications in Information Systems and Technologies: Volume 2, edited by Álvaro Rocha, Hojjat Adeli,Gintautas Dzemyda, Fernando Moreira, and Ana Maria Ramalho Correia, 337–52. Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-030-72651-5_33.

Hagen, Anja Nylund. 2015. “The Playlist Experience: Personal Playlists in Music Streaming Services.” Popular Music and Society 38, no. 5: 625–45. https://doi.org/10.1080/03007766.2015.1021174.

Hess, Daniel S. 2019. “The Waiting Is the Hardest Part: The Music Modernization Act’s Attempt to Fix Music Licensing.” Journal of Law, Technology & Policy 1: 187–209. http://illinoisjltp.com/journal/wp-content/uploads/2019/05/Hess.pdf.

Hoyt, Alex. 2011. “Watch Wilco’s New ‘Born Alone’ Video and Read the Story Behind Its Lyrics.” The Atlantic, September 7, 2011. https://www.theatlantic.com/entertainment/archive/2011/09/watch-wilcos-new-born-alone-video-and-read-the-story-behind-its-lyrics/244656.

Höglund, Heidi. 2017. Video Poetry: Negotiating Literary Interpretations. Åbo, Finland: Åbo Akademi University Press.

Hunnings, Kelly J. 2019. “Creating Multimodal Documents to Understand Literature.” The Journal of Interactive Technology and Pedagogy, June 11, 2019. https://jitp.commons.gc.cuny.edu/creating-multimodal-documents-to-understand-literature.

Janulevičienė, Violeta and Deimantė Veličkienė. “Contemporary Shakespeare’s Sonnets Interpretation: From Monomodality to Multimodality.” Journal of Language and Literature 6, no. 4: 208–12.
https://www.researchgate.net/publication/309111381_Contemporary_Shakespeare’s_sonnets_interpretation_From_monomodality_to_multimodality.

Jeppesen, Scott. 2017. “Humanizing the Deities of Jazz.” Music Educators Journal 103, no. 4: 57–63. https://doi.org/10.1177%2F0027432117697799.

Krukowski, Damon. 2018. “How to Be a Responsible Music Fan in the Age of Streaming.” Pitchfork, January 30, 2018. https://pitchfork.com/features/oped/how-to-be-a-responsible-music-fan-in-the-age-of-streaming/.

Lesser, Bryan. 2018. “Record Labels Shot the Artists, But They Did Not Share the Equity.” The Georgetown Journal of Law & Public Policy 16, no. 1: 289–314. https://www.law.georgetown.edu/public-policy-journal/in-print/volume-16-number-1-winter-2018/record-labels-shot-the-artists-but-they-did-not-share-the-equity.

Marshall, Lee. 2015. “‘Let’s keep music special. F—Spotify’: On-Demand Streaming and the Controversy Over Artist Royalties.” Creative Industries Journal 8, no. 2: 177–89. https://doi.org/10.1080/17510694.2015.1096618.

McVee, Mary, Lynn Shanahan, and Nancy Bailey. 2008. “Using Digital Media to Interpret Poetry: Spiderman Meets Walt Whitman.” Research in the Teaching of English 43, no. 2: 112–43. https://www.researchgate.net/publication/234652965_Using_Digital_Media_to_Interpret_Poetry_Spiderman_Meets_Walt_Whitman.

Owsinski, Bobby. 2021. “Spotify Justifies Its Payouts With Royalty Data Site ‘Loud&Clear.’”Forbes, March 21, 2021. https://www.forbes.com/sites/bobbyowsinski/2021/03/21/spotify-justifies-its-payouts-with-loudclear/.

Shipka, Jody. 2013. “Including, but Not Limited to, the Digital: Composing Multimodal Texts.” In Multimodal Literacies and Emerging Genres, edited by Tracey Bowen and Carl Whithaus, 73–89. Pittsburgh: University of Pittsburgh Press.

Siles, Ignacio, Andrés Segura-Castillo, Mónica Sancho, and Ricardo Solís-Quesada. 2019. “Genres as Social Affect: Cultivating Moods and Emotions through Playlists on Spotify.” Social Media + Society 5, no. 2: 1–11. https://doi.org/10.1177/2056305119847514.

Spotify. 2021. “Company Info.” Accessed October 16, 2021. https://newsroom.spotify.com/company-info/.

Walvoord, Barbara E. 2010. Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass.

About the Author

Stephen Grandchamp is Assistant Professor of Literature and Digital Humanities at the University of Maine at Farmington, where he is also the Co-Director of the New Commons Project (a public humanities initiative sponsored by the Mellon Foundation) and Manager of the Digital Humanities Lab. His areas of research interest include: failure in the traditional bildungsroman, video game adaptations of literary texts, and the integration of digital tools into the literature classroom.

The home page, or title page, of the game.
0

Authoring an Open-Source Game for a Faculty Open Educational Resources Workshop: A Case Study

Abstract

While most college institutions running open educational resource (OER) conversion initiatives currently teach some online version of the OER faculty workshop, little scholarship exists on how this type of instruction can be optimized for best comprehension of, and engagement with, open resources, especially in a remote context. An overview of existing models of the workshop and their digital pedagogy philosophies is followed by a discussion of one initiative currently implemented by Baruch College’s Center of Teaching and Learning, which embraces online gaming as an effective modality to reinforce textual and auditory learning. The Old Régime and the OER Revolution, an online interactive tutorial created with the open tool Twine, is the first RPG for faculty to engage with open educational resource conversion issues. The game can be accessed in its current beta version. Because Twine encourages players to think in terms of choices and their implications while considering multiple scenarios, introducing this tool to college faculty via the OER workshop can also encourage its uptake in a plethora of other pedagogical contexts.

Introduction

The faculty development OER workshop is one of the most important, yet little-researched, links in the chain of adoption of open resources on campus. Most of the models that exist acknowledge the importance of concept learning for instructors encountering terms such as OER, ZTC, and open pedagogy for the first time. Moreover, teaching faculty about best practices in fair use and Creative Commons licensing—a related field—has a trickle-down effect to undergraduates. One way to achieve familiarity with open education is through hands-on learning, where instructors model examining and converting course materials to open or zero-cost and abiding by U.S. copyright law.

The digital game-based module in this case study was created as an alternative to the traditional workshop, to meet the needs of instructors as observed after several years of an OER initiative. It gives the users an opportunity to be active participants in a multimedia narrative and provides them with the flexibility of an asynchronous, iterative assignment. It also bases itself in scenario-based learning—problem-centered instructional solutions, which complement and cement the more straightforward directional learning in the regular faculty workshop. And, as a program made with a relatively uncomplicated open-source tool, it provides an attractive use case for both faculty and students.

The OER Faculty Development Workshop: Going Digital, Going Multimodal

The workshop in OER for faculty has not yet emerged as a subject of research, although a few on-campus initiatives have presented their versions, explaining decisions around context and organization. At a base level, the class primarily focuses on certain topics germane to the field: what are open resources and open education in general, what is meant by zero textbook cost, how do open licenses work, why follow best practices when pursuing free content, and how do instructors follow accessibility guidelines. Practice activities involve faculty participants searching for materials to replace assigned readings or, if the initiative is grounded in goal-based learning, determining the course objectives and the kinds of resources required to meet them. More than a workshop, this scope of subjects suggests a training program designed for specialist-facilitated, hands-on learning. However, given the relative novelty of the twenty-year-old field, no set rules exist and no single model prevails.

Several teaching bodies have seized upon the potential of the OER workshop to be taught remotely. In 2017, the City University of New York (CUNY) was awarded a $4 million package as part of a Scale Up! OER grant “to establish, sustain and enhance new and ongoing OER initiatives throughout CUNY” (CUNY Libraries 2017). Most of the work of converting a class to open involves searching across multiple databases, and variable concerns arise depending on the instructor’s discipline. The nature of instruction, even at these online-optimized institutions, thus remains of necessity hybrid, combining asynchronous content delivery with a synchronous mode for live discussion and troubleshooting. The possibility to take such courses remotely and for free has added convenience and flexibility; their parallel emergence at public commuter colleges in the CUNY and the Washington State Community college systems has been no accident.

In 2017, Lehman College-CUNY Director of Online Learning Olena Zhadko and Susan Ko, Director of Faculty Development at the NYU School of Professional Studies, developed an asynchronous, online faculty OER workshop to be offered for a two-week period several times during the year. The mini-course, designed for incorporation into the Blackboard LMS, scaffolds understanding of OERs through a module-based system, based on a tripartite organization of defining, finding, and integrating OER, as well as a discussion forum. The learners are asked to think about how open fits into their teaching and articulate their rationale for selecting a specific OER. Their final project in the workshop consists of a list of course materials replaced by open or zero-cost equivalents, with justification and description of intended use: adoption, adaptation, or creation. The library department at Lehman College has an accompanying final assignment for hands-on practice, which tests participants’ ability to link to websites, images, and video, apply attribution, and choose Creative Commons licenses (Cohen 2018).

Simultaneously with Lehman, the Washington State Board for Technical and Community Colleges—the creator of the Open Attribution Builder—developed a free, self-paced course made up of ten modules, including one on accessibility and a reflection on “Why OER Matters.” The website likewise includes “Personal Journeys Using OER,” a series of videos by on-campus practitioners. This version appears less hands-on yet also more personal than Lehman’s, emphasizing the human element of course conversions. It co-exists in a face-to-face format, offered over a two-week period at set points throughout the academic year. These inroads in digital education have opened up the possibility of yet more interactive ways for faculty to engage with the material in concert with the existing learning structure.

Creating a New Mode for the OER Workshop at Baruch College

The Baruch College-CUNY Center for Teaching and Learning joined the OER initiative the same year as Lehman and created an in-person faculty seminar accompanied by a set of slides since made publicly available on the Center’s OER pedagogy site. Baruch’s seminar, conducted in a mix of presentation and workshop elements, features backward course design (What outcomes am I looking for? How will I measure if these outcomes have been achieved?) as well as choice of platform (mainly between the Blackboard LMS and the proprietary, though open, WordPress site Blogs@Baruch). The most recent addition to the topics covered in the workshop have been open-source digital tools (Hypothes.is, StoryMap JS, Timeline JS, Voyant, and Twine), which are simple to use and versatile enough to create OERs for both teaching demonstrations and final assignments, i.e. teacher- or student-led work.

In 2019, on the suggestion of Center for Teaching and Learning Director Allison Lehr Samuels, I began creating a future addition to the workshop. In the spirit of providing opportunities for faculty to educate themselves about OER and related topics, the game, like the slides, was to be made openly available on the TeachOER website and licensed under a Creative Commons Attribution-ShareAlike license (CC-BY-SA). Four goals for this project had materialized: (a) showcasing one of the digital tools, (b) adding another mode to the regular workshop to cater to different kinds of learners, (c) creating a fully asynchronous engagement experience with the workshop, which could also serve as review material, and (d) incorporating recurrent questions that had come from workshop participants for the past three years of the OER initiative.

The technology to carry out these diverse functions had to align with the Center’s principles of using open-source tools to support interactive digital solutions whenever possible. The college had already delved into creating single-player digital training with the 2009 Interactive Guide to Producing Copyrighted Media, developed by Baruch’s Computing and Technology Center and Kognito Interactive, a paid tool used for health simulations, owned and managed by college alumni. The experience took the user through an interactive maze consisting of various copyright and fair-use related situations drawing on such resources as U.S. copyright law, copyright guidelines for CUNY libraries, and public domain information. While this work was produced before any of today’s more common open-source tools and predated the state grant-funded initiative by several years, the Baruch library’s concern for adhering to best practices in decisions around intellectual property has been behind much of its OER programming as well.

The ultimate decision to make the new OER faculty workshop model with Twine, a non-linear, interactive storytelling tool for making single-player games based on decision-making, was grounded in Baruch’s longtime support for classroom technology as well as the Center for Teaching and Learning’s embrace of game-based pedagogy. According to the Center’s team,

The prevailing theory is that games enhance and mobilize internal desires and motivations for learning…[W]hen the rules of a game are tailored around learning outcomes, students are self-motivated to learn in order to ‘win’. Role-playing games in particular have been shown to increase student engagement and motivation to read, enhance persuasive writing and speaking, and increase critical and analytical thinking. (Baruch College CTL 2020)

Pedagogy games have in fact been shown to support classroom learning and stimulate greater engagement from students (Clark 2016; Gross 2007; Hamari, Koivisto, and Sarsa 2014). Internationally renowned game designer Jane McGonigal points out the salutary effect of games on students who experience the “satisfactions of achievement and mastery" (McGonigal 2016, 231). And a 2012 study by the Mathematics Education Research Journal showed 93% of class time was spent on task when using game-based learning, compared to only 72% without it (Bragg 2012). As the anti-rote memorization approach, game-playing is directly associated with active learning and has been a part of a general pedagogical shift towards fostering creativity by working through challenges and coming up with innovative solutions (Cheka-Romero and Gómez 2018; Lameras et al. 2015).

The Baruch CTL, already experienced in active learning practices, therefore considered the gaming approach optimal for the dual task of creating an alternative mode for faculty instruction and modeling new teaching techniques. And, with the target audience being the college’s largely contingent and overextended faculty, the team’s hope was that this exercise’s immersive approach would provide a high engagement—as well as an edutainment—quotient.

Twine and Digital Pedagogy

“Games should be created by everyone” is a motto that Chris Klimas, the creator of Twine, abides by in his work (Klimas 2020). The award-winning, Baltimore-based web developer and game designer published his text-authoring tool in 2009. The target audience was writers, and the experience of creating with Twine was meant to resemble a brainstorming exercise. Paying tribute to this legacy, its home page, www.twinery.org, to this day represents a corkboard with multi-colored notes attached to it with pushpins—a working surface of inspiration.

Passages in Twine’s editor interface appear in a simple, visual flowchart, a map of text nodes connected by arrows, to be filled out, then moved and rearranged at will. The starting point is a single node, and new ones are made when a user encloses the name of another passage in double brackets to continue the story. The scripting does not require any programming experience, although basic HTML skills are needed to change the style sheet and embed images, audio, and video. These technical features add up to a flexible, user-friendly tool with a low barrier to uptake, as well as quick connection capabilities due to its text-based nature. Scholar Anastasia Salter (2016) explains that an additional strength of Twine is its deregulated publication process, meaning that games can be distributed as quickly as they are created—unlike most products of subscription-based gaming websites, which go through an extensive review process before they can go live. This affordance of the open-source tool has also encouraged a greater immediacy and unselfconsciousness of the products made with it, as will be discussed later.

The main vehicle for moving through a Twine game has traditionally been the decision made by the user at the end of each passage between two or three options, each leading to a different fork in the narrative (occasionally, for the purposes of simply proceeding with the story, there may be just one choice). The origins of this decision-tree setup can be traced to “Choose Your Own Adventure,” a series of interactive fiction created in the 1970s by Edward Packard. Klimas (2020) fondly remembers devouring the CYOA books during regular visits to the public library and, later on, trying to “repurpose the genre for something more adult.” A manual on Twine for which Klimas served as the technical editor has recommendations for strong character arcs and narrative devices such as gifts and secret powers (Ford 2016). The author, who envisioned games like Dungeons and Dragons in book form, has since seen his stripped-down writing tool take on a superhero’s cloak—many of those using Twine build role-playing games that feature fighting fantasies. Another influence was hypertext in early Macs and the culture of object-oriented programming in graphical environments.

In a surprising development, yet one that its creator has heartily welcomed, the indie gaming community originally put Twine on the map in 2012–13, crafting a whole body of “vignette” stories with messages that are often unsettling or subversive. In a field dominated by males, its association with female voices and the experience of queer and marginalized groups has lasted to this day (Harvey 2014). Titles made with Twine range from the offbeat Cat Petting Simulator to 2014’s The Depression Quest, a work initially controversial for its very existence as a “game” yet now hailed for raising awareness of clinical depression by including crossed-out “non-choices”—regular actions impossible for someone afflicted with the condition. Salter (2016) refers to this feature as “metaphors of limitations” and praises the power of Twine’s economy of means as helpful for conveying emotion and raising awareness of mental health issues. (She also stresses that it is one of the few vehicles for games with non-mainstream/taboo subjects.)

While Twine has largely flown under the radar, the darling of a niche creative community, greater exposure has recently come to it from popular artistic culture—and, to some extent, the educational environment. The new-media artist Porpentine Charity Heartscape, whose unsettling, subversive games are made largely with the tool, had her works, With Those We Love Alive and howling dogs, featured in the 2017 Whitney Biennial (Klimas 2017). The following year, Charlie Brooker, the creator of the interactive Netflix film Black Mirror: Bandersnatch (2018) announced that its hundred-plus page script was written entirely in Twine (Aggarwal 2019). Around the same time, instructors began discussing their own teaching projects made with the tool and posting assignments made for humanities courses on the open web (McGrath 2019) (McCall 2017, 2019) although the practice remains limited to select classes by the enthusiastic few.

Salter credits Twine’s orientation toward the agency of the main character and lack of a peer-reviewed publishing model with its rising prominence: “Compared to the intensive team-based productions of 2D and 3D games, text-based games continue to offer a space for individual production.” In an industry known for technological extravagance, Twine is the plucky underdog that can achieve inspired—and pedagogically sound—results within a surprising economy of means.

The Baruch Twine Game: Scenario-Based Learning in a “CYOA” Format

According to its reviewers, Twine’s greatest affordance is that it deepens understanding and engagement through exploring alternatives. The Old Régime and the OER Revolution game also aims to mimic the branching, decision-making process of selecting the materials for a class: why and how should faculty abide by best practices? What are the consequences and trade-offs of certain decisions one makes in choosing freely distributed over proprietary copies—for legal repercussions, plain peace of mind, pedagogical freedom? And, more generally, why should instructors care about these issues if they do not yet teach with open resources? The title, juxtaposing “the old ways” and the “OER turn,” alludes to the main character’s specialization: in the game, “you” are a professor of French history. Yet more broadly, it references the pedagogical transformation involved in switching to open resources, which cover a wide range of mediums and modalities and involve not just a replacement of materials but also an examination of the way one teaches.

The decision that launches the game is not to teach in the usual way, as the introduction says of the main character:

You have always struggled with how unsatisfactory and expensive textbooks on the subject are…Given that your students come from so many diverse and international backgrounds, you wish there were some texts that talked more at length about their histories. (Tsan 2020)

As explained in a footnote, the prototype for this hero is Helmut Loeffler, a professor in the History Department of Queensborough Community College, CUNY, who authored a text, Introduction to Ancient Civilizations (2015), with a small grant, creating a book that understandably eschewed timelines, charts, or illustrations (Loeffler 2019). The non-fiction aspect of this “origin story” of the interactive fiction game was intentional, since the overarching goal of any faculty OER workshop is to inspire by actual example.

The Old Régime and the OER Revolution game follows Scenario-Based Learning (SBL), a popular instructional strategy in online training. In this approach, facilitators use situations close to ones that might come up and introduce applicable problems, to which the participants try to figure out the answers. Because SBL is grounded in the premise of applicability for the learners, the role-playing game aspect of many Twine products does not apply here. There are no magic friends, secret weapons or helpful gifts picked up, and the experience stays in the realm of the believable. (However, more than one aside, such as the French history professor’s waxing eloquent over a War and Peace passage she is choosing for her open syllabus, or the character of a curmudgeonly faculty member determined to resist the incursions of OER content, have been included for the purposes of levity or edutainment).

The single-player Old Régime game was designed to be non-competitive and low-stakes, a “game” in the sense of an adventure rather than a quest that results in “winning” or “losing.” In this sense, it is inspired by Jane McGonigal’s assertion that “good game developers know that the emotional experience itself is the reward” (McGonigal 2011, 244). Like most Twine tutorials created for learning and instruction, it follows the simple mode of SBL, “used to validate the learner’s recall and basic comprehension” and “good for basic problem solving” (Pandey 2018). In a similar vein, a Twine game recently made at Hostos Community College, CUNY, “encourages college-level English Language Learners to practice grammar concepts as they play the game narrative” (Lyons and Lundberg 2018). Making the wrong choice of several options sends the user back to do the exercise again until they get it right and can proceed with a visit to the Metropolitan College of Art (Met)—a relatable exercise for Hostos ELL students who frequently go to the real-world Met on field trips. The Old Régime aims for a more branching narrative, since, due to variations in the availability and permissions of course material, the choices taken by an instructor can only sometimes be classified in the right or wrong category.

While The Old Régime does not frequently correct the user’s choices, it has built-in capabilities for addressing decisions that may result in an impasse. If the answer is blatantly incorrect (“copy a 1992 book in its entirety and distribute the PDF to your class”) or fraught (“embed the YouTube link to a culinary video directly in your course website”), it leads to an “explainer”—a tactic common to Twine narratives where the user can explore links embedded within the body of the text for extra information and then go back to the main narrative. The explainers used in the game align almost exactly with the principal concepts being communicated: OER, zero-cost materials, saving money on textbooks, copyright/fair use doctrine, Creative Commons licenses, and open pedagogy (see Figure 2). Since the premise is that participants go through this experience to learn rather than be tested on their knowledge, the game aims to take them through the real-life consequences of certain actions. For instance, one storyline explains that, in a rare case, you may receive a DMCA (Digital Millennium Copyright Act) notice to take down illegal digital content, and another shows how always having a rationale for why you are obeying the fair use doctrine in a specific case might save trouble and keep you on the right side of copyright law.

The story map of the game, which shows the entire story projected on one screen, divided into boxes, or passage links and connected by arrows.
Figure 1. The “story map” view of the Old Régime and the OER Revolution game.
A screenshot of the game, showing a page from the Syllabus 1 story path.
Figure 2. A screenshot of the game.
A screenshot of the game, showing an explainer for DMCA (Digital Millennium Copyright Act) notices.
Figure 3. A screenshot of the game.

Scenario-based learning is often explained in the context of scaffolding the skills being taught to the learners (Clark and Mayer 2012). As the cognitive psychologist Jerome Bruner, who first coined the term while inspired by Lev Vygotsky’s Zone of Promixal Development theory, has posited, the goal here is to provide a series of instructional techniques and support to increase the learner’s understanding, with the end result of increased independence (Wood, Bruner, and Ross 1976). The Old Régime begins with an explanation of the main concepts in the open community, then goes on to offer the user the choice of working through a trio of syllabi types: (a) a traditional list of copyrighted resources, (b) a set of zero-cost multimedia resources, some of them of questionable provenance, and (c) a non-traditional syllabus focused on assignment creation, i.e. open pedagogy (a section paying tribute to recent developments and publications in the field). The issues within escalate in terms of complexity, from questions such as “Can I distribute a PDF I got off the Internet?” to “What digital tools can students use to create multimedia assignments?” While faculty can choose which syllabus to start with, depending on their interests and background, as well as go back and forth between the syllabi, there is a natural progression built into the game from less to more open permissions.

Although most Twine games are text-based, with visuals, if any, used for illustrative purposes, the images, audio, and video in The Old Régime serve an integral purpose: to demonstrate the variety of resources—and issues—a faculty member might come across in their OER course-conversion journey. One page, for instance, features a poster for the Andrzej Wajda-directed film Danton (1983) on Wikipedia, discussed in the game as an example of a fair-use argument (according to the Wikimedia Foundation, the commercial poster had to be used on the website since it was the only available visual of the film). Even on pages where the image cited is not directly related to an argument over openness, the purpose of including it is to reinforce correct attribution practices—each picture in the game comes from Wikimedia Commons and carries either a public domain or a Creative Commons license. A visualized broken link message reinforces a discussion of permanence, and a video from the New York Times’ cooking section is embedded next to a paragraph about the potential dangers of “link rot.”

A screenshot of the game discussing fair use using the example of the poster for the movie Danton on Wikipedia.
Figure 4. A screenshot of the game.

In recognition of the range of curricular decisions available to faculty, the game is signposted throughout with a system of symbols that explain to the learner whether they are being told about a resource that is fully open (green O—“this resource can be used, modified and redistributed freely”), open with restrictions (yellow O—“look out for restrictions on modification and/or redistribution”), and zero textbook cost (yellow Z—“look out for restrictions on use, modification and/or redistribution”). This iconography was originally developed by CTL staff member Pamela Thielman to classify the teaching materials posted on the TeachOER website, as well as to appeal to audiovisual learners, in a sustained commitment to multimodality.

The home page, or title page, of the game.
Figure 5. The home page of the game.
The ending page of the game showing the player having completed the path to OER conversion.
Figure 6. The ending page of the game.

Reflections and Potential Applications

The Old Régime and the OER Revolution game had its planned launch during the Spring 2021 semester OER faculty development workshop, running remotely for the first time with two sections of fourteen participants in total. Due to other priorities that emerged during the COVID-19 pandemic, this initiative did not involve gathering feedback on the game. Moreover, the small sample size has made it difficult to draw causal inferences between game playing and seminar completion. Nonetheless, the anecdotal evidence from having faculty participants play the game—for many, their first encounter with the topic of open-source education—has been positive. The number of questions and points of confusion that the seminar leaders get—especially about OER vs. ZTC and where to look for open-source images—has lessened significantly. One area that needed clarification from a number of seminar participants is the meaning of a Creative Commons license and the mechanism for creating one. This observation is being used to create an update (i.e. a sub-scenario) to the game that focuses specifically on these questions.

Because of the comparative ease of screen sharing and the availability of virtual help, Baruch’s CTL has launched a concomitant initiative to encourage faculty to create their own assignments made with open-source tools. Despite the predictable roadblocks to this effort in pandemic times, it has met with success so far: several faculty members have authored interactive course resources predicated on input from students, which they are currently using to teach their classes. Twine has not featured among their chosen platforms so far, most likely because it still enjoys less visibility on an institutional level and is also perceived as a “more involved” and time-consuming choice. Knight Lab tools, which do not require CSS stylesheet language, especially StoryMap JS, are currently a bigger draw among Baruch instructors. Although more hands-on tutorials are needed to dispel the understandable hesitation many faculty members still feel about creating their own resources, the ensemble of these efforts has gradually led to a greater engagement with, and appreciation of, OER on campus.

In-person survey of the game with pie charts. Part 1. Have you had previous training about OER and/or Open Pedagogy? Yes-both: 60% Yes-OER: 20% No training: 20%
Figure 7. Results to question regarding previous OER training from in-person survey of the game from Baruch-CUNY OER Showcase 2020.
In-person survey of the game with pie charts. Do you feel that you understand how to navigate the game and could play through it on your own?   Yes: 55.6% I don’t know: 33.3% No: 11.1%
Figure 8. Results to question regarding game navigation from in-person survey of the game from Baruch-CUNY OER Showcase 2020.
In-person survey of the game with pie charts.Part 3. What concept do you think you’ve learned most about during this brief, guided experience with the game?  Copyright law: 20% Twine: 20% OER: 20% Zero cost materials: 20% CC licensing: 10% Open Pedagogy: 10%
Figure 9. Results to question regarding key learning concepts from in-person survey of the game from Baruch-CUNY OER Showcase 2020.

Feedback on the game from instructional designers and Baruch faculty who playtested the game during the 2020 Baruch OER Faculty Showcase, featured in Figures 7–9, has been used to edit the current version for greater clarity and playability. A more robust set of instructions is being created and guidelines for helping players navigate through all three scenarios developed. Moreover, the game developer has set up an apparatus with learning objectives and specific questions to better target faculty engagement. A “how we made it” section and detailed instructions for making one’s own basic Twine resource will eventually accompany the game: information on where to find tutorials, what some examples with the best interactivity/greatest emotional impact are, and how the game can be hosted with campus technology. Such a content-based approach in the supplementary material means to combine subject expertise and instructional design, taking its example from openly published resources on making history simulation games and brainstorming creative writing with Twine (McCall 2016, 2017, and 2019; McGrath 2019). More educational solutions—using Twine for process modeling in entrepreneurship classes, to consider alternate plot twists in literature, or to follow scientific processes—are being considered.

These efforts are taking place alongside promising developments in Twine itself. Amherst College Press has announced the publication of EnTwine: A Critical and Creative Companion to Teaching with Twine, a companion to the new and invaluable Twining: Critical and Creative Approaches to Hypertext Narratives (2021) by Anastasia Salter and Stuart Moulthrop. In the past year, Chris Klimas has also released a new story format, Chapbook, which makes it easier to work with multimedia and create and play mobile-friendly games (The Old Régime was written in the default format, Harlowe, authored by Leon Arnott). As stated in a presentation to the Interactive Fiction Technology Foundation’s annual event, NarraScope, the Twine team hopes to add a collaborative authorship function in the future (Klimas 2019), making the tool a natural choice not just for assignment creation, but also for in-class work, either in the face-in-face or asynchronous format.

Conclusion

Instructional designers, pedagogy specialists, and librarians who wish to create their own faculty workshop for teaching about open resources are encouraged to think about building both asynchronous and interactive content into their programming. The changing content of the open education field highlights the importance of providing materials in flexible ways that permit add-ons and variations. Easy-to-use open-source tools constitute a convenient and pedagogically sound resource for the faculty workshop. Twine, with its decision-tree setup and built-in interactivity, presents one possible solution to faculty handling their scenario-based learning independently and remotely alongside other face-to-face or synchronous modalities and asynchronous discussions. The other affordance of “teaching OER with OERs” is the example it provides to faculty who might introduce tools such as Twine to their own classes, creating assignments and perpetuating a culture of best practices in online publishing. The ongoing improvements in the technology suggest that gaming pedagogy can indeed become the vehicle for teaching about openness while promoting active learning and student initiative. However, a concerted initiative that combines instructional design and subject-specific pedagogy is needed to make this vision a reality.

References

Aggarwal, Raghav. 2019. “Bandersnatch: The Marketing Weapon.” June 28, 2019. https://medium.com/@raghavaggarwal0089/bandersnatch-the-marketing-weapon-f8eb6020ed04.

Baruch College Center for Teaching and Learning. n.d. Teach OER (website). Accessed November 8, 2020. http://teachoer.org/.

Bragg, Leicha. 2012. “Testing the Effectiveness of Mathematical Games as a Pedagogical Tool for Children’s Learning.” International Journal of Science and Math Education 10: 1445–1467. https://doi.org/10.1007/s10763-012-9349-9.

Checa-Romero, Mirian and Isabel Pascual Gómez. 2018. “Minecraft and Machinima in Action: Development of Creativity in the Classroom.” Technology, Pedagogy and Education, 27:5, 625–637. https://doi.org/10.1080/1475939X.2018.1537933.

Clark, Douglas, Emily E. Tanner-Smith, and Stephen S. Killingsworth. 2015. “Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis.” Review of Educational Research 86, no. 1: 79–122. https://doi.org/10.3102/0034654315582065.

Clark, Ruth C. and Richard E. Mayer. 2012. Scenario-Based e-Learning: Evidence-Based Guidelines for Online Workforce Learning. San Francisco: Pfeiffer.

Cohen, Madeline. 2018. Responsible Use of Materials for OER: A Hands-On Workshop for Faculty. CUNY Lehman College Publications and Research, CUNY Academic Works. https://academicworks.cuny.edu/le_pubs/241/.

CUNY Libraries. 2017. “CUNY Open Educational Resources.” Accessed November 8, 2020. (2017), https://www.cuny.edu/libraries/open-educational-resources/.

Davidson, Cathy N. 2011. Now You See It: How Technology and Brain Science Will Transform Schools and Business for the 21st Century. New York: Penguin Books.

Ford, Melissa. 2016. Writing Interactive Fiction with Twine. London: Que Publishing.

Friedhoff, Jane. 2014. “Untangling Twine: A Platform Study.” DiGRA ’13 – Proceedings of the 2013 DiGRA International Conference: DeFragging Game Studies, August 2014, Volume 7.

Gross, Begoña. 2007. “Digital Games in Education: The Design of Games-Based Learning Environments.” Journal of Research on Technology in Education 40, no. 1: 23–38.

Hamari, Juho, Jonna Koivisto, and Harri Sarsa. 2014. “Does Gamification Work? A Literature Review of Empirical Studies on Gamification.” Proceedings from the 47th Hawaii International Conference on System Science 2014.

Harvey, Allison. 2014. “Twine’s Revolution: Democratization, Depoliticization and the Queering of Game Design.” Game: The Italian Journal of Game Studies (3).

Hayles, Katherine. 2002. Writing Machines. Cambridge, Massachusetts: MIT Press (Mediawork Pamphlet Series, 1). Interactive Fiction Technology Foundation (2020).

Hostos Community College. 2018. “Grammar Adventure Story.” Hostos Community College, CUNY Commons. Accessed November 8, 2020. https://commons.hostos.cuny.edu/esl/.

Huang, Wendy Hsin-Yuan and Dilip Soman. 2013. A Practitioner’s Guide to Gamification of Education, Research Report Series Behavioral Economics in Action, University of Toronto.

Jackson, Brianne and Virginia Thompson. 2018. “Exploring Twine and Its Use in Active Learning Online.” In Proceedings of Society for Information Technology & Teacher Education International Conference, edited by Elizabeth Langran and J. Borup. Washington, DC: Association for the Advancement of Computing in Education (AACE): 1533–1536. https://www.learntechlib.org/primary/p/182731/.

Klimas, Chris. 2017. “Interactive Fiction Appears in the Whitney Biennial.” The Interactive Fiction Technology Foundation blog, April 2, 2017. Accessed November 8, 2020. https://blog.iftechfoundation.org/2017-04-02-interactive-fiction-appears-at-the-whitney-biennial.html.

———. 2019. “Twine: Past, Present and Future.” Blog sharing of NarraScope presentation at the Massachusetts Institute of Technology, Cambridge, MA, 21 June 2019. https://chrisklimas.com/blog/2019-06-21-twine-past-present-future/.

———. 2020. Interview by Katherine Tsan, December 21, 2020.

Kumar, M. S. Vijay. 2012. “The New Landscape for the Innovative Transformation of Education.” Social Research: An International Quarterly 79, no. 3: 619–630. https://muse.jhu.edu/article/528185.

Lameras, Petros, Thrasyvoulos Tsiatsos, Panagiotis Petridis, Dimitris Tolis, Fotis Liarokapis, Despina Anastasiadou, Aristidis Protopsaltis, Maurice Hendrix, and Sylvester Arnab. 2015. “Creative Thinking Experimentations for Entrepreneurship with a Disruptive, Personalised and Mobile Game-Based Learning Ecosystem.” 2015 International Conference on Interactive Mobile Communication Technologies and Learning (IMCL), Thessaloniki: 348–352. http://dx.doi.org/10.1109/IMCTL.2015.7359617.

Lehr Samuels, Allison, Pamela Thielman, and Katherine Tsan. 2019. “Helping Faculty Navigate OER and Copyright Law: Two Approaches (Instructional and Curricular Design).” Presentation at the 18th Annual CUNY IT Conference, John Jay College of Criminal Justice, New York, NY.

Loeffler, Helmut. 2019. “Experience with Authoring and Using an OER Textbook.” The 16th Annual Open Education Conference, Phoenix, AZ, October 30, 2019.

Lundberg, Karin and Catherine Lyons. 2018. “Using Twine to Deliver a Grammar-Linked Creative Writing Assignment in a Hybrid ESL Course.” The Free Library, November 1, 2018. Accessed November 8, 2020. https://www.thefreelibrary.com/Using+Twine+to+Deliver+a+Grammar-Linked+Creative+Writing+Assignment…-a0563684036.

Lyons, Kate and Karin Lundberg. 2018. “Twine—Using Non-Linear Storytelling in Your Pedagogy,” Hostos Community College Commons. Accessed November 8, 2020. https://commons.hostos.cuny.edu/edtech/2018/02/05/twine-using-non-linear-storytelling-in-your-pedagogy/.

McCall, Jeremiah. 2016. “Twine, Inform and Designing Interactive History Texts.” Play the Past (website). Accessed November 8, 2020. http://www.playthepast.org/?p=5739.

———. 2017. “Historical Twine Project Rubric.” Gaming the Past: Historical Video Games in the Classroom and Beyond (website). Accessed November 8, 2020. https://gamingthepast.net/simulation-design/twine-interactive-fiction-tool/twine-rubric-for-interactive-history-projects/.

———. 2019. “Twine Hist Rubric 9th Grade History 1.5.” Gaming the Past: Historical Video Games in the Classroom and Beyond (website). Accessed November 9, 2020. https://gamingthepast.net/simulation-design/twine-interactive-fiction-tool/twine-rubric-for-interactive-history-projects/.

McGonigal, Jane. 2011. Reality Is Broken: Why Games Are Better and How They Can Change the World. New York: Penguin Random House.

———. 2016. SuperBetter: The Power of Living Gamefully. New York: Penguin Random House.

McGrath, Jim. 2019. “Digital Storytelling.” Spring 2019 syllabus, Brown University, Provincetown. Accessed November 8, 2020. http://www.digitalstorytelling2019.jimmcgrath.us/.

Ohler, Jason. 2006. “The World of Digital Storytelling.” Educational Leadership: 63 (4) 44–47.

Pandey, Asha. 2018. “7 Examples on Scenario-Based Learning for Formal and Informal Learning.” EI Design. Accessed November 16, 2020. https://www.eidesign.net/7-examples-scenario-based-learning-sbl-formal-informal-learning/.

Piedra, Nelson, Janneth Chicaiza, Jorge López, and Edmundo Tovar Caro. 2016. “Integrating OER in the Design of Educational Material: Blended Learning and Linked-Open-Educational-Resources-Data Approach," 2016 IEEE Global Engineering Education Conference (EDUCON), Abu Dhabi, 2016: 1179–1187. https://doi.org/10.1109/EDUCON.2011.5773299.

Queneau, Raymond. 1981. Conte à votre façon. Contes et propos. Paris: Gallimard.

Rizvić, Selma, Dusanka Boskovic, Vensada Okanovic, Sanda Slijvo, and Merima Zukić. 2019. “Interactive Digital Storytelling: Bringing Cultural Heritage in a Classroom.” Journal of Computer Education: 143–166. https://doi.org/10.1007/s40692-018-0128-7.

Salter, Anastasia. 2016. “Playing at Empathy: Representing and Experiencing Emotional Growth through Twine Games.” 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH), Orlando, FL, 2016: 1–8. https://doi.org/10.1109/SeGAH.2016.7586272.

Salter, Anastasia and Scott Moulthrop. 2021. Twining: Critical and Creative Approaches to Hypertext Narratives. Amherst: Amherst College Press. http://dx.doi.org/10.3998/mpub.12255695

Tsan, Katherine. 2020. “The Old Régime and the OER Revolution.” TeachOER (website). Accessed November 8, 2020. http://teachoer.org/the-old-regime-and-the-oer-revolution-game/.

Tytler, Sarah. 2017. “‘TwitFic’, Twine, and Student-Centred Learning: Combining Creativity and Coding in the Classroom.” Africa International Journal of Management Education and Governance: 21–34.

Wilson, Rebecca, Jon Saklofske, and INKE Research team. 2019. “Playful Lenses: Using Twine to Facilitate Open Social Scholarship through Game-Based Inquiry, Research, and Scholarly Communication. KULA: Knowledge Creation, Dissemination, and Preservation Studies 3, no. 1: 5. http://doi.org/10.5334/kula.11.

Wood, David, Jerome S. Bruner, and Gail Ross. 1976. “The Role of Tutoring in Problem Solving.” Journal of Child Psychiatry and Psychology 17, no. 2: 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x.

Zhadko, Olena and Ko, Susan. 2017. “OER Sustainable Scale Up: Faculty Development as Key Strategy (OER).” Presentation at the 16th Annual CUNY IT Conference: Bridging the Gaps, John Jay College of Criminal Justice, New York, NY. https://docs.google.com/presentation/d/1u4JIGhu6b08l_gwaIWH8kpmy9KD10yR9yJE_iTz2Zv8/edit?usp=sharing.

———. 2019. “Refresh your Course with Open Educational Resources.” The Teaching Professor (blog). Accessed November 8, 2020. https://www.teachingprofessor.com/topics/online-learning/refresh-your-course-with-open-educational-resources/.

Acknowledgments

The author/game designer would like to thank Chris Klimas for the insights and background knowledge shared in his interview, Hamad Sindhi and Dimitrios Papadopoulos for sharing useful bibliographical sources, Pamela Thielman for creating the pie charts based on faculty responses in Figures 7–9 and suggesting necessary changes to the game, and Allison Lehr Samuels and the whole staff of the Baruch Center for Teaching and Learning for their invaluable help and support throughout the creative and editorial process.

About the Author

Katherine Foshko Tsan (PhD, MLS) focuses on open educational resources and the digital humanities at the Center for Teaching and Learning (CTL), Baruch College, CUNY. Her scholarly background is in Academic Libraries and Modern European History. She designed The Old Régime and the OER Revolution game for Baruch CTL’s TeachOER website with input from her colleagues and is now completing edits and additions to the resource. She welcomes comments at [email protected].

0

Resisting Surveillance, Practicing/Imagining the End of Grading

Abstract

We write this piece as a collective of activist students, doctoral teaching fellows, and adjuncts from our home campus of the City University of New York (CUNY) because our varying experiences of surveillance are deeply felt, though profoundly contradictory and asymmetrical. We highlight the deeply rooted, white supremacist, classist, and ableist surveillance practices that have long been in place in higher education in general, tracing the history and legacy of those at CUNY in particular. Our article suggests that grading systems in higher education settings are part of a larger network of surveillance technologies that students and faculty are subjected to and/or enact, reflective of schooling’s place in a “carceral continuum” (Shedd) premised on anti-Blackness and colonialism. We do not believe that grading is something that can be made more fair, just, or anti-racist. To resist surveillance in higher education is to embrace the end of grading. After an overview of these contexts and assertions, we offer a series of reflections, tracing juxtaposing moments where we individually or collectively taught, learned, and/or organized outside/against grading systems. Our experiences emerge from a range of contexts—a campus writing center, CUNY’s Macaulay Honors College, pre- and post-COVID college classrooms, an adjunct-led campus grade-strike campaign, counter-institutional learning and mutual aid spaces—and offer spaces of overlap and divergence. Ultimately, we aim to not just critique but suggest entry points toward the unthinking and undoing of surveillance toward a truly anti-carceral, liberatory university.

Authors’ Note

Participation in crafting this article took different forms. Marianne and Anna initiated this collaboration, invited the other contributors and did the bulk of the work tying it all together. Joaly, Jane, Hailey and Andréa through their individual narratives and edits contribute to shaping the argument; their names are listed in alphabetical order. When citing this article, we ask that you write the names of all the co-authors.

Histories and Legacies of Surveillance at CUNY

We write this piece as a collective of activist students, doctoral teaching fellows, and adjuncts from our home campus of the City University of New York (CUNY) because our varying experiences of surveillance are deeply felt, though profoundly contradictory and asymmetrical. We recognize that we are in a moment when there may be more attention to the presence of surveillance in higher education, given the Black Lives Matter movement and more publicized calls for getting cops off campus, as well as the shift to digital educational technologies during the COVID crisis, many of which surveil students, staff, educators, and community members without their knowledge.

Yet, our own feelings of being policed and surveilled in our schools are only in part digital and are far from new. Recognizing that colleges are spatially grounded—not floating institutions disconnected from the dynamics of power, extraction, exploitation, and resistance of the peoples and places that surround them (Meyerhoff 2019; Baldwin 2021)—we locate our experiences in the specific historical and material contexts of CUNY and New York City. Back in 1969, Black and Puerto Rican student activists and radical accomplices occupied City College at CUNY to demand, among other things, that the then largely white university system more accurately reflect the city’s demography by instituting an Open Admissions policy (Okechukwu 2019; Jordan 1969; Ferguson 2012). These activists won this demand, but as more and more Black and brown students gained entry into CUNY’s four year campuses, carceral and militarized tactics—like surveillance, policing, and push out—became more prominent and sophisticated (Gumbs 2014; Glück et al. 2014). This intensified starting in the early 1990s, when CUNY implemented its own centralized, independent security force authorized to arrest and “use physical and deadly force” (Aptekar, Mullin, and Carroll 2021; Kannan 2019), in a broader city and national context that embraced broken windows policing and mass incarceration (Pagan et al. 2020; Hunter Envoy 2013).

When we walk onto our campuses today, we are subjected to technologies of surveillance so routine and normalized that they risk being taken for granted. We walk through metal detectors and/or flash our college ID cards (Laymon 2014); pass countless “peace officers” (and often NYPD officers) on our way to class; notice cameras that might be taping our every move; navigate convoluted, windowless campus hallways that offer little space to congregate; are enclosed in classrooms where professors or administrations can hover and eavesdrop without consent (Kynard 2016). In fact, in the 2010s, the NYPD engaged in widespread surveillance and infiltration of Muslim students associations in the city, including at CUNY. It had deep and damaging consequences: Muslim students at Brooklyn College reported their distress and fear for their safety, as they were forced to practice a kind of self-surveillance, monitoring their own political and scholarly activities on campus, but also their thoughts and aspirations (Theoharis 2016; Brooklyn College Islamic Society and Muslim Women’s Educational Initiative 2020). Importantly, digital campus surveillance has intensified in recent years, especially during the COVID-19 pandemic. In 2020, for instance, the CUNY Board of Trustees opted to continue a contract with TurnItIn, despite much protest from student and faculty activists due to the program’s well-known surveillance practices (Feathers 2020). But in noting all of these examples above, we specifically want to expand the typical notion of technology as solely digital or as relatively new to suggest that there are more deeply rooted tactics of surveillance that have long been in place in higher education. In doing so, we move beyond dominant approaches in the field of surveillance studies that attend more to “practices as they relate to knowledge about people (often distributed via cyberspace or computer database) than to issues of mobility and space” (Perry 2011, 93).

Our understanding of technologies of surveillance in higher education as multifaceted and expansive draws on educational sociologist Carla Shedd’s notion of “carceral continuum,” which contrasts with more traditional conceptions of the “school to prison pipeline.” Understanding disciplinary technologies of prisons and schooling as profoundly linked rather than discrete, Shedd sees “a nexus of institutions and . . . processes that come together” to produce an individual’s “perceptions and experiences with punishment or punitiveness” (Shedd 2020). Indeed, for Erica Meiners (2007), “As the interlocking relationships between schools and the judicial system increased in the 1990s”—in the K–12 sector as in public universities like CUNY—schools have not only increasingly “resemble[ed] prisons and apply the same disciplinary and surveillance technologies, but they also use the same language, ‘pedagogies,’ and philosophies espoused by prisons and jails” (3). Actors along the carceral continuum, then, include not just police officers and security guards, but also educators and administrators who draw on astoundingly similar technologies of sorting, containment, and silencing (Sojoyner 2016).

Grading, Surveillance, and Their Undoing

In particular, we locate grading systems in higher education contexts within a network of surveillance technologies that students and faculty are subjected to and/or enact. Recent publications drawing on the Foucaultdian panopticon and Deleuze’s notion of “control society” have theorized grades as a technology of surveillance (Johnson 2021; Nemorin 2017; Nieminen 2020). In relation to surveillance, grades are both product—assigned by teachers based on “a complex network of observations and choices”—and process, as “various stakeholders surveil students, teachers, and their respective institutions through grades and grading trends” (Johnson 2021, 56). We too view grading systems as indissociable from higher education’s broader practices of extraction and social control, with a reach beyond the space/time of one course. Grades that land on report cards become markers of worthiness and the basis for the attribution of degree, grant, scholarship, exclusion, punishment, remediation.

At the same time, we explicitly center the inherently racialized and colonial nature of surveillance. In Dark Matters: On the Surveillance of Blackness, Simone Browne (2015) theorizes surveillance through African enslavement and the slave ship, beyond Foucault’s race-evasive, totalizing notion of panopticon. Browne understands surveillance not as “something inaugurated by new technologies,” but “as ongoing”; to do so “insist[s] that we factor in how racism and anti-Blackness undergird and sustain the intersecting of surveillances of our present order” (2015, 8). Indeed, the history and legacy of higher education is inextricable from the transatlantic slave trade (Wilder 2014), indigenous land theft (la paperson 2014; Grande 2018), and research practices such as eugenics premised on the dehumanization of Black, Indigenous, and/or disabled people (Dolmage 2017). These racist, colonial, and ableist foundations in turn inform violent pedagogical practices that often reify, not disrupt, race-class sorting (Kynard 2016)—like grading.

The growing critical literature on grading tells us that grades reflect graders’ biases, too often replicating normative definitions of “good” writing, participation, or comportment contingent on white, cis, straight, middle/upper class, non-disabled, English monolingual ideals (Kynard 2008). As Sojoyner (2016) notes, grading measures “compliance” more than anything else (176), whether “critical thinking” skills, writing ability, or even effort (181–82). In this sense, it is possible to see how the succession of grades (across countless classes and semesters) builds up into a surveillance apparatus that records and cumulatively gauges students’ displayed adherence to fundamentally racist, classist, xenophobic, ableist, sexist, and queerphobic norms.

Our analysis of grading at CUNY, then, explicitly attends to these dynamics, aiming to depart from universalizing, decontextualized approaches to non-traditional grading. For instance, in tracing the longer history and context of non-letter grading, studies often cite liberal arts colleges like Antioch, Hampshire, and St. Johns as examples (Blum 4). Left unsaid is that such colleges are also wealthy, predominantly white, private, and very expensive. (A rare public-school example, Evergreen State, has been undergoing rapid defunding.) In contrast, on CUNY’s predominantly Black and brown campuses, administrations have rarely embraced non-traditional grading practices. Meanwhile, prominent texts around equitable and/or anti-racist assessment often focus on contract/labor-based grading (Elbow 1997; Inoue 2019)—a practice that, we argue, still falls prey to racial-capitalist logics of “contracting.” The idea of student-teacher contracts as a way out of punitive grading practices becomes particularly fraught in light of activists’ work tracing the university’s imbrication in the debt economy (Schirmer et al. 2021), within a broader landscape of public higher education’s racialized austerity.[1] The Debt Collective alerts us to the uncanny resemblance between credit reports (A+/A/A-) and students’ transcripts (A+/A/A-). Schooling socializes students as debtors, who owe work and time to their teachers-creditors (Wozniak 2021). We thus understand labor-based contract models—in which students self-monitor their productivity and owe teachers these records of their labor—as still inevitably reproducing the by-default racist, classist, gendered, and ableist order of the classroom. This contractual relationship disproportionately penalizes sick and disabled and/or working-class students—who have to balance their schoolwork with full-time jobs, parenting/caretaking, managing their own health, and other responsibilities. We miss such complexities through race/class/gender/disability-evasive frameworks of schooling and assessment.

This is also why we hesitate to use the fraught categories of “academic freedom” and/or “rights to privacy” in our critique of surveillance—ideals that have never existed in earnest for many students and staff, given the white supremacist, colonial foundations of US higher education. “Academic freedom” in particular has been leveraged to repress and surveil Black, brown, and/or indigenous, activist scholars (Salaita 2014) and to sponsor research practices premised on racialized surveillance, including in education scholarship (Kynard 2016). Likewise, “the right to privacy” is fundamentally contingent; as Perry (2011) suggests, our tendency to “talk about privacy in bourgeois terms” masks the “racialized practices of surveillance,” most violently targeting Black people, “justified through racial narratives about social disorder, invasion, and moral decay” (86). In this sense, surveillance is material and embodied (Browne 2015); so too are its manifestations in pedagogical practices like grading. Following Perry, we understand surveillance—in this case, as it manifests in the classroom—as more than just stealing and/or storing information about people, and their capacity to comply with regulations and rules. Grading, too, is spatial and material in its power to restrict the minds, bodies, and affect of students, faculty and staff, regulating how we control our bodies, move across space, gather in collective.

Given these dynamics, we do not believe that grading is something that can be made more fair, just, or anti-racist. We see such imagined “remedies” for grading as too often “get[ting] caught in the logic of the [carceral] system itself” (Gilmore 2007, 242). We instead move toward what the abolitionist scholar-activist Ruth Wilson Gilmore (2007) calls “non-reformist reforms”: “changes that, at the end of the day, unravel rather than widen the net of social control through criminalization” (242). With grading as a starting point, our discussions below aim to not just critique but suggest entry points to unthink and undo surveillance toward a truly liberatory university (Reed 2020).

Process, Positionality, and Our Narratives

We offer a series of reflections, juxtaposing moments where we individually or collectively taught and/or learned outside/against grading systems. Writing journal articles won’t undo surveillance. Through narrative, we instead hope to center on-the-ground, lived experiences, as we believe students and faculty already know what we have to share. We foreground our identities as well as our positions/engagements with CUNY, but, importantly, our experiences offer spaces of overlap and divergence. As opposed to a more unidirectional format in which faculty analyze or even appropriate their students’ experiences to theorize about education and learning, we attempt a horizontal approach, where the authors co-shape the argument/story. That said, we don’t overlook the differences among us. The contributors who teach as doctoral student fellows and/or adjuncts—Andréa, Anna, Jane and Marianne—are all white, which is also indicative of the fact that unlike nearly every other campus in the system, The Graduate Center, CUNY, is predominantly white. We recognize the complexity of speaking as teachers, as many faculty push back and strive to make grading less ableist and racist, while others embody white saviorism as they teach mostly Black and brown students. On the other hand, Hailey (also an organizer) who identifies as an East and Southeast Asian USian and Joaly, who identifies as Afro-Caribbean, Afro-Latina(x), and Dominican-American, write as students who have been subjected to grading.

Taken together, these narratives highlight how grades/grading systems are part of a surveillance apparatus deployed in universities to keep students and faculty on check. Anna discusses how her work as a writing tutor, which did not require giving grades, shaped her understanding of students’ thoughts, ideas, and frustrations. Hailey critiques CUNY’s Honors program, which rewards white and privileged students under the guise of grade-based meritocracy, and describes her coalitional work carving out spaces of counter-institutional learning and mutual aid. Jane writes as an adjunct and an organizer about the choice to give all their students As—initially as a response to the early days of the pandemic, and now as a way to reimagine their relationships with students. Joaly writes about grading systems forcing students to perform, apologize and conform. Andréa writes about the fears around giving As systematically and unconditionally to all students: “what will my WPA [writing program administrator] say?” How do precarious employment situations, being financially strapped, caring for children, disabilities impact how adjuncts use grades? Pursuing this reflection, Marianne writes about an adjunct-led collective experiment of refusal of grading and what it revealed about surveillance when a Spring 2020 pledge to withhold grades circulated at CUNY.

Our Narratives

“answer the prompt unambiguously” (Anna)

For the past few years, I’ve been working as a “non-teaching adjunct” at Baruch College’s Writing Center. When the pandemic hit, that was my only teaching job (I’d had hopes to teach something new but with 2,000 adjunct jobs cut prior to the fall, that was no longer an option.) Truthfully, though, I found the work of my “non-teaching” position—which, despite its name, is just as pedagogical as any other adjunct gig—to be somewhat less stressful than classroom-based teaching. Classroom teaching is my passion, but while finishing my own coursework and now my dissertation, having a job with a discrete number of hours and that didn’t leave campus with me felt more feasible. I realize now that the task that I found most stressful about teaching in a more traditional setting was assigning grades—which I found both anxiety-inducing and immensely time consuming—not to be confused with responding to students’ writing or giving them suggestions and connections.

I was especially grateful for the chance to work with students at the Writing Center in spring 2020. In a moment in which everything was uprooted, everyone was dispersed across the city, and a foreboding sense of uncertainty, precarity, and fear followed every CUNY student I knew (to greater and lesser degrees), I appreciated that I could work individually with people at the Writing Center. Even on Zoom, it offered a degree of intimacy and connection with those on campus that had otherwise completely disappeared, for both me and the students.

Through these conversations with Baruch students, I got the sense that many of their professors were quite rigid in their grading, noting on their assignment sheets that they, for instance, never allowed extensions; automatically took off a certain number of points for incorrectly citing something in MLA (even when MLA format was never taught in class itself); vaguely noted that they would penalize for not adopting the correct “professional tone” while writing memos (this was, after all, a business school, but these instructions were often in place as early as first-year writing classes.) Not all professors espoused this kind of pedagogical approaches to writing and assessment, but this sort of rigid, hyper-disciplinary thinking pervades every space on campus, including pedagogical ones.

As one example, recently I was working with an undergrad on a project for a class in the business school; they’d chosen to write about Dubai. It was for an assignment that asked students to do something like evaluate strategies of economic development in a particular region, giving pros and cons. The student argued that Dubai exemplified thoroughly successful economic planning; though its development resulted in displacement, it was ultimately a net positive. I’m no expert on the region, but I do know that Dubai is pretty famous for its devastating inequality. So I asked them: how are you thinking through this? Can we complicate this “net positive” thinking, as it wasn’t a net positive for many in the area? The student completely understood my point, demonstrating in conversation a sharp and in-depth analysis of these complexities, but was wary to bring this into their writing. They were anxious about receiving a good grade on the assignment and so they felt they really needed to embrace a pro-business mindset and answer the prompt unambiguously.

This type of black-and-white analysis was not something overtly required on the paper’s assignment sheet; although this attitude probably stemmed from their professor, it was bigger than that too. This pressure to think and write in these ways pervades the institution’s ethos. This for-profit, all-or-nothing approach to thinking, I think, was why I felt an astounding amount of strain on students to continue to perform at a “standard” level and less leniency with grades than I expected from the part of professors. And this ethos seemed to persist even under COVID: in the semesters after March 2020, I met with an unprecedented number of students on academic probation for failing courses, seeking support drafting appeals against their expulsion (often by having to narrate their own trauma to be read by administrators they’d never met).

But because I was not the one assigning grades for this student, I was able to have this kind of nuanced conversation with the student around form and capitalism that their professor could not. Even in my own teaching, where I explicitly give students a great deal of flexibility to try new and creative things in their writing, it takes time for some of them to feel comfortable submitting assignments that don’t conform to a rigid structure akin to an ideal SAT essay (a structure that high school and even college professors praise them for), because they do not trust that I will not penalize them for it. And why would they? In my own experience as a student, even professors who claim flexible or radical pedagogical approaches, or teach activist texts and content can still fall prey to enacting typical violent modes of student discipline and surveillance.

In a sense, grading—within a network of other carceral pedagogical technologies—surveils student capitalist productivity: whether a student is modeling a properly respectable comportment, linguistic repertoire, and attitude that will supposedly bolster their success once they become full capitalist subjects. In the case of Baruch, grading is in part used to assess which students will continue to uphold Baruch’s remarkable “social mobility index” (currently first in the country, a focal point of Baruch’s advertising and branding), and which would fall short—even if the cause was a family member’s illness or death, managing several jobs as their household’s breadwinner, or navigating a global pandemic—and thus lose their place at the college.

Baruch offers a stark example of how logic of capitalist productivity infuses into grading practices, but this kind of ethos is widely pervasive in higher education. I felt it myself as a stressed-out, neuroatypical early undergraduate student more anxious about following the correct format of an assignment than exploring new ideas. But aside from isolated moments when I experienced ableism (especially before I got formalized accommodations my sophomore year), I rarely felt surveilled, per se, on a day-to-day basis because as a white student at the progressive, wealthy, predominantly white private school I attended, my grade-based anxieties did not have the same structural, deeply racialized resonances as they do for many CUNY students. Interestingly, it is campuses like my former one that are most well-known for embracing non-traditional grading structures (e.g. Hampshire College, Reed College). That the CUNY system—a primarily Black and brown, working-class institution—implicitly sanctions punitive grading practices is no coincidence: its students are also more frequently subjected to punitive modes of surveillance and policing “outside” the classroom as well.

Unsurprisingly, then, I didn’t encounter any students while teaching at Baruch who were taking courses utilizing non-punitive or anti-oppressive modes of grading. In some ways, though, the best aspects of writing center work might offer guidance of what a different mode of being in intellectual community with students could look like: one that is not bound up in a contractual relationship premised on surveillance, but instead is built around dynamic conversation, exploration, curiosity, collaboration.

“to cultivate, to grow, to study, and to imagine other ways-of-being” (Hailey)

From 2016 to 2020, I attended Brooklyn College as part of the Macaulay Honors program––a program that “awards” its members with free tuition, a free laptop, academic advisors, and various other resources. The program, however, primarily determines access to these substantial resources through a “meritocratic” grade-point system that, in practice, has historically privileged already well-served, funded school districts. From its creation in 2001 until 2018, the Macaulay Honors College did not even accept students transferring from community college, highlighting some of the strategic ways that CUNY regulates and maintains internal inequity favoring a select few.[2] Grading as a surveillance apparatus does not merely begin and end within one’s college career, but extends outward, punishing and excluding certain high school districts, students, etc. well before they can even set foot on the campus. White students and—in certain instances often determined by race, gender, class, and immigration status—non-Black students of color have continued to benefit primarily from programs like Macaualay. Responding to this inequity, CUNY for Abolition and Safety, Macaulay Diversity Initiative, and Macaulay Peace Action (2020) wrote in an open letter to the Macaulay administration: “With a CUNY population greater than 75% POC, why is our university wide college honors program 50% white ([as of] 2018)?”

While the program primarily serves white students and non-Black students of color, the material privileges are also felt in the kind of education being afforded compared to their so-called peers at CUNY. In several Macaulay seminars at Brooklyn College aimed at “creating opportunities to learn using New York City’s unique intellectual, professional and cultural capital” (“New York City Advantages” n.d.), professors would assign projects that had students photographing, interviewing, and studying Brooklyn residents without their consent. It is no wonder, then, that an institutionally designated student space––both physically (as an exclusive lounge for honors students) and intellectually (specialized curricula)––in which access is determined by grading rooted in anti-Black, sexist, classist, ableist, and queerphobic terms––would support students and faculty who most conform to ideals of exceptionalism, elitism, and individualism.[3]

In contradistinction to the institutional spaces of learning, relaxing, and socializing are spaces on campus, both forged, cared for, and tended to by students, workers, faculty, and various other members of the CUNY community. These spaces are never permanent as the University constantly lays siege to them via defunding, school police, surveillance, and more. The Brooklyn College Student Union, Free CUNY!, and the Kingsborough Community College Urban Farm were all groups and spaces I was a part of that practiced a kind of learning that was full of care. Study in these spaces is not motivated by an individualistic desire to achieve high grades but out of a shared commitment to each other and our communities. Through shared study we are able to collectively create zines and teach-ins on austerity, school policing, CUNY movement history, and more.[4]

The Kingsborough Community College urban farm, a space which itself was hyper-policed and surveilled, often utilized post-work conversations to discuss the conditions of the farm as a space of work and study under an anti-Black administration. Even whilst providing and tending to resources meant to counter the inherently racist, sexist, and violently policed neighborhoods where food apartheid required a more attentive and thoughtful gardening practice, the urban farm at Kingsborough had to carry out its work whilst also defending and preserving the farm space from CUNY policing and surveillance practices at every turn. Educators on the farm would intentionally and carefully grow vegetables like okra, callaloo, collards, and more that were not commonly or typically consumed in white East Coast cultures and their dietary preferences. The food growing practices on the Kingsborough farm were intentional, strategic, and sought to undermine a wider city struggle inflected and informed by the policing practices that continue to shape neighborhoods, communities, and their borders that are constantly being re-configured to align and oblige white tastes, desires, and ways of life. Yet, those intentional and careful practices were also consistently undermined from within, unable to grow and cultivate a space where policing could not pervasively extend. Today, the farm workers have not been re-hired, the farm budget cut and barren, and the community college is currently advertising for “volunteers” to work on the farm for free, to carry out an agricultural politic and practice that does not serve the interests of the people—but the interests of the university. Regardless, there remains now more than ever a need for spaces like the urban farm to cultivate, to grow, to study, and to imagine other ways of being beyond the terms by which the university determines the most feasible, productive, or worthwhile.

“a pressure-free environment” (Jane)

One of the first things I noticed when I started teaching as an adjunct in the fall of 2018 was the power imbalance in the classroom. It’s an uncomfortable feeling—especially since I had gotten my bachelor’s degree from the same institution in the summer of 2017, so the students in my classroom had been my peers just a year earlier. As professors—even as adjuncts—we are generally able to set the rules, expectations, norms, and culture of our classes, and to be as strict or as lenient as we want with grading. That means we hold tremendous power over students’ lives, material conditions, and futures, whether we want to or not.

In March 2020, when the pandemic was at its peak in New York City and classes had been abruptly shifted online amid much anxiety and confusion, a group of undergraduate and graduate students and adjuncts from the activist group Free CUNY—which works for an anti-racist, tuition-free public university—met to discuss strategies for challenging strict grading and surveillance practices. An undergraduate student, sarah g, came up with a name for the most widely embraced proposal: “A for All.” An article (Goldberg et al. 2020) and an FAQ (2020) reinforced the campaign.

Grading is the primary mechanism through which our power is enforced, so when I threw out my rigid points-based grading system and embraced A for All, I thought it would take the edge off of the power imbalance I felt as a faculty member. I was surprised to be wrong about that, and to realize that I still held the same power in the classroom, because many students don’t trust that I am serious about A for All. In fact, they are right to be wary, because I could still change the grading scheme midway through the semester, even though it’s written into the syllabus. I know I would never do this, but how do they know that?

Still, I found A for All to be a big improvement over traditional grading. There are different ways to approach A for All. The way I use it, I give points for class participation, and students who earn enough points get an A+ grade, which at Queens College still counts as a 4.0 for GPA purposes. I also track attendance, not to punish students who miss class but to make sure I follow up with them to offer extra support. It’s still surveillance, but hopefully it loses some of its power when there are no real-world punitive consequences.

For students, A for All releases them from stress and anxiety and opens them up to learning and engaging creatively on their own terms. After the end of the spring 2021 semester I asked students to email me their impressions of A for All grading. Their comments emphasized the “pressure free environment” (all quotes used with permission):

    • A for All provides a pressure free environment allowing students to worry less and therefore can focus better even more on schoolwork.

I think the A for All policy takes a lot of pressure off. I didn’t think oh I’m gonna get an A I don’t need to try. I actually thought about my answers and did the readings not just look for answers to make sure I would get a good grade it made it less nerve-racking. I thought oh I’m never gonna get back anything back from this Professor because we are all getting As but it wasn’t like that. I got work back in a timely manner and I wasn’t focused on the grade so I really analyzed the feedback and I think that was new for me and really helps.

Honestly one thing I really think which was completely different in this class from all other classes is that when you know that you don’t have to study just for the sake of grade and you can actually use your energy to just learn new things in the lectures, the pressure less environment allows us to be a lot more creative.

A for All was also good for me. The pandemic chaos of the spring 2020 semester was followed by a fall semester of family crisis for me, just as I was teaching a new course for the first time and using a new online platform. A for All grading became a way to not only show grace to students—many of whom were struggling with similar family crises, unforgiving job schedules, and the strains of online learning—but to also extend that grace to myself. A for All gave me confidence that my personal crisis and any constraints it put on my teaching would at least have no impact on financial aid, degree completion, or future opportunities for students.

For these reasons, I especially encourage adjuncts to use A for All as a way to simultaneously resist the pressures that the system places on us and to act in solidarity with students. Practicing A for All also helps to avoid negative student evaluations and conflicts over harsh grading that can be used as pretexts to “non-reappoint” contingent faculty. And with A for All, students have no reason to cheat or to make up excuses for missing class, so as contingent faculty we can let go of our anxieties around that.

Despite the many advantages of A for All, it seems like adjuncts often resist it more than full-time faculty do. Are we using our power over students to compensate for our low pay and undignified treatment by the administration (and sometimes by our full-time colleagues)? Could it be that adjuncts feel a need to show that we are just as academically and pedagogically qualified as full-time faculty, and harsh grading becomes a weapon to prove our rigor, or a way to be taken seriously by students and colleagues? Given the white supremacist and patriarchal nature of the academy, BIPOC faculty, especially women, are more likely to face challenges to their authority by peers and students who question their professionalism and competence (Gutiérrez y Muhs et al. 2012)—as well as increased institutional surveillance (particularly in the case of Black women faculty members; see Kynard 2016)—and thus may face extra barriers in adopting counter-institutional pedagogies. As a rank-and-file activist in the faculty and staff union, I am often unsure of the best way to address these concerns with colleagues. If a faculty member is using overly harsh grading practices or otherwise acting against student interests, can we convince them of the importance of solidarity with students as we defend their rights as workers to teach how they want? How do we step in when student rights and faculty rights collide?

Mostly I try to use my own experience as a model and (hopefully) as inspiration for colleagues interested in trying A for All grading. For the spring 2021 semester, I was finally able to develop a more deliberate and organized approach to teaching within the A for All framework. With no need to stress out about deadlines or grades, I can consistently respond to student work with individualized comments, often in the form of questions, designed to encourage critical reflection and draw students into a conversation. When students don’t do the work, I reach out and ask if there is anything I can do to support them in getting back on track. This kind of individualized attention takes time, and I am able to do it because I am teaching only one class with 25–30 students and do not currently have to hustle multiple jobs to survive.

I have also been fortunate to feel supported in my decision to use A for All. I am completely open about my grading framework. I describe it in my syllabus and have even managed to post it on the course registration system. In an online teaching workshop, the instructor gave me positive feedback on the A for All description in my draft syllabus. At least one student review on the ratemyprofessors.com website references my A for All grading, and a student evaluation mentioned it in a comment. No one has raised objections. I do not plan on going back to a regular grading system. A for All is here to stay.

“to be open and vulnerable to get sympathy from professors” (Joaly)

During the Spring 2020 semester, as the pandemic crept in, colleges all over the world uprooted and changed to an online, distanced learning. I exhaled deeply when one of my professors (Jane) announced that everyone in the class would be receiving an A in the course. This was the first time I even heard that this was possible. But I could not fully relax, as only one of five classes that I was taking that semester would be doing this.

In Fall 2020, none of my professors came up with an innovative grading system. In our return to normalcy, the looming 11:59pm deadlines made their return. I remember one night coming home from my job as an “essential worker” and sitting down to do an assignment. Juggling 18 credits in one term and working a full-time job meant that I was always cutting it close to deadlines. I had two options: (1) turn in work that I was not happy with, but that would be on time or (2) turn in the assignment late, but that would at least be something good, something I would be proud to submit. I decided to turn it in late, 66 minutes late to be exact.

The next morning I woke up to an email: “This is late. It was due by 11:59PM. This was turned in at 1:06AM. Future assignments submitted late will not be accepted.” A short email, but one that made my heart sink. I remember reading the email heartbroken, defeated. More than ever I realized my mistakes were on full display. My professors could see me accessing the website in the middle of the night, the only time I could get work done some days. They could see me turning in work at odd hours. I told myself, “if it’s a minute late, don’t even bother; they could see the timestamp.” This type of surveillance not only made me feel anxious, but also made me wonder how this data could impact my grades. I found myself wanting to explain and apologize to professors for emailing or submitting late at night. I had to be open and vulnerable to get sympathy from professors. Some were understanding, offering extensions and reassurance, while others expressed sympathy but not any tangible support or leniency. It reminded me that at its core, higher education centered grades and productivity, not students and learning. My experience was that professors do not trust students to self-manage or have discipline, therefore we needed strict deadlines, tests, rubrics, and filler assignments to prove we were engaged. Even if it meant that the quality of our assignments were reduced and that authentic learning was not happening.

Even before the COVID-19 pandemic, many of my classes felt like a performance. I had mastered the art of skimming through scholarly text to be able to regurgitate just enough information to write a response paper or to score participation points for the day. I found myself giving hollow responses to my classmates on discussion boards or writing assignments and curating work that I thought my professors would like. For a long time, this hindered my voice as a student. I didn’t have room to be creative, to make mistakes, to experiment, or to challenge myself academically. Every class and assignment felt make-or-break. Experimenting academically was a risk—a risk that if not accepted by the professor meant that I could receive a poor grade in the course, potentially affecting my academic standing and financial aid. At this point, academia was about producing, not about learning or understanding. Universal grading systems imply that every student is the same. That every student has the same amount of time, resources, or interests. Rigid grading systems reinforce unhealthy classroom power dynamics and gender biases, and punish students, especially students of color, for not conforming to oppressive academic standards. It leaves little space for creativity, mistakes, and healthy student-teacher relationships that are fostered on trust and mentorship and not on power and punishment.

It wasn’t until I became an Urban Studies student, a branch of the School of Social Science at Queens College, that I was able to take courses with professors who promoted my individuality as a student and allowed me to take positive risks in my writing and research. Through addressing rigid grading policies and academic policing, we’ll be taking great leaps in making higher education more accessible and valuable to students.

“With one search result he could see the average grades in my classroom” (Andréa)

5/19/21

My heart is beating out of my chest. 28 A+s. What happens if my Writing Program Administrator, who has already shown that her politics don’t align with mine, calls me out? Asks me for data to support this decision? Asks me for student work, proof that the students deserve these grades? My stomach is in a knot.

“Your grades have been posted successfully”

What if the compliance people come after me? Who even sees how grades are entered? I don’t know the process. It’s not transparent.

Fluttering my lips holy shit this is a major act of defiance against the institution that pays me to research what I want. They pay me, but they don’t pay me enough, it’s basically an honorarium so they can keep filling classrooms and taking tuition from students.

As always, I’m ready to fight, like Bri said, I feel the tension in my body.

I’m submitting grades the second day after the semester ends. This is a calculated decision, a choice not to submit on the first day because it could look suspicious to whoever is watching. But if someone’s watching should I have pushed it to the end of the grading period to seem less suspect?

————
I started giving As to every student in 2016 after my first semester teaching (so this now marks five years and nine semesters later) because subjective grading quickly made no sense— who am I to wield this power over students especially when my disabilities and pregnancies started affecting my capacity to show up bi-weekly and stick to a strict schedule?

As an adjunct, even with my disabilities, I still have class privilege and whiteness so I’ve felt “safer” to go against the grading criteria set out by the institution. But it never feels completely safe. I was close with the first WPA I worked with; when we were trying to build a case for my hybrid course (pre-COVID), he compiled grades from across the First Year Writing courses to see if there was a significant difference between students who attended in person and online. What stuck with me was the fact that a data grab of grades by the WPA was possible. With one search result he could see the average grades in my classroom.

So now I’m nervous and defiant. I turn in different syllabi to the department than the ones I teach. My syllabi are surveilled. Even sharing this in an article feels dangerous—co-optable by the institution at best, grounds expulsion/firing at worst. The meta onion peel layers of academic surveillance. I use citations in my syllabus to ground my decision in the event someone questions me. Feeling increasingly paranoid about the consequences of giving all of my students an A, I reached out to Anna to see if she had any citable sources on resisting/abolishing/getting rid of grades. It was serendipitous when she shared the collective piece A for All because I felt both connected to other CUNY adjuncts and had something concrete to hand to an administrator if I were called out. I cited “A for All” (Goldberg et al. 2020) in my grading policy section because I believe what it outlines and so I have a framework to point to.
This is the first semester I gave all A+s. The students asked for them, wanted to do extra for A+s and I said no, you deserve them regardless. And then we realized A+s can be reparative on a GPA making it the only acceptable grade.

“as you normally would” (Marianne)

In May 2020, faculty at CUNY collectively attempted to disrupt final grade submission. Until that point, uploading my grades at the end of each term had seemed both inevitable and dull. My well-intentioned pedagogical attempts to decenter grading over the course of the semester seemed always limited by the fact that eventually I still had to submit final grades. It was something I did on my own; it marked the end of my teaching work. The grade withholding campaign taught me there was so much more at stake. It helped me see final grades as yet another mechanism that keeps students, staff, and faculty in check, and thus a potential pressure point to disrupt the administration’s austerity plans.

The word was out that CUNY was about to lay off hundreds, maybe thousands, of adjuncts. The City, Hunter, Staten Island, and Queens College administrations announced drastic budget cuts under the pretext of an anticipated enrollment drop combined with decreased state funding. Brooklyn College’s president asked all department chairs to prepare for 25% cuts in budget and identify which courses to cancel by May 5 (Sandoval 2020). At John Jay College, the provost told faculty to prepare for $21 to $55 million in cuts and to lay off 40% of the faculty (Pereira 2020). Both the scale (millions of dollars and thousands of adjuncts) and the narrative justifying these drastic cuts (“we are going through a crisis”) was reminiscent of the 1970s. This orchestrated defunding of an institution that primarily serves Black and brown working class students, therefore, was not unprecedented.

On May 1st, backed into a corner, 120 students, faculty, and staff participated in a town hall hosted by Rank and File Action (RAFA, a collective of union members), and the idea of a collective withholding of grades emerged (Rank and File Action 2020b). Rank-and-file union members then drafted a resolution calling on all CUNY faculty to withhold grades. Unsurprisingly, union leadership condemned the initiative, and voted it down during a May 11 special Delegate Assembly (PSC-CUNY 2020). But especially in light of the recent Santa Cruz graduate students’ strike—the idea of organizing a wildcat campaign to withhold grades gained steam. This possibility proved disruptive enough to worry college admins. The day following the Delegate Assembly, in a move that seemed intended to undermine a possible grade strike, CUNY postponed sending adjuncts reappointment letters. Instead of May 15—the date initially agreed on during contract negotiations—adjuncts would now have to wait until May 29 to know whether they would be reappointed (Bowen 2020b). This seemingly innocuous calendar modification was perverse in its effects. Grades were due on May 28. This sudden delay meant that adjuncts would learn about their employment status for the Fall, only after the submission date for the grades. The same day, faculty at Brooklyn College received an email from the Provost outlining step-by-step instructions to submit grades (Lopes and O’Reilly 2020), detailing the credit/non credit grading policy,[5] and reminding faculty that grades were due “no later than” the 28th.

The denunciation and pressure from both the union leadership and college administration made me feel upset and at times demoralized. But as some comrades pointed out, they also signaled that something about this tactic was potentially impactful.

On May 15, RAFA hosted another town hall, and soon after sent faculty an “urgent call to action to defend jobs at CUNY”:

we are organizing a university-wide grade strike to tell the administration that we will not accept one single layoff, furlough, or non-reappointment of any faculty or staff . . . To make such a strike successful we will need the support of the vast majority of the CUNY faculty, and so [RAFA] has created . . . [a] pledge [tiny.cc/s20cunygradeaction] ask[ing]every teaching faculty member at CUNY [to] withhold grades until at least May 28, the last day to submit grades for many classes… If we reach the minimum threshold of 70 percent of the faculty, we pledge to withhold grades further until management agrees to negotiate for the reinstatement of all faculty and staff, or as long as is needed. (Rank and File Action 2020c)

Over the following days, rank-and-file activists engaged in discussion with students and colleagues at the registrar and bursar offices. We talked about grades a lot, but not like we do in pedagogy-oriented discussions that usually focus on learning. We sought to learn more about what grades do after we turn them in. We asked how grades dictate student loan repayment and visa status, and limit or allow access to courses, majors, scholarships, and services like advisement or transfer planning. This moment laid bare for us that grades are instrumental to the system of selection, ranking, and exclusion on campus.[6]

The pledge circulated, and faculty signed on. There was even outside media coverage (Dunn 2020; Hoff 2020). On May 21, Brooklyn College’s union chapter adopted a resolution calling for all faculty to hold on to their grades until the very last day (Davis 2020). A few days later RAFA (2020d) sent a list of demands to all College Presidents, Chancellor Felix V. Matos Rodriguez, Mayor Bill de Blasio, and Governor Andrew Cuomo: “Enough is Enough: We Demand the Reinstatement of all Faculty and the Repeal of any Future Tuition Hikes or Student Fee Increases.” The action was gaining momentum: it received support from the Freedom Socialist Party (Rank and File Action 2020f) and Graduate Workers of Columbia (Rank and File Action 2020e), RAFA released an FAQ about grade withholding (2020a), and students created a solidarity pledge to support the action (Free CUNY! 2020).

As we continued to gather signatures, college administrators and the union leadership doubled down on scare tactics to undermine the collective efforts around the grade withholding. On May 26, the union president sent an email to “update [members] on the fast-developing union actions this week,” urging them to denounce the grading action: “the PSC opposes any call for faculty to submit their spring semester grades after the deadlines set by the colleges. Any request you may receive to submit grades after they are due has not been authorized by the union. The PSC recommends that all part-time and full-time faculty submit their grades by the established date, as you normally would” (Bowen 2020a). This email’s explicit call to proceed “normally” felt out of tune, as there was little sense of normalcy in a moment where thousands of adjuncts were about to be laid off. But beyond this, it helped me consider grade submission under a different light. This end-of-the-semester ritual I used to treat as a mundane, bureaucratic task turned out to be key to the surveillance of faculty, students and staff. Every day of that final week, I received an email from the Registrar with a personalized message and an attached “grade submission report”’ informing me of the number of days left to submit. These emails put us on notice: CUNYfirst is watching you. However, I do not want to give the false impression that we were up against some kind of high-tech big brother apparatus. Surveillance was not an add-on technological layer the admin deployed at the eleventh hour. It was there from the start; it was embedded in the very practice of the grade submission. CUNY is not a well-oiled machine enacting NSA-like surveillance. Surveillance is operationalized through institutional procedures of ranking and documentation like grade submission.

A few hours after the union president’s message, the Brooklyn College Registrar invited all teaching faculty to virtual office hours “to assist faculty members who are having difficulties submitting grades” (2020). The same day, the Provost sent a warning to all faculty: “Now that the spring semester is over, you must submit your grades on or before this Thursday, May 28. This year, more than ever, students are anxious about grades. Not turning in grades on time harms students’ financial aid and their ability to register for the courses they need. Additionally, students cannot graduate or complete applications for graduate school without their grades. Please submit your grades as soon as possible” (Lopes 2020). It’s pretty hard to know the extent to which faculty bought into the guilt-inducing messaging. But in their efforts to pit students against faculty, these emails helped me understand grade submission as a mechanism to divide faculty from students: the administration relies on the submission of grades to keep everyone under control.

How do faculty act in solidarity with students when their relations are mediated by the looming assignment of final grades, which hovers over all our classroom interactions? How do we subvert and eventually get rid of grades, not only through our classroom practices throughout the semester, but also at the end of it? Can we imagine different end points to our courses—outdoor gatherings, online celebrations, honoring collective growth?

In ten days, about 800 faculty had signed onto the pledge—much less than the 70% needed to begin an actual grade strike and withhold grades past the submission deadline, but nonetheless a remarkable number. Collectives like Queens College Adjuncts Unite and #CutCovidNotCUNY invited faculty to participate in last-minute grade submission parties on Zoom, attempting to turn an anti-climactic moment into a celebration of our collective power, a demonstration of what might have been. In the end, there were too few of us, our demands were not met, and, ultimately, about 3,000 adjunct faculty and staff were laid off. Yet the campaign showed that what seemed inevitable could in fact be challenged.

Conclusion

In seeing our narratives together, we are struck by just how rigid typical modes of assessment and learning feel. Drawing on the work of abolitionist writer-organizer Mariame Kaba (2021), we recognize that unthinking these processes might feel like an impossibility, just as with other carceral technologies—policing, prisons. We have never known a school without grading. To create one, we must build on already-existing practices of teaching, learning and organizing that center student and faculty liberation, as it is cultivated through everyday pedagogies, actions, and refusals (Grande 2018). The stories we tell here are only examples of the multitude of ways students and teachers fight grade-based surveillance: through uplifting counter-institutional modes of thinking and creating in classrooms; collectively interrogating the racist and colonial underpinnings of higher education and possibilities of resistance; sharing information about which courses and campus spaces offer refuge and resources for dissent and those to avoid; and centering the material needs of the staff, students, precarious faculty, and of the community members off campus. In these ways, advocating for the end of grading forces us to confront a wide range of questions, not only about learning, but also about the function colleges play for our communities. We are pushed to reimagine the purpose of schooling as not simply a means of race-class sorting, but a vehicle for creativity and social change.

The day-to-day pedagogies and actions of comrades at CUNY and beyond inspire us to move forward in this work. We are emboldened to publicly advocate for grading practices that have actively been denounced by the institution because of the longer and continued genealogy of organizing at CUNY and beyond, knowing that we are part of a larger community and legacy of counter-institutional organizing, even when the stakes were literally life or death. We think of the many other radical adjuncts, staff, and students who have organized before us and into the present—from the Black and Puerto Rican student activists at the front lines of the Open Admissions struggle, to the numerous modes of organizing that emerged to combat racialized dispossession at CUNY in light of the COVID-19 pandemic.

As students and/or contingent faculty, we know that writing this article may make us vulnerable to scrutiny from campus authorities. We note this not to artificially promote our work’s value or evoke pity, but to emphasize that these topics have real-life, material implications for us; because our precarity tells us that this institution is not made for us, and motivates us to demand not small, incremental piecemeal reforms but big changes. We write together, then, not to celebrate our own accomplishments but to articulate a common intention and find strength in community with another. We see this article as a form of collective accountability, a public commitment to continuing to imagine and practice toward an anti-carceral education that’s not just free of racialized surveillance but is a site of collective liberation.

Notes

[1] Beyond debt, many scholars have traced how the neoliberal university has increasingly emulated the for-profit sector by embracing market logics of competition and austerity, as manifested through adjunctification, skyrocketing tuition, the popularization of pedagogical approaches centered on “standards,” cutting support programs like mental-health care and food pantries, corporate partnerships and donors, and the regional and global expansion of campus real estate (Bousquet 2008; Baldwin 2021; Reed 2020). Such shifts sharply target Black, brown, working-class, and/or disabled students and staff (Chatterjee and Maira 2014; Dolmage 2017; Okechukwu 2019; Rice-Evans 2020).

[2] As part of a “pilot program” Macaulay Honors College accepted 18 students from Bronx Community College and Manhattan Community College (Jaschik 2018).

[3] The anti-Black racism is documented by Jesi Taylor Cruz in the May 17, 2017 article “A Seat in the Honors Academy,” which is no longer available in Excelsior, a Brooklyn College publication. It is archived online here.

[4] Can be found on Instagram @freecuny.

[5] A March 30 Board of Trustee resolution had introduced a Credit/No Credit policy where students had the option to leave their Spring 2020 out of the calculation of their GPA.

[6] Of course, many more lessons and reflections (about many more things than grading and surveillance) emerged from this campaign, but they do not belong to this article!

References

“‘A for All’: Frequently Asked Questions (FAQ).” 2020. https://docs.google.com/document/d/1VCg4UWC0JbiqRdhXAHTLuIq1u5UC2Npn74LFkQSeeIc.

Aptekar, Sofya, Corinna Mullin, and Karanja Keita Carroll. 2021. “Abolition vs. the Myth of ‘Public Safety’: Past and Present Struggles for a Liberated CUNY.” CUNY Struggle (blog). January 29, 2021. https://cunystruggle.org/2021/01/29/abolition-vs-the-myth-of-public-safety-past-and-present-struggles-for-a-liberated-cuny/.

Baldwin, Davarian L. 2021. In the Shadow of the Ivory Tower: How Universities Are Plundering Our Cities. New York: Bold Type Books.

Bousquet, Marc. 2008. How the University Works: Higher Education and the Low-Wage Nation. New York: New York University Press.

Bowen, Barbara. 2020a. “Stop Thousands of Layoffs at CUNY This Week!,” May 26, 2020.

———. Letter to Delegate Assembly listserv. 2020b. “Adjunct Notification Deadline Extended to May 29,” May 13, 2020.

Brooklyn College Islamic Society, and Muslim Women’s Educational Initiative. 2020. “‘Watched’ Discussion.” Facebook Live. 2020. https://www.facebook.com/watch/live/?ref=watch_permalink&v=275998503813213.

Brooklyn College Office of the Registrar. 2020. “Virtual Office Hours to Assist with Grade Submission Difficulties,” May 26, 2020.

Browne, Simone. 2015. Dark Matters: On the Surveillance of Blackness. Durham: Duke University Press.

CUNY for Abolition and Safety, Macaulay Diversity Initiative, and Macaulay Peace Action. 2020. “OPEN LETTER TO MACAULAY HONORS COLLEGE ADMINISTRATION.” 2020. https://docs.google.com/document/u/0/d/1IMJc4gKn-ibHJmVYKZIHdOIL-_YIp3YkCeGZZ8NYsT4/mobilebasic.

Davis, James. 2020. “Submit Your Spring 2020 Grades on May 28, Not Sooner.” Brooklyn College PSC-CUNY (blog). May 21, 2020. https://pscbc.blogspot.com/2020/05/submit-your-spring-2020-grades-on-may.html.

Dolmage, Jay. 2017. Academic Ableism: Disability and Higher Education. Ann Arbor: University of Michigan Press.

Dunn, Julia. 2020. “CUNY Urged to Delay Spring Grades, Prepares for Possible Grade Strike.” CBS News | WRGB Albany. May 21, 2020. https://cbs6albany.com/news/local/cuny-urged-to-delay-spring-grades-prepares-for-possible-grade-strike.

Elbow, Peter. 1997. “Grading Student Writing: Making It Simpler, Fairer, Clearer.” New Directions for Teaching and Learning 69: 127–40.

Ferguson, Roderick A. 2012. The Reorder of Things: The University and Its Pedagogies of Minority Difference. Minneapolis: University Of Minnesota Press.

Free CUNY! 2020. “CUNY Student Pledge of Solidarity with Faculty Withholding Grades for Spring 2020.” May 2020. https://docs.google.com/forms/d/e/1FAIpQLSdTMk_tlSqeUU10DCWI5D_IxDTZdoYFQy0mHNXXHopZ0YkojQ/viewform.

Gilmore, Ruth Wilson. 2007. Golden Gulag: Prisons, Surplus, Crisis, and Opposition in Globalizing California. Berkeley: University of California Press.

Glück, Zoltán, Manissa Maharawal, Isabelle Nastasia, and Conor Tomás Reed. 2014. “Organizing Against Empire: Struggles over the Militarization of CUNY.” Berkeley Journal of Sociology 58: 51–58.

Goldberg, Jesse, Jane Guskin, Vani Kannan, Marianne Madoré, Conor Tomás Reed, and Dhipinder Walia. 2020. “A for All (Yes, All!): Transforming Grading during COVID-19.” Medium (blog). May 3, 2020. https://medium.com/@conortomasreed/a-for-all-yes-all-transforming-grading-during-covid-19-a3a24de4e249.

Grande, Sandy. 2018. “Refusing the University.” In Toward What Justice?: Describing Diverse Dreams of Justice in Education, edited by Eve Tuck and K. Wayne Yang, 47–65. New York: Routledge.

Gumbs, Alexis Pauline. 2014. “Nobody Mean More: Black Feminist Pedagogy and Solidarity.” In The Imperial University: Academic Repression and Scholarly Dissent, edited by Piya Chatterjee and Sunaina Maira, 237–60. Minneapolis: University of Minnesota Press.

Gutiérrez y Muhs, Gabriella, Yolanda Flores Niemann, Carmen G. González, and Angela P. Harris, eds. 2012. Presumed Incompetent: The Intersections of Race and Class for Women in Academia, 1st edition. Logan: Utah State University Press.

Hoff, James Dennis. 2020. “CUNY Faculty On Verge of Wildcat Strike.” Left Voice, May 19, 2020. https://www.leftvoice.org/cuny-faculty-on-verge-of-wildcat-strike/.

Hunter Envoy. 2013. “Kroll Report Released,” February 5, 2013. https://hunterenvoy.net/articles/kroll-report-download.

Inoue, Asao B. 2019. Labor-Based Grading Contracts: Building Equity and Inclusion in the Compassionate Writing Classroom. Fort Collins: WAC Clearinghouse.

Jaschik, Scott. 2018. “Another College Ends Ban on Transfer Students.” Inside Higher Ed, November 19, 2018. https://www.insidehighered.com/admissions/article/2018/11/19/cunys-honors-college-ends-ban-transfer-students.

Johnson, Gavin P. 2021. “Grades as a Technology of Surveillance: Normalization, Control, and Big Data in the Teaching of Writing.” In Privacy Matters: Conversations about Surveillance within and beyond the Classroom, edited by Estee Beck and Les Hutchinson Campos, 53–72. Boulder: Utah State University Press.

Jordan, June. 1969. “Black Studies: Bringing Back the Person.” Evergreen Review, 39–41, 71–72.

Kaba, Mariame. 2021. We Do This ’til We Free Us: Abolitionist Organizing and Transforming Justice. Chicago: Haymarket Books.

Kannan, Vani. 2019. “Taking a Lead from Student Movements in a ‘Political Turn.’” In Writing Democracy: The Political Turn in and Beyond the Trump Era, edited by Shannon Carter, Deborah Mutnick, Stephen Parks, and Jessica Pauszek, 130–37. New York: Routledge.

Kynard, Carmen. 2008. “Writing While Black: The Colour Line, Black Discourses and Assessment in the Institutionalization of Writing Instruction.” English Teaching: Practice and Critique 7, no. 2: 4–34.

———. 2016. “This Bridge: The BlackFeministCompositionist’s Guide to the Colonial and Imperial Violence of Schooling Today.” Feminist Teacher 26, nos. 2–3: 126–41.

la paperson. 2014. A Third University Is Possible. Minneapolis: University of Minnesota Press.

Laymon, Kiese. 2014. “My Vassar College Faculty ID Makes Everything OK.” Gawker (blog). November 29, 2014. http://gawker.com/my-vassar-college-faculty-id-makes-everything-ok-1664133077.

Lopes, Anne. 2020. “Grades Submission Due,” May 26, 2020.

Lopes, Anne, and Lillian O’Reilly. 2020. “Spring 2020 Grading Deadline and Guidance for Faculty,” May 14, 2020.

Meiners, Erica R. 2007. Right to Be Hostile: Schools, Prisons, and the Making of Public Enemies. New York: Routledge.

Meyerhoff, Eli. 2019. Beyond Education: Radical Studying for Another World. Minneapolis: University of Minnesota Press.

Nemorin, Selena. 2017. “Post-Panoptic Pedagogies: The Changing Nature of School Surveillance in the Digital Age.” Surveillance & Society 15, no. 2: 239–53.

“New York City Advantages.” n.d. Macaulay Honors College. Accessed October 29, 2021. https://macaulay.cuny.edu/about-macaulay/nyc-advantages/.

Nieminen, Juuso Henrik. 2020. “Disrupting the Power Relations of Grading in Higher Education through Summative Self-Assessment.” Teaching in Higher Education, April, 1–16.

Okechukwu, Amaka. 2019. To Fulfill These Rights: Political Struggle Over Affirmative Action & Open Admissions. New York: Columbia University Press.

Pagan, Teona, Daniel Vasquez, Elizabeth Bazile, Hailey Lam, and Diana Kennedy. 2020. “It’s Time for CUNY to Say Goodbye to Cops: Fighting for a Free University.” The Abusable Past (blog). November 17, 2020. https://www.radicalhistoryreview.org/abusablepast/its-time-for-cuny-to-say-goodbye-to-cops-fighting-for-a-free-university/.

Pereira, Sydney. 2020. “CUNY Adjunct Layoffs Are Already Happening Ahead Of Cuomo’s Expected Budget Cuts.” Gothamist, May 14, 2020. https://gothamist.com/news/cuny-adjunct-layoffs-are-already-happening-ahead-cuomos-expected-budget-cuts.

PSC-CUNY. 2020. “Special Delegate Assembly—Minutes.” https://psc-cuny.org/sites/default/files/Special%20DA%205-11-20.pdf.

Rank and File Action (RAFA). 2020a. “Rank and File Action – Grade Withholding Action FAQ.” Google Docs. May 2020. https://docs.google.com/document/d/1h4g76gdmbeoNqe1iPP36CYFB1IG4DfeliRNiyciZIFY.

———. 2020b. “We Missed You at Our May 1 Town Hall!,” May 5, 2020.

———. 2020c. “Urgent Call: Pledge to Withhold Grades to Defend CUNY Jobs,” May 18, 2020.

———. 2020d. Letter to College Presidents, Chancellor Felix V. Matos Rodriguez, Mayor Bill de Blasio, and Governor Andrew Cuomo. “Enough Is Enough: We Demand the Reinstatement of All Faculty and the Repeal of Any Future Tuition Hikes or Student Fee Increases,” May 24, 2020. https://static1.squarespace.com/static/5ec57bde87d3dc61a24d2da5/t/5ecbfe8629db2e1e76cdede3/1590427270605/Enough+is+Enough_+We+Demand+the+Reinstatement+of+all+Faculty+and+the+Repeal+of+any+Future+Tuition+Hikes+or+Student+Fee+Increases.pdf.

———. 2020e. “Academic Solidarity.” https://www.rankandfileaction.com/home/academic-solidarity.

———. 2020f. “Community Solidarity.” https://www.rankandfileaction.com/home/community-solidarity.

Reed, Conor Tomás. 2020. “Hot City: Realizing the Dream of a Liberation University.” Verso (blog). September 8, 2020. https://www.versobooks.com/blogs/4848-hot-city-realizing-the-dream-of-a-liberation-university.

Rice-Evans, Jesse. 2020. “Open Access Pedagogy: A Manifesto.” Anti-Ableist Composition 1: 5.

Salaita, Steven. 2014. “Normatizing State Power: Uncritical Ethical Praxis and Zionism.” In Imperial University: Academic Repression and Scholarly Dissent, edited by Piya Chatterjee and Sunaina Maira, 217–35. Minneapolis: University of Minnesota Press.

Sandoval, Gabriel. 2020. “Looming CUNY Budget Cuts Have Faculty and Students Fearing for the Future.” THE CITY, May 4, 2020. https://www.thecity.nyc/education/2020/5/4/21257170/looming-cuny-budget-cuts-have-faculty-and-students-fearing-for-the-future.

Schirmer, Eleni, Jason Wozniak, Dana Morrison, Joanna Gonsalves, and Rich Levy. 2021. “Making the Invisible Visible: Organizing against the Instructionally Harmful, Antidemocratic Effects of Institutional Debt.” AAUP Journal of Academic Freedom 12: 1–18.

Shedd, Carla. 2020. Black and Latinx students, institutional racism, and the carceral continuum. Interview by Jon Moscow and Amy Halpern-Laff. Ethical Schools Podcast. https://ethicalschools.org/2020/01/black-and-latinx-students-institutional-racism-and-the-carceral-continuum/.

Sojoyner, Damien. 2016. First Strike: Educational Enclosures in Black Los Angeles. Minneapolis: University of Minnesota Press.

Theoharis, Jeanne. 2016. “‘I Feel Like a Despised Insect’: Coming of Age Under Surveillance in New York.” The Intercept. February 18, 2016. https://theintercept.com/2016/02/18/coming-of-age-under-surveillance-in-new-york/.

Wilder, Craig Steven. 2014. Ebony and Ivy: Race, Slavery, and the Troubled History of America’s Universities. New York: Bloomsbury Publishing.

Wozniak, Jason. 2021. “Debt, Accumulation, and Education.” Virtual conference paper, October 13, 2021, American Studies Association (ASA) Conference.

About the Authors

Joaly Burgos (she/her) is a recent graduate of Queens College where she earned a bachelor’s degree in Urban Studies and Planning. She is currently pursuing a master’s in Urban Affairs also at Queens College.

Jane Guskin (she/they) is an adjunct instructor in Urban and Labor Studies at Queens College and a PhD student in sociology at the Graduate Center, CUNY. Jane is a delegate representing Queens College in the PSC-CUNY union and is active in CUNY-wide rank-and-file antiracist and anticolonial worker organizing.

Hailey Lam (she/they) is a recent graduate of the Graduate Center, CUNY, where they earned their MA in Liberal Studies, and Brooklyn College, where they earned a bachelor’s degree in English and Political Science.

Marianne Madoré (she/her) is pursuing a PhD in sociology at the Graduate Center, CUNY, and teaches undergrad courses at Brooklyn College and John Jay College.

Andréa Stella (she/her) is a white neuroqueer mother and PhD student in composition and rhetoric at the Graduate Center. She teaches Writing for Engineering at City College of New York and researches ways to weave access and abolitionist pedagogies into STS writing classrooms.

Anna Zeemont (she/her) is a PhD candidate in English at the Graduate Center, where she studies composition and rhetoric with a concentration in American Studies. She’s also a Fellow at the GC’s Teaching & Learning Center and has taught and worked in multiple capacities across the CUNY system.

Toward Abolishing Online Proctoring: Counter-Narratives, Deep Change, and Pedagogies of Educational Dignity

Abstract

A future with ubiquitous academic surveillance is not sealed, not yet. In this essay, I discuss how online proctoring companies sell their technology with stories inspired by the edtech imaginary. Higher education institutions, in turn, often repeat these narratives, as evidenced by the ways institutions frame the technology as neutral, convenient tools for facilitating assessments. I propose a possible path toward abolishing online proctoring by authoring counter-narratives. I identify two spaces for constructing counter-narratives. First, we can apply a cognitive perspective to policy implementation to shift individual educators’ understanding of online proctoring through dissonance-producing institutional resources. Second, we can build collective partnerships between administrators, staff, faculty, and students to achieve deep change in our assessment practices. This potential path forward is guided by dual commitments: to reject online proctoring and the intersectional harms endured by students forced to use the technology; and to uproot the underlying pedagogies of policing and punishment that support online proctoring and replace them with pedagogies of educational dignity. I end my essay with a call to adopt an abolitionist approach to ridding education of online proctoring. By exercising abolitionist principles of refusal and care, along with a rejection of reform as an acceptable middle ground, we can move closer to creating the kinds of learning environments and relationships that cultivate students’ educational dignity.

The story of online proctoring is difficult to disentangle from surveillance and policing. Companies with names like Honorlock and Respondus Monitor conjure images of a patriarchal panopticon. Then there’s Proctortrack’s origin story. The chief technology officer for Verificient Technologies, the company that developed Proctortrack, arrived at the idea for the online proctoring technology after working on a Transportation Security Administration (TSA) project that included searching video footage for facial expressions deemed abnormal (Singer 2015). A version of the TSA’s security theater, online proctoring is further evidence of Sasha Costanza-Chock’s observation that “The same cisnormative, racist, and ableist approach that is used to train the models of the millimeter wave scanners [used by the TSA] is now being used to develop AI in nearly every domain” (2020, 5). I worry that the mechanized dehumanization experienced by individuals from nondominant groups at airport security is now being normalized in education due to online proctoring.

The attempts to make prejudiced technology prosaic are facilitated by online proctoring companies and their commitment to an edtech imaginary and its powerful storytelling. Audrey Watters describes the edtech imaginary as a collection of “stories we invent to explain the necessity of technology, the promises of technology; the stories we use to describe how we got here and where we are headed” (2020). Read the statements from online proctoring CEOs and the claims made by companies on their websites, and you can see the edtech imaginary at work. Online proctoring is supposedly necessary because, in the words of ProctorU’s CEO, without it, cheating will increase and pose “a severe threat to all higher education’” (Feathers and Rose 2020). The hollowness of the edtech imaginary is further illustrated in the diminishing story sold by Proctorio. Beginning in January 2019, the company promised institutions their “software eliminates human error [and] bias” (Proctorio 2019). The company’s homepage declared their software’s impressive capability until April 2021. On April 19, 2021, the Federal Trade Commission warned companies not to claim their algorithms can erase bias (Jillson 2021). Within days Proctorio’s promise of unbiased technology shrank to “Our software attempts to remove human bias and error” (Proctorio 2021). Visit the company website today, and you will find the edited sentence has disappeared. The edtech imaginary features many such revisionist narrators.

I want to consider the other elements of the edtech imaginary described by Watters: how we got here and where we might go. I’m trying to understand how institutions and people in power too often come to believe edtech’s glossy narratives about the past, present, and future. I’m also searching for the sites where we can share our counter-narratives. Alongside counter-narratives, I’m seeking ways we might uproot the pedagogies of policing and punishment that make online proctoring possible and replace them with pedagogies of educational dignity.

Educational dignity is critical for enacting a just present and future (Espinoza and Vossoughi 2014; Espinoza et al. 2020). Educational dignity is “the multifaceted sense of a person’s value generated via substantive intra- and inter-personal learning experiences that recognize and cultivate one’s mind, humanity, and potential” (Espinoza et al. 2020, 326). Online proctoring and its shallow definitions of learning are incompatible with educational dignity because of the technology’s hostility toward every individual forced beneath a webcam’s glare. The technology can harden internalized oppression, especially for nondominant students (Bali 2021), through its built-in racism and ableism. Further, online proctoring positions educators as police officers and students as criminals, straining inter-personal learning experiences. Online proctoring, I should note, is not the sole source of negative intra-personal and inter-personal learning experiences. Acknowledging its encoded opposition to educational dignity, however, can encourage us to view its abolishment as a part of a larger project to help educators develop and practice pedagogies of educational dignity.

I turn now to possible ways of conducting that larger project. I will review the institutionalization of online proctoring; describe the importance of how institutional resources frame online proctoring; offer a case for how to create deep change in the ways educators understand online proctoring and its alternatives; and conclude with a call to take an abolitionist approach to ridding online proctoring from education.

A future with ubiquitous academic surveillance is not sealed, not yet.

How Did We Get Here?

The critiques of online proctoring are numerous. Online proctoring replicates inadequate assessment methods (Leafstadt 2017). Online proctoring can exacerbate a student’s anxiety, particularly a student with high anxiety (Woldeab and Brothen 2019), which in turn can have a negative impact on students’ ability to demonstrate their learning (Eyler 2018). Online proctoring technology is racist (Feathers 2021; Swauger 2020); ableist (Brown 2020; Zhu 2021); and it invades students’ privacy (Cahn et al. 2020; Germain 2020). Put another way, online proctoring not only reinforces ineffective, harmful pedagogies; it’s also a deeply unethical technology.

Joining these critiques are the thousands of students who have documented and shared their experiences with online proctoring:

I know that I’m going to have to try a couple times before the camera recognizes me…I have a light beaming into my eyes for the entire exam…That’s hard when you’re actively trying not to look away, which could make it look like you’re cheating…[The software] is just not accurate. So I don’t know if it’s seeing things that aren’t there because of the pigment of my skin.
—Femi Yemi-Ese, student at the University of Texas at Austin (quoted in Caplan-Bricker 2021)

I’ve despised using this software…. On one occasion, I was “flagged” for movement and obscuring my eyes. I have trichotillomania triggered by my anxiety, which is why my hand was near my face. Explaining this to my professor was nightmarish.
—Bea, student at Tarrant County College (quoted in Retta 2020)

It’s really cruel to have students come to class and expect to learn, and then treat them, essentially, like criminals and make them install programs that look for all their information and force them to give tours of their home.
—Anonymous, student at the University of Washington (quoted in Hipolito 2020)

The chorus of student criticism has apparently not done enough to slow institutions and faculty from deploying the technology against students. For example, Proctorio’s CEO claimed his company helped to proctor 25 million exams at 1,000 institutions in 2020 (Harwell 2020).

To help to explain the growth of such an apparently toxic technology, it’s important to note that institutional use of online proctoring predates the coronavirus pandemic. The existing institutional knowledge and resource infrastructure, combined with the coronavirus pandemic’s demands for quick and cheap solutions to complex teaching and learning problems, meant online proctoring could take root farther and faster than might otherwise have been the case. The upheaval also presented educational technology companies an opportunity to activate the edtech imaginary and present themselves as partners ready and able to assist institutions’ pivot to remote emergency teaching. In some cases, non-online proctoring companies joined together with online proctoring companies, marketing their wares directly to educators for free (Top Hat 2020). The companies’ beneficence can be understood as an attempt to deepen their connections to institutions as well as to circumscribe what we imagine when we envision online learning and its possibilities.

Once a technology becomes well-established at an institution, it can be difficult to uproot (Arthur 1994). As a consequence of the pandemic, institutions have made substantial financial investments in online proctoring technology. The University of California at Santa Cruz, for instance, spent $200,000 for online proctoring in 2020–2021, and the institution’s leadership plans to continue to fund online proctoring (Harwood 2021). In addition to the monetary cost, institutions and their employees incurred a labor cost, too. Staff members had to learn how to use and support the technology. Faculty who decided to use the technology learned enough to do so, or they may have relied on staff and graduate students to troubleshoot technical problems, which meant any staff member or graduate student called upon to troubleshoot must have known how to fix the problems, and if not, they may have turned to the companies themselves for help. And finally, students, who rarely have a say in the matter, learned how to use the technology if they wanted to pass a class.

The money and labor sunk into online proctoring moves the institution, its employees, and its students further down the online proctoring path in a process of increasing returns (Pierson 2000) and software sedimentation (Weller 2020) so that change is difficult to contemplate let alone implement. As we’ve seen, the edtech imaginary is invested in software sedimentation. In response to criticism, online proctoring CEOs have promised friendlier interfaces and faster loading times (Deighton 2021), design “upgrades” presumably meant to make online proctoring more acceptable and ready for further sedimentation.

Another sedimentation tactic used by online proctoring’s defenders is to argue students have long been surveilled (Global Silicon Valley 2021). Since surveilling students is not new, these advocates observe, then contemporary warnings about academic surveillance are unfair. I read this argument as an attempt to make online proctoring more palatable—and thus more profitable—by conflating the technology with in-person proctoring. However, online proctoring is invasive in ways in-person proctoring is not (Fitzgerald 2021).

An in-person proctor does not demand to view a student’s bedroom. An in-person proctor is not an unflinching gaze trained to interpret students’ behavior through the singular lens of suspicion. When online proctoring executives and other adherents of online proctoring collapse the differences between in-person and online proctoring, they are reaching into the edtech imaginary. The story that emerges is a history of assessment practices meant to make their technology appear to be an uncontroversial extension of how students have always completed homework, quizzes, and tests. Do not trust online proctoring companies to be credible narrators. Their business depends on selling a specific tale of how we got here and where we should be going, and if nothing else, their public relations version of education history should be met with profound skepticism.

Elsewhere, online proctoring has been equated with older online learning technologies like “poorly recorded video lectures [and] inactive LMS discussion boards” (Selwyn et al. 2021, 13). I am concerned about the ways the edtech imaginary is succeeding to shape the discourse and frame online proctoring as a misunderstood, humdrum technology. I do not want racist, ableist academic surveillance to be a practice educators and students shrug off as an unfortunate but necessary part of learning. I do agree with Selwyn et al. (2021) that online proctoring demands we “develop counter-narratives that push back against the imagining of public education as simply a ‘tech issue’” (14). Where and how these counter-narratives emerge is an urgent question.

From Neutrality to Dissonance

Before exploring online proctoring counter-narratives, I want to consider how higher education institutions normalize online proctoring. Of 100 randomly selected US and Canadian college and university websites chosen from a sample of 2,155, “none took a critical stance toward proctoring tools or addressed the ethics of student surveillance” (Kimmons and Veletsianos 2021). Official institutional policy appears to treat online proctoring tools as neutral educational technology. The finding is perhaps unsurprising. While exceptions do exist (e.g., “Proctoring and Equity” from the Center for Innovative Teaching and Learning at Indiana University Bloomington), institutions that have invested money and labor into bringing online proctoring to campus may be hesitant or unwilling to criticize the same technology on public-facing websites. Neutrality is therefore a strategic choice. And because education is politics (Nieto 1999; Shor & Freire 1987), neutrality is a political choice too, one that aligns institutions with online proctoring companies.

Disrupting this neutrality becomes even more difficult because educators and institutions, perhaps unaware of the technology’s harms, often provide students with guiding language written by the online proctoring companies themselves. For example, Respondus Monitor offers instructors a template titled “Using LockDown Browser and a Webcam for Online Exams,” which instructors can copy and paste into a syllabus (Respondus n.d.). The syllabus template suggests to students that they “[t]ake the exam in a well-lit room and avoid backlighting, such as sitting with your back to a window” (Respondus n.d.). Missing from this recommendation and others like it is the reason why students must be in a well-lit room, sometimes having to resort to shining a bright light directly into their faces (Chin 2021): because many online proctoring companies use facial recognition technology. Not only do these technologies struggle to detect dark skin (Simonite 2019), they are built using biased datasets, leading to racialization and dehumanization (Stevens and Keyes 2021).

Online proctoring companies also shape the perception of their harmful technology at the institutional level. Just as individual educators might depend on the companies for ways to describe to students how to use the technology, so too do institutional how-to resources and websites. Institutional support pages are too often little more than hyperlinks to help guides and video tutorials created by the companies. In addition, an institutional resource page might repackage a company’s recommendations to students, such as one example when the Respondus Monitor syllabus language about lighting reappears on an institutional resource page warning students, “You may need to add more lighting to your workspace when using Respondus Monitor to ensure the program can recognize your face during the assessment” (Northwestern University n.d.). Once more, the reason why students need to add more lighting is glaringly absent. Online proctoring companies can continue to control the narrative about their technology as long as institutional resource pages are indistinguishable from the frequently asked questions websites produced by online proctoring companies. Thus, online proctoring companies have succeeded in making their technology appear benign by attempting to collapse the distinct differences between in-person and online proctoring. Companies have also benefited from instructors and institutions who frame the technology as neutral, often parroting company copy on syllabi and how-to webpages.

Taking lessons from a cognitive approach to learning and policy implementation can help explain why changing people’s understanding about online proctoring might be especially hard when the technology is presented in such a way that its functionality appears both commonplace and unambiguously advantageous.

We draw on prior knowledge and existing beliefs when interpreting new information (Bransford, Brown, and Cocking 2000). A problem arises because “New ideas either are understood as familiar ones, without sufficient attention to aspects that diverge from the familiar, or are integrated without restructuring of existing knowledge and beliefs, resulting in piecemeal changes in existing practice” (Spillane et al. 2002, 398). What does this mean for educators encountering online proctoring for the first (or fifth) time? When neutral or positive language masks the technology’s harms, then online proctoring can appear not to be so much a new idea but instead a logical, if imperfect, extension of an educator’s existing beliefs and practices.

That online proctoring can either be an outgrowth of, or seem an outgrowth of, existing beliefs and practices is evidence of a larger problem: the beliefs and practices themselves. Pedagogies of policing and punishment are the soil sustaining online proctoring. It’s not enough to weed out online proctoring. Instead, what we could use is a controlled burn.

To light a fire that removes online proctoring from higher education, start by revising institutional websites and resources to explicitly name and describe online proctoring’s harms. These revisions—these counter-narratives—need to produce cognitive dissonance in educators in order to disrupt the narrative of online proctoring as a necessary, innocuous technology. This dissonance can force educators to confront both the technology itself and the underlying beliefs about learning that help educators rationalize deploying academic surveillance against their students. A goal is to help educators “recognize an existing model as problematic and, then, to focus resources and support on attempts to make sense of the novel idea, restructuring existing beliefs and knowledge” (Spillane et al. 2002, 418). In other words, sparking a shift in a person’s thinking begins with illuminating the ways online proctoring is a problem both as a technical solution and as a pedagogical practice. A dissonance-producing institutional resource about online proctoring might look like Figure 1:

A hypothetical institution webpage introduces educators to online proctoring. The upper left of the page contains a quote from an anonymous student that reads: “‘It’s really cruel to have students come to class and expect to learn, and then treat them, essentially, like criminals and make them install programs that look for all their information and force them to give tours of their home.’” The upper right of the page contains an image of a camera's lens. The middle of the page contains the words: “Online proctoring is racist, ableist, and privacy-invading.” Beneath these words, appears a quote which reads: “‘If we understand teaching as consisting primarily of social relationships and as a political commitment rather than a technical activity, then it is unquestionable that what educators need to pay most attention to are their own growth and transformation and the lives, realities and dreams of their students.’ - Sonia Nieto”. Beneath the quote from Sonia Nieto appears the words: “How might we abandon pedagogies of policing and commit to pedagogies of educational dignity for ourselves and our students?” followed by a bullet-pointed list with the items: “Embrace a more holistic view of learning. Develop authentic assessments. Adopt ungrading practices. Foster partnerships between administrators, staff, faculty, and students.” At the page's bottom, two sentences read: “Read more about the harms of online proctoring and how to transition away from the technology.” and “Make an appointment with the Center for Teaching and Learning, an instructional designer, and/or an educational technologist.” The word “References” is at the bottom of the page, followed by two references. The first citation is: Hipolito, Matthew. 2020. “‘Going Through Your Things’: Remote Proctoring Software ‘Demeaning’ and ‘Cruel,’ Students Say.” The Daily, October 29, 2020. https://www.dailyuw.com/news/article_8b14f13e-197a-11eb-8730-c7459eeb446a.html. The second citation is: Nieto, Sonia. 1999. The Light in Their Eyes: Creating Multicultural Learning Communities. New York: Teachers College Press.
Figure 1. A hypothetical institutional webpage that uses a critical framing when introducing educators to online proctoring.

I recognize it’s unlikely the above resource will be adopted by institutions of higher education across the land. So I have another suggestion. Before providing a reader with installation directions and other troubleshooting tips, which is what many institutional resources do, the resource could prompt an educator to reflect on the technology and its effects by asking:

  • Do you believe students with dark skin should have to shine a bright light on their faces to be recognized as having a face by the online proctoring technology?
  • Do you believe diabetic students should be too afraid to check their blood sugar levels or eat a snack for fear of being labeled suspicious by the online proctoring technology?
  • Do you believe students should allow a stranger to have remote access to their personal computer?
  • Would you want to show a stranger your office or bedroom before an exam begins or while taking an exam?
  • Is your pedagogy founded on distrust, policing, and punishment?

Institutional resources about online proctoring may appear to play a seemingly small role in the larger conversation about the technology and its impacts on teaching and learning. However, understanding the resources as a vehicle the edtech imaginary uses to influence teaching and teachers themselves emphasizes the need to attend to how the resources frame online proctoring. Institutional resources about online proctoring can be understood as a policy technology—a technology about technology, if you will—or a means designed to implement policy. Other policy technologies include curricula and assessments (Spillane et al. 2019). The problem with institutional resources adopting a neutral or positive framing of online proctoring is thus twofold. First, as previously discussed, uncritical resources can produce, reinforce, and normalize academic surveillance and pedagogies of distrust, policing, and punishment by being assimilated into educators’ preexisting beliefs and practices.

A second damaging consequence exists. When an educator’s pedagogy is pushed toward policing and punishment, practices enabled in part by uncritical resources, their sense of themselves as a teacher risks being corrupted. Here Stephen Ball’s observation that “we do not do policy, policy does us” (2015, 307) helps to articulate why focusing on the resources’ language is so important. Because if our pedagogy is an outgrowth of our identities as educators, and policy shapes our sense of self, then a pedagogy of punishment wants us to become punishers. The policy “does us” by defining who we can be as teachers and who our students can be as learners. Recall a student forced to submit to online proctoring felt like a criminal because the technology positions students as inherently suspicious. And if a policy of online proctoring transforms students into criminals, then it turns teachers into police officers—and cops, I believe, should be banned from campuses.

A Story of Reform

Overcoming online proctoring and the pedagogies that maintain its use might begin with the individual, but we improve the chances of abolishing the technology when we join together to unlearn harmful pedagogies and replace them with pedagogies of educational dignity. To grow pedagogies of educational dignity, we can couple a cognitive approach to policy implementation with a stance toward learning as a fundamentally social experience (Lave and Wenger 1991; Vygotsky 1978). Many educators concerned about online proctoring have realized the social nature of learning by organizing events to learn with and from one another. Examples of collective meaning making include the Teach-In #AgainstSurveillance (Gray 2020) and the #AnnotateEdTech events (Logan and Caines 2021). These online gatherings, while vital for building community and solidarity, may nonetheless struggle to bring about the systemic change at institutions many of us seek.

To accomplish change at scale—a favorite word, I know, of the edtech imaginary—the movement against online proctoring can address the depth of educators’ beliefs and practices; the sustainability of changes over time; the spread of changes throughout an institution; and a shift in ownership over the new ideas from external to internal sources (Coburn 2003). Remember that changing an individual’s understanding requires resources and support (Spillane et al. 2002). Combine these additional resources and support with the elements for achieving deep change at scale (Coburn 2003), and the project of ridding online proctoring from campuses appears daunting.

Nonetheless, we can turn to the efforts of a coalition of administrators and staff at the University of Michigan-Dearborn for an example of institutional change at scale. In March 2020, the University shifted to remote emergency teaching. At the same time, the Office of the Provost and deans decided to publicly oppose online proctoring, and though the administrators did not ban the technology, they did strongly recommend faculty not use it (Silverman et al. 2021). In the months that followed, the staff at the Hub for Teaching and Learning Resources (the Hub) worked to implement the reform through a combination of depth, sustainability, spread, and shift (Coburn 2003). See Table 1 for how the University tried to accomplish the different dimensions of reform implementation.

Dimension of Reform Implementation How Administrators and the Hub’s Staff at the University of Michigan-Dearborn Tried to Accomplish the Dimension of Reform Implementation
Depth

Changes to educators’ beliefs, educators’ interactions with students, and educators’ pedagogies.

  • provided individual consultations to faculty new to online teaching.
  • assisted faculty to develop authentic assessments.
  • hosted a virtual guest speaker, an expert in authentic assessments with a speciality in the STEM disciplines.
  • organized multiple faculty development programs throughout the year.
Sustainability

A long-term commitment to nurturing educators’ development over time.

  • hired two additional instructional designers on two-year contracts.
  • hired graders to aid faculty in high-enrollment courses to grade and provide feedback on more time-intensive authentic assessments.
Spread

The diffusion of reform-related pedagogical principles within a course, department, and institution.

  • provided individual or group email responses from the Hub, the Office of Digital Education, and the Office of the Provost with a consistent message that the decision to avoid online proctoring was due to student privacy and equity concerns.
  • Silverman et al. (2021) acknowledge communications with faculty “could have been better wrapped into a cohesive, campus-wide message” (121) to improve spread.
Shift

Ownership of the reform transitions from an external reform to an internal reform with authority for maintaining the change left to groups and individuals.

  • Silverman et al. (2021) recommend designing experiences to “develop a shared critical digital literacy between instructors and students by discussing the ethical problems associated with remote proctoring and building a shared understanding of academic integrity in the digital age” (126).
Table 1. Applying Coburn’s (2003) concept of scale to the University of Michigan-Deaborn’s approach to shifting educators’ beliefs and practices regarding online proctoring.

We need counter-narratives. However, a strategy for abolishing online proctoring built only on counter-narratives risks ceding the terms of the debate to those set by the online proctoring companies. For this reason, we also need stories that aren’t defined solely in opposition to the likes of online proctoring CEOs and the edtech imaginary they’re entranced by. The coalition against online proctoring that emerged at the University of Michigan-Dearborn is an instructive example of one such alternative narrative, a story of how we might achieve deep change by developing a partnership organization founded on relationships (Logan 2020). What started as co-authoring a counter-narrative about online proctoring at the University became, over time, a new narrative about partnerships between administrators, staff, faculty, and students to develop equitable, authentic assessments.[1]

The example set by the University of Michigan-Dearborn demonstrates that when administrators offer support and financial resources to reimagine teaching and learning, trusting staff and faculty along the way, resistance to and refusal of online proctoring can generate a community that rejects pedagogies of policing and embraces people and our immutable educational dignity.

Where Might We Go from Here?

The future of online proctoring is still being written. Appealing to institutions’ and students’ fears, online proctoring CEOs tell their tales of worthless coronavirus diplomas (Harwell 2020) tarnishing an institution’s brand and raising questions about a student’s employability. The narrative belongs to the larger playbook drawn up by the corporate education movement and its vision of learning as human capital development (Williamson 2017). I believe learning cannot be reduced to a datapoint to be quantified, a credential to be protected at all costs.

Online proctoring companies possess a paltry view of education that produces and reinforces pedagogies of punishment. When confronted with the intersectional damages inflicted upon students by their technology, online proctoring companies insist their products are necessary, claiming the technology is an engine of equity (Norris 2021). Yet as Chris Gilliard argues, “A better remote proctoring system isn’t on the way—it can’t be—because they are all built on the same faulty and invasive ideas…about pedagogy, surveillance, and control” (@hypervisible, April 6, 2021).

Online proctoring is not like in-person proctoring. Online proctoring is not like a badly lit lecture video or an underused discussion board. Online proctoring is a manifestation of what Ruha Benjamin calls the New Jim Code, or “the employment of new technologies that reflect and reproduce existing inequities but that are promoted and perceived as more objective or progressive than the discriminatory systems of a previous era” (2019, 5–6). When institutions and educators frame online proctoring with market-based stories and their boogeymen, they risk being duped by online proctoring companies and their unreliable narrators selling dubious promises of objectivity and equity as evidence the technology works. In contrast, when the story of online proctoring is framed as the instantiation of the New Jim Code and its racist, ableist surveillance, then those who experience the technology’s inequities—students—emerge as trustworthy narrators with heartbreaking accounts of the humiliations they’ve had to endure. Their stories should be part of the evidence we use as we seek to rid online proctoring from schools.

Including online proctoring as part of the New Jim Code offers another possibility: that of the abolitionist imaginary. Abolitionist practices, suggested sava saheli singh (Pasquini 2021), can be a generative source of imagination, politics, organizing, and action in the struggle against online proctoring and other problematic educational technologies. Abolitionism’s emphasis on refusal alongside care and collectivity (Kaba 2021), for example, is essential if we are to develop pedagogies of educational dignity.

In addition, the fight against online proctoring takes on greater urgency when we understand online proctoring as the latest example of white supremaist surveillance technologies designed and deployed to police and punish. Like previous racializing information technologies used to surveil and control people (Browne 2015), online proctoring’s harms are experienced disproportionately by Black people as well as other nondominant populations. This longview of online proctoring is vital, for as Bettina Love notes, “An ahistorical understanding of oppression leads folx to believe that quick fixes to the system, such as more surveillance, more testing, and more punishment, will solve the issues of injustice and inequality” (2019, 92). It also means adopting the abolitionist stance that reform, even at the scale accomplished by the University of Michigan-Dearborn, cannot be where the story of online proctoring ends.

If the story of online proctoring is to end in freedom, we can start by telling counter-narratives and fashioning new narratives altogether. I am hopeful these stories will include accounts of honest institutional resources and websites. Of administrators who abandon online proctoring, despite paying for its false promises, and invest instead in providing support and funding for faculty and staff to develop authentic assessments. And I am hopeful we will share stories of lasting partnerships between educators and students, coalitions that accomplish deep change and grow pedagogies of educational dignity.

Notes

[1] An emphasis on authentic assessment is an essential element for building pedagogies of educational dignity. Authentic assessments are characterized by self-reflection and collaboration with others (Conrad and Openo 2018). Prioritizing self-reflection and embracing individuals’ genuine, complex selves can support educational dignity through intra-personal learning. Authentic assessment can also help students experience educational dignity through its frequent use of learning with and from other people, a crucial design choice upon which educational dignity relies (Espinoza et al. 2020).

References

Arthur, W. Brian. 1994. Increasing Returns and Path Dependence in the Economy. Ann Arbor: University of Michigan Press.

Bali, Maha. 2021. “On Proctoring & Internalized Oppression #AgainstSurveillance.” Reflecting Allowed, March 13, 2021. https://blog.mahabali.me/educational-technology-2/on-proctoring-internalized-oppression-againstsurveillance/.

Ball, Stephen. 2015. “What Is Policy? 21 Years Later: Reflections on the Possibilities of Policy Research.” Discourse: Studies in the Cultural Politics of Education 36, no. 3: 306–313. https://doi.org/10.1080/01596306.2015.1015279.

Benjamin, Ruha. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. Cambridge, UK: Polity.

Bransford, John, Brown, Ann, and Rodney Cocking, eds. 2000. How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press.

Brown, Lydia X. Z. 2020. “How Automated Test Proctoring Software Discriminates Against Disabled Students.” Center for Democracy and Technology, November 16, 2020. https://cdt.org/insights/how-automated-test-proctoring-software-discriminates-against-disabled-students/.

Browne, Simone. 2015. Dark Matters: On the Surveillance of Blackness. Durham: Duke University Press.

Cahn, Albert, Caroline Magee, Eleni Manis, and Naz Akyol. 2020. “Snooping Where We Sleep: The Invasiveness and Bias of Remote Proctoring Services.” Surveillance Technology Oversight Project, November 11, 2020. https://www.stopspying.org/snooping.

Caplan-Bricker, Nora. 2021. “Is Online Test-Monitoring Here to Stay?” New Yorker, May 27, 2017. https://www.newyorker.com/tech/annals-of-technology/is-online-test-monitoring-here-to-stay.

Chin, Monica. 2021. “ExamSoft’s Proctoring Software Has a Face-Detection Problem.” The Verge, January 5, 2021. https://www.theverge.com/2021/1/5/22215727/examsoft-online-exams-testing-facial-recognition-report.

Coburn, Cynthia. 2003. “Rethinking Scale: Moving Beyond Numbers to Deep and Lasting Change.” Educational Researcher 32, no. 6: 3–12. https://doi.org/10.3102/0013189X032006003.

Conrad, Diane, and Jason Openo. 2018. Assessment Strategies for Online Learning: Engagement and Authenticity. Edmonton: Athabasca University Press.

Costanza-Chock, Sasha. 2020. Design Justice: Community-Led Practices to Build the Worlds We Need. Boston: The MIT Press.

Deighton, Katie. 2021. “Online Proctoring Programs Try to Ease the Tensions of Remote Testing.” The Wall Street Journal, April 13, 2021. https://www.wsj.com/articles/online-proctoring-programs-try-to-ease-the-tensions-of-remote-testing-11618304400.

Espinoza, Manuel, and Shirin Vossouhi. 2014. “Perceiving Learning Anew: Social Interaction, Dignity, and Educational Rights.” Harvard Educational Review 84, no. 3: 285–313.

Espinoza, Manuel, Shirin Vossouhi, Mike Rose, and Luis Poza. 2020. “Matters of Participation: Notes on the Study of Dignity and Learning.” Mind, Culture, and Activity 27, no. 4: 325–347. https://doi.org/10.1080/10749039.2020.1779304.

Eyler, Joshua. 2018. How Humans Learn: The Science and Stories Behind Effective College Teaching. Morgantown: West Virginia University Press.

Feathers, Todd. 2021. “Proctorio Is Using Racist Algorithms to Detect Faces.” Vice, April 8, 2021. https://www.vice.com/en/article/g5gxg3/proctorio-is-using-racist-algorithms-to-detect-faces.

Feathers, Todd, and Janus Rose. 2020. “Students Are Rebelling Against Eye-Tracking Exam Surveillance Tools.” Vice, September 24, 2020. https://www.vice.com/en/article/n7wxvd/students-are-rebelling-against-eye-tracking-exam-surveillance-tools.

Fitzgerald, Bill. 2021. “There Is No Such Thing as an ‘Online Proctoring System.’” FUNNYMONKEY, August 21, 2021. https://www.funnymonkey.com/2021/08/there-is-no-such-thing-as-an-online-proctoring-system/.

Germain, Thomas. 2020. “Poor Security at Online Proctoring Company May Have Put Student Data at Risk.” Consumer Reports, December 10, 2020. https://www.consumerreports.org/digital-security/poor-security-at-online-proctoring-company-proctortrack-may-have-put-student-data-at-risk/.

Gilliard, Chris (@hypervisible). 2021. “A better remote proctoring system isn’t on the way—it can’t be—because they are all built on the same faulty and invasive ideas of about pedagogy, surveillance, and control.” Tweet, April 6, 2021. https://twitter.com/hypervisible/status/1379501433763078145.

Global Silicon Valley. 2021. “The Academic Surveillance Debate.” Filmed August 2021 at ASU+GSV 2021, San Diego, CA. Video, 40:09. https://www.youtube.com/watch?v=cX_svCuqzyU.

Gray, Brenna Clarke. 2020. “Against Surveillance Teach-in.” YouTube video, December 1, 2020. https://www.youtube.com/watch?v=tAuoj3VYy_s.

Harwell, Drew. 2020. “Cheating Detection Companies Made Millions During the Pandemic. Now Students Are Fighting Back.” Washington Post, November 12, 2020. https://www.washingtonpost.com/technology/2020/11/12/test-monitoring-student-revolt/.

Harwood, Zoe. 2021. “Surveillance U: Has Virtual Proctoring Gone Too Far?” YR Media, n.d. Accessed June 24, 2021. https://interactive.yr.media/has-virtual-proctoring-gone-too-far/.

Hipolito, Matthew. 2020. “‘Going Through Your Things’: Remote Proctoring Software ‘Demeaning’ and ‘Cruel,’ Students Say.” The Daily, October 29, 2020. https://www.dailyuw.com/news/article_8b14f13e-197a-11eb-8730-c7459eeb446a.html.

Jillson, Elisa. 2021. “Aiming for Truth, Fairness, and Equity in Your Company’s Use of AI.” Federal Trade Commission, April 19, 2021. https://www.ftc.gov/news-events/blogs/business-blog/2021/04/aiming-truth-fairness-equity-your-companys-use-ai.

Kaba, Mariame. 2021. We Do This ‘til We Free Us: Abolitionist Organizing and Transforming Justice, edited by Tamara K. Nopper. Chicago: Haymarket Books.

Kimmons, Royce, and George Veletsianos, G. 2021. “Proctoring Software in Higher Ed: Prevalence and Patterns.” EDUCAUSE Review, February 23, 2021. https://er.educause.edu/articles/2021/2/proctoring-software-in-higher-ed-prevalence-and-patterns.

Lave, Jean, and Etienne Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge, UK: Cambridge University Press.

Leafstedt, Jill. 2017. “Online Courses Shouldn’t Use Remote Proctoring Tools. Here’s Why.” EdSurge, April 19, 2017. https://www.edsurge.com/news/2017-04-19-online-courses-shouldn-t-use-remote-proctoring-tools-here-s-why.

Logan, Charles, and Autumm Caines. 2021. “Creating Counter-Media Texts in the Open With #AnnotateEdTech.” Presentation at OERxDomains21, April 27, 2021. https://docs.google.com/presentation/d/18zlOTBLdRni2Ul173Jpa_iwJD6gYO4aEnMFJzyhnjFk/edit?usp=sharing.

Logan, Charles. 2020. “Refusal, Partnership, and Countering Educational Technology’s Harms.” Hybrid Pedagogy, October 21, 2020. https://hybridpedagogy.org/refusal-partnership-countering-harms/.

Love, Bettina. 2019. We Want to Do More Than Survive: Abolitionist Teaching and the Pursuit of Educational Freedom. Boston: Beacon Press.

Nieto, Sonia. 1999. The Light in Their Eyes: Creating Multicultural Learning Communities. New York: Teachers College Press. 

Norris, Ashley. 2021. “Exam Integrity Is an Issue of Equity.” American Consortium for Equity in Education. Accessed June 23, 2021. https://www.ace-ed.org/exam-integrity-is-an-issue-of-equity/.

Northwestern University. n.d. “Respondus Lockdown Browser and Monitor.” Accessed October 19, 2021.https://canvas.northwestern.edu/courses/1580/pages/6-dot-8-respondus-lockdown-browser-and-monitor.

Pasquini, Laura. 2021. “Between the Chapters #25: Searching for the Commons in the Wasteland with @savasavasava & @audreywatters.” Between the Chapters podcast episode, April 29, 2021. https://share.transistor.fm/s/442cc432.

Pierson, Paul. 2000. “Increasing Returns, Path Dependence, and the Study of Politics.” The American Political Science Review 94, no. 2: 251–267. https://doi.org/10.2307/2586011.

Proctoring and Equity. (n.d.). Center for Innovative Teaching and Learning. Accessed June 23, 2021. https://citl.indiana.edu/teaching-resources/diversity-inclusion/proctoring/index.html.

Proctorio. 2019. “Homepage on January 26, 2019.” Accessed October 19, 2021. https://web.archive.org/web/20190126031131/https://proctorio.com/.

Proctorio. 2021. “Homepage on April 30, 2021.” Accessed June 24, 2021. https://web.archive.org/web/20210430212733/https://proctorio.com/.

Respondus. n.d. “Respondus Monitor Resources.” Accessed October 19, 2021. https://web.respondus.com/he/monitor/resources/.

Retta, Mary. 2020. “Exam Surveillance Tools Monitor, Record Students During Tests.” Teen Vogue, October 26, 2020. https://www.teenvogue.com/story/exam-surveillance-tools-remote-learning.

Selwyn, Neil, Chris O’Neill, Gavin Smith, Mark Andrejevic, and Xin Gu. 2021. “A Necessary Evil? The Rise of Online Exam Proctoring in Australian Universities.” Media International Australia: 1–16. https://doi.org/10.1177/1329878X211005862.

Shor, Ira, and Paulo Freire. 1987. A Pedagogy for Liberation. Westport, CT: Bergin & Garvey.

Silverman, Sarah, Autumm Caines, Christopher Casey, Belen Garcia de Hurtado, Jessica Riviere, Alfonso Sintjago, and Carla Vecchiola. 2021. “What Happens When You Close the Door on Remote Proctoring? Moving Towards Authentic Assessments with a People-Centered Approach.” To Improve the Academy: A Journal of Educational Development 39, no. 3: 115–131. https://doi.org/10.3998/tia.17063888.0039.308.

Simonite, Tom. 2021. “The Best Algorithms Struggle to Recognize Black Faces Equally.” Wired, July 22, 2019. https://www.wired.com/story/best-algorithms-struggle-recognize-black-faces-equally/.

Singer, Natasha. 2015. “Online Test-Takers Feel Anti-Cheating Software’s Uneasy Glare.” New York Times, April 5, 2015. https://www.nytimes.com/2015/04/06/technology/online-test-takers-feel-anti-cheating-softwares-uneasy-glare.html.

Sonnemaker, Tyler. 2020. “Tech Companies Promised Schools an Easy Way to Detect Cheaters During the Pandemic. Students Responded by Demanding Schools Stop Policing Them Like Criminals in the First Place.” Business Insider, November 1, 2020. https://www.businessinsider.com/proctorio-silencing-critics-fueling-student-protests-against-surveilalnce-edtech-schools-2020-10.

Spillane, James P., Brian J. Reiser, and Todd Reimer. 2002. “Policy Implementation and Cognition: Reframing and Refocusing Implementation Research.” Review of Educational Research 72, no. 3: 387–431. https://doi.org/10.3102/00346543072003387.

Spillane, James P., Jennifer L. Seelig, Naomi L. Blaushild, David K. Cohen, and Donald J. Peurach. 2019. “Educational System Building in a Changing Educational Sector: Environment, Organization, and the Technical Core.” Educational Policy 33, no. 6: 846–881. https://doi.org/10.1177/0895904819866269.

Stevens, Nikki, and Os Keyes. 2021. “Seeing Infrastructure: Race, Facial Recognition and the Politics of Data.” Cultural Studies 35, no. 4–5: 833–853. https://doi.org/10.1080/09502386.2021.1895252.

Swauger, Shea. 2020. “Our Bodies Encoded: Algorithmic Test Proctoring in Higher Education.” Hybrid Pedagogy, April 2, 2020. https://hybridpedagogy.org/our-bodies-encoded-algorithmic-test-proctoring-in-higher-education/.

Top Hat. 2020. “Top Hat Partners With Proctorio to Deliver Free, Secure Online Proctored Tests and Exams Remotely. Top Hat, April 2, 2020. https://tophat.com/press-releases/top-hat-partners-with-proctorio/.

Vygotsky, Lev Semyonovich. 1978. Mind in Society: The Development of Higher Psychological Processes. Michael Cole, Vera John-Steiner, Sylvia Scribner, and Ellen Souberman, eds. Cambridge: Harvard University Press.

Watters, Audrey. 2020. “The Ed-Tech Imaginary.” Hack Education. June 21, 2020. http://hackeducation.com/2020/06/21/imaginary.

Weller, Martin. 2020. 25 Years of Ed Tech. Edmonton: Athabasca University Press.

Williamson, Ben. 2017. Big Data in Education: The Digital Future of Learning, Policy and Practice. Thousand Oaks, CA: Sage.

Woldeab, Daniel, and Thomas Brothen. 2019. “21st Century Assessment: Online Proctoring, Test Anxiety, and Student Performance.” International Journal of E-Learning and Distance Education 34, no. 1. http://www.ijede.ca/index.php/jde/article/view/1106/1729.

Zhu, Eva. 2021. “‘Foolproof’ Exam Software Creating Barriers for Students With Disabilities.” Healthy Debate, March 1, 2021. https://healthydebate.ca/2021/03/topic/exam-software-disabilities/.

About the Author

Charles Logan is a PhD student in learning sciences at Northwestern University. A former high school English teacher and university educational technologist, his research interests include critical digital pedagogy, co-authoring counter-narratives to oppose sociotechnical and edtech imaginaries, and designing learning experiences to support educational dignity. He is on Twitter @charleswlogan.

0

Back Doors, Trap Doors, and Fourth-Party Deals: How You End up with Harmful Academic Surveillance Technology on Your Campus without Even Knowing

Abstract

In this paper we describe fourth-party vendor relationships between remote proctoring tools and other companies in higher education with a specific focus on the remote proctoring company Proctorio. We unpack the problematic nature of such relationships in general but note that they are exacerbated when dealing with technologies as harmful as remote proctoring. Fourth-party relationships are particularly troublesome because those who work at institutions of higher learning are often unaware of their existence or can do little to impact or change them. We present a “harm index” reviewing literature around the harms of remote proctoring systems. We describe the nature of different types of fourth-party relationships and perform a content analysis of the partnerships listed on Proctorio’s website. We use an autoethnographic approach to share our experience as instructional designers, at an institution which has taken steps to limit the use of remote proctoring, and of attempting (and succeeding) to get the fourth-party integration of Proctorio removed from our learning management system’s integration with McGraw Hill Connect. The paper concludes with a discussion of the discourse and rhetoric used to rationalize this harmful technology, and our recommendations for how institutions might exert more control over fourth-party integrations with harmful surveillance technology.

Introduction

The year is 2026, and Regional State University (RSU) has bounced back from the enrollment woes that resulted from the COVID-19 pandemic of 2020 and 2021. After intensive faculty development efforts during the pandemic, the majority of instructors on campus became skilled online instructors. RSU set itself apart during the pandemic by rejecting remote proctoring systems by issuing several strong recommendations against such tools to their faculty and offering faculty development opportunities around authentic assessments. RSU was featured in several national news articles for their stance on remote proctoring and celebrated for their attention to student privacy. New students were attracted to RSU because of its reputation for having strong online programs and also because of RSU’s commitment to student privacy and authentic assessments.

Aisha is a new student who specifically came to RSU because of its reputation for providing an excellent education with a personal touch. She had a friend, a year older than her, who had ended up at one of the for-profit online education schools and was routinely subjected to various forms of invasive surveillance. So, when Aisha’s math professor told the class that they would be using a remote proctoring system for some of their homework assignments she was a little put off. She knew the school did not have an official ban on such technologies but she had read the news articles and the Provost’s and the President’s statements advocating for student privacy. She tried to talk to her professor and asked specific questions about what data would be collected and how long it would be held but the professor only directed her to a long “terms of service” and privacy policy that was hard to understand. She decides to bite the bullet and just use the technology for this one class.

On her first attempt to use the technology she has to pass an identity check using facial recognition. She keeps getting an error that the software cannot detect her face and decides to call RSU’s technology help desk to see what the problem might be but no one there seems to know about this software. They ask her how she got access to this system and in tracing her steps she mentions that she started by clicking a link in the Learning Management System (LMS). So they transfer her to Summer, an administrator of the LMS. Summer tells Aisha that though she got to this proctoring system through the LMS that the school does not have any administrative access to this remote proctoring system and cannot help her in any way—that she will need to contact the remote proctoring company’s help line.

Little does Aisha know that Summer is a student privacy advocate and is shocked to hear that some students are being subjected to a remote proctoring system. Summer begins the process of trying to figure out what is going on but while waiting to hear back from the customer support rep at the math homework system she sees that RSU is in the national news—headline: “RSU Student Humiliated.” The article features Aisha who is recounting her experience with the remote proctoring company’s help line that Summer referred her to, and it just sounds horrible. They asked Aisha to jump through various hoops including shining a light on her face and even asked her to remove her hijab. Matters are made worse when a week later the proctoring company suffers a major data breach potentially exposing a massive amount of student data on the dark web. Summer is at a complete loss for what to do. Protecting student privacy is part of her job, but the company violating students’ privacy has no real relationship or accountability to RSU. She finds a clause buried in the university’s contract with the math homework system saying that they consider the contract to be binding for themselves but also for any partnerships that they enter into. Summer wonders if this means that the math homework company is legally responsible for the remote proctoring company—even so, it sure feels like the damage to student trust will fall back on the university.

The story above is a semi-fictional speculative account of potential harms that can come from fourth-party vendor relationships, in which outside companies partner with one another with little oversight by the educational institution. Ross (2017) explains how “speculative methods are particularly important for the study and analysis of digital education because of its rapidly changing nature, and the need to anticipate potential ‘unintended consequences’ of such rapid changes.” We based this story on real events that we experienced in our own professional context and the documented experiences of other students, faculty, and staff, which are reviewed in our remote proctoring Harm Index (Table 2). In this paper we will define fourth-party vendor relationships and investigate how the harms of remote proctoring technology are particularly pernicious in the context of a fourth-party deal.

We describe the nature of different types of fourth-party relationships with a focus on the remote proctoring company Proctorio’s partnerships. Our decision to focus on Proctorio is due to our practical experience working with the product through an integration with McGraw Hill Connect, a program that provides electronic textbooks and other learning materials. This paper adds to the larger body of literature that takes a critical view of educational technologies, in that these technologies often act as “mechanisms of economic capture, surveillance, and control” (Paris et al. 2021) and often work against the core pedagogical goals of educational institutions. In that institutions of higher education have a responsibility to treat students’ data responsibly and monitor the data privacy practices of vendors with access to student data, we suggest that fourth-party partnerships obstruct educational institutions’ regular oversight of student information.

Methodologies

Literature review

As remote proctoring technologies are relatively new and knowledge about them and their harms is rapidly evolving, the literature we have reviewed includes not just scholarly works but also news stories, especially accounts from students who have been exposed to remote proctoring technology in educational settings. Because we aim to define and detail the harms of these technologies, both generally and in the context of fourth-party deals, we found it necessary to focus on literature by and about students, faculty, and staff who had been negatively impacted by these technologies.

Content analysis

Our methodology includes content analysis of Proctorio’s website, help documentation, and publicly available information about their partnerships with other education technology companies (Hsieh and Shannon 2005). We analyze these materials to understand the nature of the relationships between Proctorio and its partner companies. The appendix outlines the documents analyzed in our content analysis.

Authoethnography

We use collaborative autoethnography to analyze our experience identifying the existence of a fourth-party surveillance company operating on our university campus, and requesting that the third party restrict the fourth-party from working with our campus users (faculty and students). Autoethnography is a method used to systematically analyze and interpret personal experiences (Ellis, Adams, and Bochner 2011). Collaborative autoethnography enables researchers to work together to combine their shared and personal narratives to make meaning out of experiences (Chang, Ngunjiri, and Hernandez 2016). Our collaborative ethnography was constructed using alternating personal narratives, in which we engage in critical reflection on the experience of identifying and ultimately removing a fourth-party proctoring service from the campus where they are both employed as instructional designers. Our perspective is one of current instructional design and educational technology professionals in higher education, employed at University of Michigan–Dearborn (a small, undergraduate-focused campus of the U of M). Both of us were already opposed to remote proctoring for the reasons detailed in Table 2 before the events in our autoethnography unfolded and had participated in public conversation about the harms of remote proctoring on social media. We additionally co-authored a peer-reviewed paper with colleagues about authentic assessment as an alternative to remote proctoring and issued a press release with Fight for the Future about the paper and expressing our difficulties with the McGraw Hill partnership with Proctorio. These experiences shape our narrative in that we likely had more prior knowledge of the harms of remote proctoring and the potential roadblocks to removing it from our campus. As our narrative will show, the process was complex, frustrating, and time-consuming despite our prior knowledge and preparation.

Defining the Fourth-Party Relationship

Three circles show a kind of venn diagram. Circle 1 is labeled Originating Party (1st and 2nd party): Institution and Direct Constituents - Students/Faculty/Public. Circle 2 is overlapping with Circle 1 and is labeled The 3rd Party - Outsourced Vendors. The overlap between these two circles is meant to show their relationship and reads Contract negotiated by institution; support staff from institution. Circle 3 is positioned under Circle 2 and is overlapping Circle 2. The overlap between these two is meant to show their relationship and reads Contract negotiated between vendors. In the lower left of the image there is text that reads Lack of relationship between Originating Party and 4th Party(ies) with a large arrow pointing to the lack of overlap in circles between Circle 1 and Circle 3
Figure 1. Diagram of fourth party relationships.

Third party services are common in the educational technology industry. For example, instead of building a platform on which to administer online courses, a university will often contract with a company to provide a Learning Management System (LMS). The company hosts and maintains the software, and the university pays a fee to use it. A fourth-party relationship comes about when vendor A, with whom the institution has an established relationship, partners with vendor B with whom the institution does not have an established relationship (Aldoriso 2020). Language can be confusing here because the vendor may refer to the partners as their third-party partners. This is because they see themselves as the originating party. However, this is an erasure of the institution as the originating party.

This kind of vendor relationship in higher education is a problematic one in that it is once removed from a direct contractual relationship with the higher education institution. Contracts and or user agreements with institutions may simply note that external services may be offered through Vendor A’s product, but the use of Vendor B’s product is governed by Vendor B’s terms of use. Note that these clauses are general, referring to a hypothetical Vendor B, rather than a specific one. For example, McGraw Hill Education’s terms of use states,

The Products may contain links or connections to third-party websites, services, or other technology provided by third parties that are not owned or controlled by us (“Third-Party Services”). When you access Third-Party Services, you accept that there are risks in doing so, and that we are not responsible for such risks. A Third-Party Service may include or be accompanied by a separate service, license, privacy policy, or other agreement (“Third-Party Agreement”), in which case that Third-Party Service is provided solely under the terms of that separate Third-Party Agreement. (McGraw Hill Education 2020)

Here, we see McGraw Hill (a third party to the university) referring to other services as a third party. This is confusing because what they refer to as a third party in these terms of use is the same as what we refer to in this paper as a fourth party because we recognize the university as the originating party. Even though McGraw Hill’s “third parties” (fourth parties to the university) are yet unknown to the user, McGraw Hill has already absolved itself of responsibility for these products. While it is entirely possible that in the process of using an add on (such as a Proctorio to McGraw Hill) the user will have to accept the fourth parties terms of use, it is only the end user (likely a faculty member or student) that comes in contact with those terms, rather than a representative of the institution with expertise in learning technology and student data privacy.

Fourth-party partnerships almost always introduce a new feature or functionality to the third-party product. Depending on the product and where any particular individual exists in the university hierarchy, new features from this relationship may be perceived as desirable or neutral (Gogia 2021). However, these partnerships merit increased suspicion when the fourth-party product does not have an obvious connection to the third-party product, or does not have an educational value. While such spurious partnerships could be viewed as simple money-making opportunities, these relationships can have dangerous implications when they involve problematic technologies that have a history of harming students, such as remote proctoring, because these technologies make their way into classrooms with little institutional input or support.

Jones et al. (2020) argue that institutions of higher education are examples of “information fiduciaries” (Balkin 2016), meaning that they have a particular obligation to treat the data of their primary stakeholders (students) responsibly. Fourth-party agreements constitute a threat to educational institutions’ responsibility to their students in terms of data privacy because, as we describe above, they expose students and their information to technologies that may harm their students or operate with bias.

Doorways Between Companies and Institutions

There are many examples of university maintained systems that allow for the possibility of integration with third-party or fourth-party vendors. For example, Google Apps for Education allow for various integrations with their suite of tools and Zoom has a marketplace where vendors can be allowed to offer integrations. One common option for a university-maintained Learning Management System (LMS) has long been the possibility to use plug-ins, and more recently the Learn Tools Interoperability (LTI) protocol (Severance, Hanss, and Hardin 2010), for integrating tools. There is an important distinction between a “true” fourth-party partnership and an integration option with a university maintained system. In instances where there is a possibility for integration, a university has some direct influence to work with the vendor to negotiate and navigate university policies around such integrations; for instance data privacy or procurement protocols. In some way (either by a dedicated employee or by a university committee) the university works directly with the vendor. This is different from fourth-party relationships which are inherently relationships between vendors which bypass the university entirely. It is also important to note that these kinds of integrations can actually act as the mechanism through which fourth-party vendors can come in; considering that once a third party is integrated that third party can then easily partner with a fourth party.

Relationships between a third-party and a fourth-party vendor can vary widely depending on the business partnership but ultimately these relationships act as doors into the institution. Just as there are many different types of doors: french doors, sliding doors, split doors; which can be in multiple states (open, closed, ajar), there are many different kinds of relationships between third- and fourth-party vendors. It is important to understand these relationships as they can deeply alter the power dynamics that are at play. Below are several relevant examples of relationships that can apply to partnerships between educational technology companies:

    • Integration Possibility – an option for two technologies to work together when an institution has separate agreements with both of them. Often a precursor to a fourth -party integration but not necessarily a fourth-party integration in and of itself.
    • Free – the third party offers the fourth-party service or product for free within their system.
    • Freemium – the third party offers some basic functions of the fourth party’s service or product for free with an option to pay for more advanced features. The free product in the Freemium model may not be totally free for the user—they may “pay” using an alternative currency known as “mind share,” or the “development of awareness for the provider’s brand and the consideration for purchase of future commercial products and services” (Pujol 2010). The free version, while a product or service in itself priced at $0, also serves as a type of marketing device for the paid product.
    • Resell – the third party offers the fourth party’s service or product at a cost. The cost can be charged to the institution, but is also frequently placed on the student.

Identifying the Doors: A Content Analysis of Proctorio’s Fourth-Party Partnerships

The remote proctoring company Proctorio lists a number of partnerships with other companies on its website, which are all presented under the category of “assessment platforms.” Our content analysis of these partnerships is summarized in Table 1 and details of the sites reviewed can be found in the Appendix.

Partner Nature of relationship to Proctorio (free, freemium, resell, integration) How partner defines their business (assessment, learning) Pricing for students/institutions How partner describes what Proctorio does
TopHat Student Resell – Top Hat resells Proctorio by directly charging students Active Learning Platform Each student pays $30 for access to TopHat Pro and then $10 per course for access to Proctorio “Partnership ensures higher education institutions transitioning to remote teaching can preserve the integrity of their tests and exams.”
McGraw Hill Connect Freemium/Student Resell – McGraw Hill Connect provides a freemium Proctorio plan directly to instructors, resells a more advanced product directly to students Course Management and Adaptive Learning Free for all instructors to require for their students. Instructors can additionally require a premium option for $15 a semester paid by students “You’re in control. Ensure your course’s academic integrity.”
Cirrus Assessment Integration possibility – Integration is possible with separate agreement with Proctorio End to end assessments platform “To make use of Proctorio you should have an agreement with Proctorio. Once the agreement is final, Proctorio will share K&S Keys for Cirrus to setup. Once the K&S details are received by Cirrus it will take a maximum of 24 hours to setup.” “Recordings of the exam sessions can be viewed and cheating behaviour is automatically flagged by the AI.”
Ans* Integration possibility – Integration is possible with separate agreement with Proctorio “Ans* is designed to support paper, digital and hybrid examinations. With a click of a button, you can convert a digital test into a face-to-face exam and vice versa.” “When a licence has been acquired, Ans* will support you in setting up the configuration by following these steps:…” “By enabling the integration with Proctorio, the security of the administration of the exam is increased. Proctorio collects information that can be used to handle fraud procedures of your institution. With online proctoring, you’re able to administer exams in a more secure way anyplace, anywhere.”

“Within the gradebook, suspicious behavior of the student is flagged.”

Ascend Learning Integration possibility – Ascend can facilitate the integrations without a separate agreement with Proctorio Integrated software, assessment, and analytics solutions Unknown “Proctors are monitoring for odd or disruptive behavior. Do not engage in misconduct or disruption. If you do, you will be dismissed, and your exam will not be scored.”
Derivita Freemium – unclear how much is paid for extra features and who pays “Derivita is a first of its kind STEM technology platform and complete computer algebra system.” “Lockdown” at no cost, other features “Derivita has been fully integrated with Proctorio’s remote proctoring platform. This enables educators to administer STEM assignments within the LMS, using Derivita’s content and technology, while ensuring rigorous adherence to academic integrity standards.”
EvaExam Unknown – While the partnership with EvaExam is listed on Proctorio’s site, EvaExam’s site does not detail the cost structure or nature of the partnership Testing Platform Unknown “ID verification, recording video, audio, the participant’s screen, and any web traffic on the system used may be centrally controlled, automated, and are legally compliant with the additional Proctorio plugin.”
QuestionMark Unknown – likely freemium. Questionmark offers an AI proctoring service called “record and review” that is facilitated by Proctorio, but also offer a live remote proctoring service Assessment platform Unknown “The automated system observes and records the exam session on video, for potential review later. The system flags potential anomalies, such as a second person on screen. When the system flags an anomaly, the customer can review it or send it to Questionmark for inspection. This makes it harder for a test-taker to cheat or to copy the exam questions to pass onto others.”
Table 1. Overview of Proctorio partnerships with “Assessment Platforms.”

The Proctorio website groups all of these partnerships together as “assessment platforms,” a decision which obscures the significant differences between the services that the partner provides (e.g. textbooks, assessments, active learning) but also what the nature of the partnership is between the company and Proctorio. For example, at the time of this writing, Proctorio’s partnership with McGraw Hill Connect allows any instructor who has adopted a McGraw Hill text to enable Proctorio on student activities that are assigned through the Connect platform (and additionally to enable a paid premium product for students), while Proctorio can only be used on the Cirrus Assessment platform when an institution has a separate (paid) agreement with Proctorio—an agreement that likely requires some attention from someone at the university.

Loose Hinges: Inconsistent Messaging in Fourth Party Relationships

Fourth-party relationships can introduce confusion about the purpose and functionality of the products involved. Proctorio defines itself as a tool for promoting academic integrity, which gives instructors information that they can use to make their own decisions about whether cheating has occurred during a given assessment. However, because many instructors encounter and use Proctorio’s product through these other fourth-party products, they may receive all of their information about Proctorio through the help documentation of this other vendor. These Proctorio partners and resellers can describe the purpose of Proctorio’s product in their documentation in ways that are not entirely congruent with Procotrio’s stated purpose. For example, at the time of this writing, Cirrus Assessment’s help documentation page titled Integration with Remote Proctoring from Proctorio states that “Cheating behaviour is automatically detected by the AI” (see Appendix).

Screenshot of Cirrus Assessment website. Text reads: On Friday the 6th (last week) we released an integration with the remote proctoring solution of Proctorio. We've run several successful pilots this week and are now opening up the possibility of remote proctoring through Proctorio for all our customers. The integration is seamless and is fully automated. Highlighted text reads: Cheated behavior is detected by the built-in AI. (red arrow points to highlighted text) How does it work? Remote proctored exams cannot be started unless in a proctored mode.
Figure 2. Screenshot from Cirrus help documentation.

Although we strongly object to the idea that any behavior detected by an AI can represent “cheating behavior,” this statement from Cirrus is notably at odds even with Proctorio’s public statements about their product. On their FAQ page under the question “Does Proctorio utilize algorithmic decision making?” Proctorio states that “No,… Proctorio’s software does not perform any type of algorithmic decision making such as determining if a breach of exam integrity has occurred. All decisions regarding exam integrity are left up to the exam administrator or institution” (see Appendix).

Screenshot showing text from Proctorio FAQ with Proctorio logo in upper left corner. Text reads No, Proctorio only uses algorithms for face and gaze detection if certain settings are enabled on an assessment. Proctorio's software (bolded) does not (end bolded text) perform any type of algorithmic decision making, such as determining if a breach of exam integrity has occurred. All decisions regarding exam integrity are left up to the exam administrator or institution.
Figure 3. Proctorio’s FAQ question about AI decision making.

Additionally on the Proctorio FAQ page for the question “How do you decide what behaviour counts as ‘cheating’?”, Proctorio themselves state, “Only the exam administrator or the institution can dictate what type of behaviour they want to monitor over the course of an exam. Exam administrators will then review exam attempts to determine whether any flagged behavior was truly infringing on the integrity of the exam” (see Appendix). Moreover, Proctorio’s own “acceptable use policy” prohibits punishment of students based solely on Proctorio reports. The policy states that “Institutions and their representatives are prohibited from making any negative decisions regarding exam integrity or from imposing any other negative consequence or detriment on an End User based partly or entirely on Proctorio’s analysis” (see Appendix).

Screenshot of Proctorio website showing the page URL and the Proctorio logo. Text on page reads Institutions and their representatives are prohibited from making any negative decisions regarding exam integrity or from imposing any other negative consequence or detriment on an End User based partly or entirely on Proctorio’s analysis. Institutions and their representatives must conduct their own independent, non-automated review, and analysis of any relevant data (including any available audio, video, screen recordings or images that were the subject of Proctorio’s analysis or that the Institution otherwise deems relevant) prior to imposing any such negative consequence or detriment
Figure 4. Excerpt from Proctorio’s Acceptable Use Policy.

Miscommunications like these about what the product is for and how it can be used, show how misleading fourth-party relationships can be. But while fourth-party partnerships are complex they are not inherently bad. Returning to our speculative narrative from the introduction, the math homework system we mention could integrate a digital graphing calculator from an outside vendor to assist students with their work with little negative consequence. However, when fourth-party deals are made with companies that offer harmful products these relationships become particularly problematic.

What’s the Harm?

Given that fourth-party integrations can expand the functionality of a third-party tool, often for free, what is the harm? Why would some schools not want free access to more functions? To explore this we review literature around the various harms that have been caused by remote proctoring systems. Reports of the many harms of remote proctoring systems have been widely documented especially since the beginning of the COVID-19 pandemic. Walker et al. (2020) outlined harms of remote proctoring technologies specific to nursing students, the nursing profession, and to the public. To address the important question of “What is the harm?” we have indexed harms from a broad perspective to create our Harm Index of Remote Proctoring Systems, which we present in Table 2. Here we are exploring three different levels of potential harm: harm to the student, harm to the institution, and harm to larger society.

Legend:
Harm to student – *
Harm to institution – +
Harm to larger society – ^
Data Security

  • Some remote proctoring companies store student recordings for years (Gogia 2020) *.
  • Remote proctoring companies have experienced data breaches (Abrams 2020)*.
  • Teaching students to install software that undermines the integrity of their computing environment can ingrain poor data security habits (Fox Cahn et al. 2020)*^.
Does Not Prevent Cheating/There are better kinds of assessments

  • Students have found ways around proctoring software and publicize these hacks (Geiger 2021)+.
  • Real-life tasks are a better assessment of real-life skills – students are assessed the the most effective ways (Crosslin 2021; Feathers 2020b; Silverman et al. 2021)*.
Test Anxiety

  • Remote proctoring systems can raise student anxiety (Chin 2020)*.
  • Student performance can suffer when they have test anxiety (Woldeab and Brothen 2019)*.
Accessibility

  • Remote proctoring features may not be compatible with adaptive technologies such as screen readers(Office of Information Technology 2021)*+.
  • Basic access to these technologies are often difficult (Feathers 2020a)*+.
Bias – Race, Ability, Gender

  • Those with certain kinds of disabilities can trigger cheating flags of no fault of their own – tics, eye movements, self-massage, needing to go to the bathroom (Brown 2020)*+^.
  • Flagging these behaviors is a feature not a bug of AI proctoring systems – they are designed to look for “atypical” behavior (Patil and Bromwich 2020) *+^.
  • Algorithmic proctoring uses facial recognition/detection technology which can fail to recognize those with dark skin (Clark 2021)*+^.
  • Students can be locked out of exams if a face is not detected in the frame (Chin 2021)*.
  • Reaching out to support can lead to degrading practices to “troubleshoot the problem” like being asked to shine a light on your face (Caplan-Bricker 2021)*.
  • AI identification methods can be compromising for trans and non-binary students (Swauger 2020)*+ .
Invasion of Privacy

  • Room scans are invasive and intrusive – they can reveal personal information the student doesn’t wish to share (Harwell 2020a)*.
  • Product features sometimes ask students to show parts of their bodies (their lap) in inappropriate ways (Harris 2020) *+.
Cost

  • Proctoring can cost upwards of $500,000 a year (Harwell 2020b)+.
  • High costs of proctoring borne by students or budget-squeezed institutions (Malone 2019; Wan 2019)+^.
  • Additional costs from the possibility of court cases and public relations (McKenzie 2021)+.
Liability

  • Proctoring services may not always be in compliance with state or local laws about student surveillance or collection of biometric data (Long 2021)*+^.
  • Human proctors are “alone” with students and may harass or otherwise harm them while the student is involved in course activities (Bhat 2021)*+.
Digital Divide/Digital Redlining

  • Many remote proctoring technologies require expensive hardware (laptop, webcam, microphone) that students may not have or software (a certain browser, a browser extension) that students may not consent to installing (Selinger and Gilliard 2021; Yun 2020)*.
  • Internet bandwidth is not the same everywhere and some students may struggle with connections (Flaherty 2021)*.
Larger harms to freedoms and society

  • Chilling effect on academic freedom – regarding research and choice in teaching
    • Australian researchers found that their research was hindered due to the litigious nature of proctoring companies and the larger negative climate around remote proctoring (Selwyn et al. 2021) +^.
    • Academic Integrity Researcher Phillip Dawson had to return grant funds because he could not find a remote proctoring company that would let him research their tool to see if it actually prevented cheating (CRADLEdeakin 2020)+^.
    • Some instructors are not given a choice about using this technology *+^.
  • Normalization of surveillance on students and faculty
    • Surveillance technologies are used in conjunction with human rights violations all over the word – proctoring normalizes surveillance for students (Fox Cahn et al. 2020)*^.
    • Short distance from surveillance of students in learning activities to surveillance of faculty during official university business (teaching, communications, etc.) (@hypervisible 2020)*+^
  • Degrading Trust
    • Remote proctoring systems have implications for eroding student trust (Stewart 2020)*+^.
Table 2. Harm index of remote proctoring systems.

Closing the Backdoor

The following is an autoethnographic reflection between us (the two authors) about our experience discovering and investigating a fourth-party proctoring option at our university, which had a stated anti-proctoring technology stance (Silverman et al. 2021). These events took place between February 5th and May 15th of 2021 and it is of importance to note this is the time it took to have a technology removed that was never vetted or approved by the university in any way. McGraw Hill’s partnership with Proctorio was established as early as February of 2020 (@mheducation 2020) potentially in response to the mass transition to online teaching during the COVID-19 pandemic. However, as this autoethnography will show, no one at the institution seemed to have any knowledge of it for a year. The partnership began to draw wider attention when a parent’s group published an open letter asking McGraw Hill to end their partnership with Proctorio (Ongweso 2020).

Autumm: In February of 2021 I was browsing twitter when I came across conversations about a McGraw Hill Connect (MHConnect) partnership with Proctorio. I did some googling and came across their webpage outlining this partnership and stating that the proctoring options would be part of any text with a 2019 copyright or newer. Days prior, I had been in a Canvas administrators meeting for our institution where the MHConnect integration had been discussed. It was noted that the new integration was mostly a “pass through” over to the publisher’s private platform, which they controlled. I had thought this was a good thing at first, that perhaps it meant that less of our student’s data would be passing between systems but then realized when I saw the Proctorio partnership that this would mean that our students could be subjected to this harmful technology without much oversight from our institution. This upset me for personal reasons but also because our campus had specifically rejected remote proctoring since the beginning of the pandemic. I reached out to Sarah on Twitter DM to vent.

Sarah: When Autumm contacted me to tell me about the McGraw Hill Connect partnership with Proctorio and how it might affect our students I was first and foremost angry. But then I immediately thought about our campus decision to reject remote proctoring (as Autumm mentions) and how that should carry some weight in our working relationship with McGraw Hill. After all, we are a customer and partner of theirs, and it is very common for customers of technology companies to ask to have the product configured to their liking (just take a look at several different universities’ instances of the Canvas LMS to see an example of edtech customization). I suggested to Autumm that we attempt to have Proctorio removed from MH Connect platform for UM-D users, assuming that this would be an easy request to fulfill.

Autumm: When I contacted Sarah I was feeling that we were somewhat at a dead end because of the nature of the integration that I had heard about in the admin meeting days prior. It did not sound like other integrations that I’d had experience with (having LMS admin experience from previous roles) that would bring outside functionality to the LMS—and over which an LMS admin would have more control. Rather, this integration just seemed to pass credentials over to another McGraw Hill controlled platform. Talking with Sarah she questioned my assumptions and asked many questions about configurations and customizations that might be possible. She made good points and I felt like it was a long shot but I reached out to our institution’s lead LMS administrator to point out this new partnership and question if customizations or a shut-off might be possible. He was not aware of this partnership but agreed that it was concerning and said that he would reach out to McGraw Hill. After waiting about two weeks and hearing nothing I reached back out to him and he said he would reach out again. Two days later he wrote to me to say he had met with representatives of McGraw Hill who said that they would turn the integration off for our school but that there were conditions: (1) that they wanted to look and see if there was anyone using the integration and (2) that they needed two weeks to turn it off. I was unhappy about the two week waiting period, especially since we had already waited two-weeks to initially get a response, and I never fully understood why it was required. Later that day one faculty member was identified as already using the integration and McGraw Hill additionally requested an email from the associate provost to perform the shut off. Arrangements were made to work with the faculty member who was using the system to find alternatives and the official email from the associate provost requesting the shut off went out. However, a month later Sarah was working with a faculty member who was using MHConnect and found that the options for proctoring were still there.

Sarah: That particular faculty member hadn’t wanted to use the proctoring feature but noticed that it became available. I think it is important to note that it was a complete coincidence that I was working with an instructor that was using MHConnect, and thus was able to verify that the proctoring feature had not been turned off. Absent my working relationship with this faculty member, none of the instructional designers on our campus would have access to Connect or any fourth-party tools that are connected to it. Together, Autumm relayed that the proctoring feature was not turned off to the LMS administrator and our associate provost, who contacted McGraw Hill again. At this point, we were assuming there was some sort of technical misunderstanding. We eventually heard back that while McGraw Hill could turn off the integration for users that accessed Connect through the LMS, it could not turn it off for those that logged in directly through the Connect site. Late in April I tweeted, “Frustrating day for resisting surveillance and e-proctoring. Found out that McGraw Hill cannot disable the Proctorio integration in “Connect” for all our users. This integration is built on the presumption that eproctoring is an uncomplicated value-add to any course. It is not.” Evidently, someone from McGraw Hill saw this tweet, and reached out to our associate provost by email offering to have a Zoom call in which we could clarify the details of McGraw Hill’s position.

Sarah and Autumm: We accepted the offer for a Zoom call, and decided to use it as an opportunity to better understand how Proctorio partners with McGraw Hill in addition to discussing how Proctorio could be removed for our campus users. We discussed a plan for McGraw Hill to deactivate Proctorio for all our campus users, both those that log in through our LMS and those that log in through the McGraw HIll Connect site. We then inquired as to whether the Data Privacy Agreement (DPA) we had signed with McGraw Hill covered other partnerships that they chose to make, such as the one with Proctorio. They responded that they viewed the DPA as applicable to any other technology companies that they partner with, meaning that our original DPA covered Proctorio being used through McGraw Hill Connect by our users. They also reiterated that they did not want to force Proctorio on anyone, and that they were happy to pursue various avenues to restrict its use on our campus if that is what we desired. In response to our dissatisfaction that they had integrated Proctorio into our users accounts without informing us or asking permission, they maintained that if we had a campus policy against remote proctoring, it was primarily our responsibility to inform faculty of the policy and enforce it.

Discussion and Recommendations

We have described the problematic nature of current fourth-party partnerships, but there is potential for more problematic future partnerships. Fourth-party partnerships may exist explicitly to circumvent campus decisions or policies (such as administrative policies, or faculty governance) or to respond to budget and purchasing constraints (Gogia 2021). The loopholes which are created as part of these deals can do real harm. Considering how we speak about and rationalize such technologies is an important part of analyzing how they end up existing on our campuses.

Caines (2021) described a weaponization of care around surveillance technologies where they are sold and rationalized under a rhetoric of care and gives specific examples from remote proctoring companies. Adjacent to this frame we also see remote proctoring companies using what Herzog (2010) called the “banality of surveillance.” With this construct we see companies making the case that the technology is essential and, though it may not be perfect, we need to suffer with the drawbacks because the good outweighs the bad (McFarland 2021). Third parties that form relationships with remote proctoring companies also implicitly make the case that remote proctoring is essential (and harmless) by integrating it into their product without user or institutional consent. But the banality of surveillance around remote proctoring is nothing but smoke and mirrors, as our Harm Index (Table 2) shows. The harms suffered by students, institutions, and larger constructs such as academic freedom from these technologies is very real. Additionally, multiple educators have pointed out that other, more authentic kinds of assessments do not even require exams (Crosslin 2021; Silverman et al. 2021).

Relationships between educational technology companies and educational institutions can be fraught. Often, there is a disconnect between how institutionally approved technologies are chosen and whether they support the kind of education the institution wants to provide (Cohn 2021). Our first recommendation for institutions to have more control over fourth-party integrations is for universities to consider not only which technologies to adopt, but which technologies, and specifically what functionalities, they do not want to adopt. Methods and tools need to be created to assist with alignment of the university’s mission, strategic plan, or other guiding principles for evaluating not just the benefits but also the harms of technologies. The addition of new functionalities to an existing educational technology tool through a fourth-party relationship can be particularly subtle and they often come with a techno-utopian sales rhetoric that fails to imagine that this addition could be anything but a good thing. Without consideration of where boundaries exist for the institution in regards to what technology is acceptable and what is not it is impossible to express issues with fourth-party relationships. Passing resolutions and offering recommendations against such technologies can go a long way against limiting the use of harmful technology.

Our second recommendation is for institutions to leverage their influence as direct paying customers, or as the provider of a sales environment of these tools, to demand that the third party remove the surveillance functions provided by the fourth-party companies. Enterprise systems are regularly configured to customers specifications. Shutting off these integrations is technically possible. Success in getting a fourth-party integration removed may vary depending on the specifics of the fourth-party partnership (as discussed above). We speculate that McGraw Hill was willing to grant our request for several reasons. For one, on balance it is better for them to retain us as a satisfied customer than to insist on proctoring functionality in our school. In addition, there is the name recognition of our institution and our relationship with our flagship university. Finally, it is important to note that we did take steps to bring public light on our situation. We wrote a peer-reviewed paper about our experiences and issued a press release (Fight for the Future 2021), which could have impacted our ability to successfully get this integration removed. While resisting harmful fourth-party integrations is a difficult, time-consuming, and unpredictable endeavor, we hope that other educators feel empowered to do so based on our experiences.

References

Abrams, Lawrence. 2020. “ProctorU Confirms Data Breach after Database Leaked Online.” Bleeping Computer. August 9, 2020. https://www.bleepingcomputer.com/news/security/proctoru-confirms-data-breach-after-database-leaked-online/.

Aldoriso, Jeff. 2020. “What Is Fourth-Party Vendor Risk Management? What You Need to Know.” SecurityScorecard. June 25, 2020. https://securityscorecard.com/blog/what-you-need-to-know-fourth-party-vendor-risk-management.

Bhat, Prajwal. 2021. “NMIMS Students Allege Harassment by Online Proctor during Exams.” The News Minute, January 10, 2021. https://www.thenewsminute.com/article/nmims-students-allege-harassment-online-proctor-during-exams-141207.

Brown, Lydia. 2020. “How Automated Test Proctoring Software Discriminates Against Disabled Students.” Center for Democracy and Technology. November 16, 2020. https://cdt.org/insights/how-automated-test-proctoring-software-discriminates-against-disabled-students/.

Caines, Autumm. 2021. “The Weaponization of Care.” Real Life Magazine. May 24, 2021. https://reallifemag.com/the-weaponization-of-care/.

Caplan-Bricker, Nora. 2021. “Is Online Test-Monitoring Here to Stay?” The New Yorker, May 27, 2021. https://www.newyorker.com/tech/annals-of-technology/is-online-test-monitoring-here-to-stay.

Chang, Heewon, Faith Ngunjiri, and Kathy-Ann C Hernandez. 2016. Collaborative Autoethnography. Routledge.

Chin, Monica. 2020. “Exam Anxiety: How Remote Test-Proctoring Is Creeping Students Out.” The Verge, April 29, 2020. https://www.theverge.com/2020/4/29/21232777/examity-remote-test-proctoring-online-class-education.

———. 2021. “ExamSoft’s Proctoring Software Has a Face-Detection Problem.” The Verge. January 5, 2021. https://www.theverge.com/2021/1/5/22215727/examsoft-online-exams-testing-facial-recognition-report.

Clark, Mitchell. 2021. “Students of Color Are Getting Flagged to Their Teachers Because Testing Software Can’t See Them.” The Verge, April 8, 2021. https://www.theverge.com/2021/4/8/22374386/proctorio-racial-bias-issues-opencv-facial-detection-schools-tests-remote-learning.

Cohn, Jenae. 2021. “Advice | Who Chooses What Ed Tech to Buy for the College Classroom?” The Chronicle of Higher Education. June 3, 2021. https://www.chronicle.com/article/who-chooses-what-ed-tech-to-buy-for-the-college-classroom.

CRADLEdeakin. 2020. Assessment in a Postdigital World. CRADLE Conference 2020: Symposium. https://www.youtube.com/watch?v=5PzmS-1AFBc&t=2248s.

Crosslin, Matt. 2021. “Proctoring Is Not Essential.” ELearningInside News. April 13, 2021. https://news.elearninginside.com/proctoring-is-not-essential/.

Ellis, Carolyn, Tony E Adams, and Arthur P Bochner. 2011. “Autoethnography: An Overview.” Historical Social Research/Historische Sozialforschung. JSTOR, 273–90.

Flaherty, Colleen. 2021. “U of Illinois Says Goodbye to Proctorio.” Inside Higher Ed, February 1, 2021. https://www.insidehighered.com/news/2021/02/01/u-illinois-says-goodbye-proctorio.

Feathers, Todd. 2020a. “Students Have To Jump Through Absurd Hoops To Use Exam Monitoring Software.” Motherboard – Tech by Vice, November 9, 2020. https://www.vice.com/en/article/88anxg/students-have-to-jump-through-absurd-hoops-to-use-exam-monitoring-software.

———. 2020b. “Colleges Say They Don’t Need Exam Surveillance Tools to Stop Cheating.” Motherboard – Tech by Vice, November 16. https://www.vice.com/en/article/88ag8z/colleges-say-they-dont-need-exam-surveillance-tools-to-stop-cheating.

Fight for the Future. 2021. “University Advocates E-Proctoring Alternatives, but Struggles to Remove e-Proctoring Option from McGraw-Hill Connect Platform.” Fight for the Future. April 1, 2021. https://www.fightforthefuture.org.

Fox Cahn, Albert, Caroline Magee, Eleni Manis, and Naz Akyol. 2020. “Snooping Where We Sleep: The Invasiveness and Bias of Remote Proctoring Services.” Surveillance Technology Oversight Project. https://static1.squarespace.com/static/5c1bfc7eee175995a4ceb638/t/5fd78bac79515d2e1fde4bb7/1607961518518/Snooping+Where+We+Sleep+Final.pdf.

Geiger, Gabriel. 2021. “Students Are Easily Cheating ‘State-of-the-Art’ Test Proctoring Tech.” Motherboard – Tech by Vice, May 5, 2021. https://www.vice.com/en/article/3an98j/students-are-easily-cheating-state-of-the-art-test-proctoring-tech.

Gogia, Laura. 2020. “Online Proctoring: A Call for Action for Academic Technology Leaders.” The Tambellini Group. October 26, 2020. https://www.thetambellinigroup.com/online-proctoring-a-call-for-action-for-academic-technology-leaders/.

———. 2021. “Hiding in Plain Sight: Online Proctoring as ‘Just Another Feature.’” The Tambellini Group. April 7, 2021. https://www.thetambellinigroup.com/hiding-in-plain-sight-online-proctoring-as-just-another-feature/.

Harris, Margot. 2020. “A Student Says Test Proctoring AI Flagged Her as Cheating When She Read a Question out Loud. Others Say the Software Could Have More Dire Consequences.” Insider. October 4, 2020. https://www.insider.com/viral-tiktok-student-fails-exam-after-ai-software-flags-cheating-2020-10.

Harwell, Drew. 2020a. “Mass School Closures in the Wake of the Coronavirus Are Driving a New Wave of Student Surveillance.” Washington Post, April 1, 2020. https://www.washingtonpost.com/technology/2020/04/01/online-proctoring-college-exams-coronavirus/.

———. 2020b. “Cheating-Detection Companies Made Millions during the Pandemic. Now Students Are Fighting Back.” Washington Post, December 12, 2020. https://www.washingtonpost.com/technology/2020/11/12/test-monitoring-student-revolt/.

Herzog, Todd. 2010. “The Banality of Surveillance: Michael Haneke’s Cache and Life after the End of Privacy.” Modern Austrian Literature 43 (2). Austrian Studies Association: 25–41.

Hsieh, Hsiu-Fang, and Sarah E. Shannon. 2005. “Three Approaches to Qualitative Content Analysis.” Qualitative Health Research 15, no. 9: 1277–88. doi:10.1177/1049732305276687.

@hypervisible. 2020. “End of Essay Brings up Important Point. There Are Countless Reasons Instructors Should Reject These Surveillance Systems, but One Is Self-Preservation. It’s Only a Matter of Time before These Systems Are Turned on Instructors—in Some Cases They Already Are. Https://T.Co/IPlGFAMm0V.” Tweet. Twitter. September 11, 2020. https://twitter.com/hypervisible/status/1304394596902993924.

Long, Waverly. 2021. “NU Faces Lawsuit for Storing Students’ Biometric Data.” The Daily Northwestern, February 19, 2021. https://dailynorthwestern.com/2021/02/18/campus/nu-faces-lawsuit-for-improperly-capturing-and-storing-students-biometric-data/.

Malone, Grace. 2019. “Students Required to Pay for Online Exam Proctoring Service.” The Daily Illini, October 28, 2019. https://dailyillini.com/news/2019/10/28/students-required-to-pay-for-online-exam-proctoring-service/.

McFarland, Scott. 2021. “Proctoring Online Exams or Tests Is Essential.” ELearningInside News. March 23, 2021. https://news.elearninginside.com/proctoring-online-exams-or-tests-is-essential/.

McGraw Hill Education. 2020. “Terms of Use: Consumer Purchase Terms.” December 1, 2020. https://www.mheducation.com/terms-use.html.

McKenzie, Lindsay. 2021. “DePaul Sued Over Online Proctoring Tool.” Inside Higher Ed, March 11, 2021. https://www.insidehighered.com/quicktakes/2021/03/11/depaul-sued-over-online-proctoring-tool.

@mheducation. 2020. “Thanks to a New Partnership with @proctorio, Our Connect Platform Now Has Additional Capabilities – like Browser Locking and Remote Proctoring – That Support Academic Integrity and Assessment Security. Read More: https://t.co/mDzCIbKJay.” Tweet. February 4, 2020. https://twitter.com/mheducation/status/1224797280462692352.

Office of Information Technology. 2021. “Proctorio – Accessibility.” University of Colorado – Boulder. Accessed June 5, 2021. https://oit.colorado.edu/services/teaching-learning-applications/proctorio/accessibility.

Ongweso, Edward Jr. 2020. “2,000 Parents Demand Major Academic Publisher Drop Proctorio Surveillance Tech.” Motherboard – Tech by Vice, December 21, 2020. https://www.vice.com/en/article/88am8k/2000-parents-demand-major-academic-publisher-drop-proctorio-surveillance-tech.

Patil, Anushka, and Jonah Engel Bromwich. 2020. “How It Feels When Software Watches You Take Tests.” The New York Times, September 29, 2020. https://www.nytimes.com/2020/09/29/style/testing-schools-proctorio.html.

Proctorio. n.d. “Frequently Asked Questions.” https://proctorio.com/faq.

Pujol, Nicolas. 2010. “Freemium: Attributes of an Emerging Business Model.” Available at SSRN 1718663.

Selinger, Evan, and Chris Gilliard. 2021. “Abolish A.I. Proctoring.” Medium. OneZero. April 7, 2021. https://onezero.medium.com/abolish-a-i-proctoring-c9e017dd764f.

Selwyn, Neil, Chris O’Neill, Gavin Smith, Mark Andrejevic, and Xin Gu. 2021. “A Necessary Evil? The Rise of Online Exam Proctoring in Australian Universities.” Media International Australia, April 1. doi:10.1177/1329878X211005862.

Severance, Charles, Ted Hanss, and Joseph Hardin. 2010. “IMS Learning Tools Interoperability: Enabling a Mash-up Approach to Teaching and Learning Tools.” Technology, Instruction, Cognition and Learning 7 (3–4): 245–62.

Silverman, Sarah, Autumm Caines, Christopher Casey, Belen Garcia de Hurtado, Jessica Riviere, Alfonso Sintjago, and Carla Vecchiola. 2021. “What Happens When You Close the Door on Remote Proctoring? Moving Toward Authentic Assessments with a People-Centered Approach.” To Improve the Academy: A Journal of Educational Development 39, no. 3.

Stewart, Bonnie. 2020. “Online Exam Monitoring Can Invade Privacy and Erode Trust at Universities.” The Conversation, December 3, 2020. http://theconversation.com/online-exam-monitoring-can-invade-privacy-and-erode-trust-at-universities-149335.

Swauger, Shea. 2020. “Our Bodies Encoded: Algorithmic Test Proctoring in Higher Education.” Hybrid Pedagogy, April 2, 2020. https://hybridpedagogy.org/our-bodies-encoded-algorithmic-test-proctoring-in-higher-education/.

Walker, Rachel K., Jessica Dillard-Wright, Em Rabelais, and Anna M. Valdez. 2020. “Surveillant #EdTech Harms Nursing Students, the Profession, and the Public.” Medium. December 21, 2020. https://rachelkwalker.medium.com/surveillant-edtech-harms-nursing-students-the-profession-and-the-public-6b225c57a7b3.

Wan, Tony. 2019. “As Online Learning Grows, So Will Proctors. Case in Point: Examity’s $90M Deal.” EdSurge. April 30, 2019. https://www.edsurge.com/news/2019-04-30-as-online-learning-grows-so-will-remote-proctors-case-in-point-examity-s-90m-deal.

Woldeab, Daniel, and Thomas Brothen. 2019. “View of 21st Century Assessment: Online Proctoring, Test Anxiety, and Student Performance | International Journal of E-Learning & Distance Education / Revue Internationale Du e-Learning et La Formation à Distance.” International Journal of E-Learning & Distance Education 34, no. 1. http://www.ijede.ca/index.php/jde/article/view/1106/1727.

Yun, Tom. 2020. “Math Students at Wilfrid Laurier Furious after Department Orders Them to Buy External Webcams for Exams.” Toronto Star. May 8, 2020. https://www.thestar.com/news/gta/2020/05/08/math-students-at-wilfred-laurier-furious-after-department-orders-them-to-buy-external-webcams-for-exams.html.

Appendix: Documents Analyzed from Proctorio’s Site and Partner Sites

Company Name Page Name URL
Proctorio Integration https://proctorio.com/about/integration

Archived https://web.archive.org/web/20210617040222/https://proctorio.com/about/integration

Proctorio Frequently Asked Questions https://proctorio.com/faq

Archived https://web.archive.org/web/20210617040359/https://proctorio.com/faq

Proctorio Acceptable Use Policy https://proctorio.com/policies#all&all&aup&section-acceptableusepolicy
TopHat TopHat Partners with Proctorio https://tophat.com/press-releases/top-hat-partners-with-proctorio/

Archived https://web.archive.org/web/20210310041915/https://tophat.com/press-releases/top-hat-partners-with-proctorio/

TopHat Professor: Remotely Monitored (Proctorio) Tests https://success.tophat.com/s/article/Teaching-Online-Remotely-Proctored-Tests

Does not archive properly

TopHat Choose the Right Plan for Your Course https://tophat.com/pricing/

https://web.archive.org/web/20210414132914/https://tophat.com/pricing/

McGraw Hill Connect Online Assessment Integrity https://www.mheducation.com/highered/connect/proctorio.html

Archived https://web.archive.org/web/20210616055843/https://www.mheducation.com/highered/connect/proctorio.html

Cirrus Assessments Integration with Remote Proctoring from Proctorio https://cirrus.zendesk.com/hc/en-us/articles/360012584799-Integration-with-remote-proctoring-from-Proctorio

Archived https://web.archive.org/web/20210617040838/https://cirrus.zendesk.com/hc/en-us/articles/360012584799-Integration-with-remote-proctoring-from-Proctorio

Cirrus Assessments Automated Proctoring by Proctorio https://cirrus.zendesk.com/hc/en-us/articles/360012590719-Automated-proctoring-by-Proctorio

Archived https://web.archive.org/web/20210617041251/https://cirrus.zendesk.com/hc/en-us/articles/360012590719-Automated-proctoring-by-Proctorio

Ans* Set up the Proctorio Integration https://support.ans.app/hc/en-us/articles/360011850058–Set-up-the-Proctorio-integration

Archived https://web.archive.org/web/20210301032302/https://support.ans.app/hc/en-us/articles/360011850058-Set-up-the-Proctorio-integration

Ascend Learning The ATI TEAS Exam with Proctorio https://www.atitesting.com/teas/the-ati-teas-exam-with-proctorio

Archived https://web.archive.org/web/20210413071454/https://atitesting.com/teas/the-ati-teas-exam-with-proctorio/

Ascend Learning About https://www.ascendlearning.com/about/

Archived
https://web.archive.org/web/20200923134618/https://www.ascendlearning.com/about/

Derivita Math with Integrity https://www.derivita.com/proctoring

Unable to be archived

Derivita Home https://www.derivita.com/

Trouble archiving https://web.archive.org/web/20201031103007/https://www.derivita.com/

EvaExam Online Exams https://evasys.de/en/online-exams/

Trouble archiving https://web.archive.org/web/20210617041927/https://evasys.de/en/online-exams/

QuestionMark Frequently Asked Questions https://www.questionmark.com/platform-services/faqs/
QuestionMark How do I take a “record and review” proctored assignment? https://support.questionmark.com/content/taking-a-record-and-review-proctoring-assessment
QuestionMark Proctoring Hub https://www.questionmark.com/platform-services/proctoring/

About the Authors

Sarah Silverman is an instructional designer at the Hub for Teaching and Learning Resources at the University of Michigan–Dearborn. In addition to educational technology criticism, her interests include Universal Design for Learning and Disability Studies. A scientist by training, she received her PhD in Entomology from UC Davis and worked in teaching and learning support at UC Davis and UW Madison before coming to UM Dearborn. She currently resides in New Haven, CT.

Autumm Caines is an instructional designer at the University of Michigan–Dearborn. Autumm’s scholarly and research interests include blended/hybrid and online learning, open education, digital literacy/citizenship with a focus on equity and access, and online community development. This blend of interests has led to a concern about mounting ethical issues in educational technology and recent publications and presentations on topics concerning educational surveillance, student data collection, and remote proctoring. Autumm has taught honors students at small liberal arts colleges as well as traditional students, working professionals, and veterans at a regional public university. More at autumm.org.

Images are for demo purposes only and are properties of their respective owners. ROMA by ThunderThemes.net

Skip to toolbar