Trace: Imagining COVID on Campus

The Trace Project is a collaborative, exploratory, creative data visualization/materialization project created by Nikki Stevens, Christiana Rose, and Jacque Wernimont. Jointly supported by Dartmouth’s Neukom Institute and the Digital Humanities and Social Engagement Cluster, Trace was initially conceived as an opportunity to understand more about three dimensional data representation. With the outbreak of the COVID pandemic, we’ve had to adjust our original work plan and have been working on developing ways of sharing what was meant to be an embodied experience in a more distributed, individual way.

In addition to needing to reimagine how people would interact with Trace, we also felt the need to address our work to the current public health and cultural crisis. Thus, Trace: Imagining COVID on Campus. Using Processing we’ve created a short creative data film in order to imagine how COVID would spread on the Dartmouth campus if this were a normal fall (using last year’s data as our source). This is not an epidemiological simulation, but rather a speculative rendering that is constrained by the nature of the data (swipe dependent, historical) and our own commitments to non-representational campus geographies as a way of furthering privacy protections.

We will be continuing to work on additional ways of sharing the larger explorations of privacy, data visualization, and Dartmouth community movements as part of Trace. In the meantime, you can read more about the initial impetus for the project below. If you’d like to share feedback, you can tweet or email Jacque.

The Trace Project began with a conversation exploring Charlotte Kuller’s (‘20) interest in campus data collection, which was itself an extension of conversations we had in a Dartmouth course: Data and Bodies. Inspired by those conversations the Digital Justice Lab proposed Trace as part of the Neukom CompEx grant program. As we described it in the proposal – we are exploring the use of high performance computing, large-scale activity datasets, data privacy, and creative visualization.

There is ever-growing interest in data ethics and privacy and also in robust and compelling data visualization and this project thinks specifically about the affordances of media arts to render potentially sensitive data in ways that protect privacy. In the tradition of critical digital humanities and design, we plan in the final outcomes of the project to account for both the energy and currency costs of the project itself in order to ask “are the insights of this project worth the costs incurred?”. While there is a significant body of literature that deals with speculative fiction, speculative design, and speculative engineering, our concept of “speculative accounting,” which works to understand costs and income streams of possible future data deployments, is novel.

This project uses an 118 million record data set of anonymized activity in the Dartmouth ecosystem between June 2018 and December 2019. Using a variety of machine-learning tools and techniques, we plan to develop an interactive installation using a series of suspended individually addressable LED lights. One of the affordances of the individually addressable lights is that we can create a dynamic three dimensional but non-representational “map” of activity that people explore in both time and space. To help understand the costs of this work, we will synthesize the accountings from both prior phases to develop a speculative accounting methodology that allows both the program team and our viewers to evaluate the energy and fiscal resources that will be used to generate work like “Traces.” We plan to create a speculative “balance sheet” that will both render visible the known costs and speculate about future storage and usage costs for this kind of work. This will function as a prompt for viewers to consider the value of computational data analysis, large scale data storage, and new media installations which is often taken as a given.

Covid Black partnership

COVID Black (Purdue University), a Black Digital Humanities (Black DH) collective and an early response taskforce on Black health and data and the Digital Justice Lab (part of Dartmouth’s Digital Humanities and Social Engagement Cluster) are pleased to announce a strategic collaboration to develop several small-scale digital projects that will address Black communities impacted by COVID-19. By connecting COVID Black’s expertise in theories and methods around data and the Black lived experience with the Digital Justice Lab’s expertise in computational tools, the collaboration represents a progressive and community-centered means for effecting positive change in how Black health data is collected and disseminated. We hope that the joint-effort between COVID Black and the Digital Justice Lab will serve as a model for other cross-organizational collaborations around a shared vision and common goals.

For more about COVID Black please read here 

Eugenic Rubicon project update

by Caroline Casey

As the summer draws to a close, I am wrapping up research I have contributed to the Eugenic Rubicon project under the direction of Dr. Jacque Wernimont and Dr. Meredith B. Ferguson at Dartmouth College. At Dartmouth, I am an undergraduate Quantitative Social Science major with an interest in the human dimensions of data. I hope to contribute to research that promotes health care access and holds power to account, so I was naturally drawn to Jacque’s project. Using my background in statistics and skill in R, my role this summer was to study the over 20,000 available records of patients who were sterilized in California from 1919 to 2014. My goal in this work is to balance the desire to summarize these records in a cohesive manner with the imperative to treat every record as the violent ethics violation that it was so as not to flatten the data in any way that minimizes what it represents. In providing a detailed account of my work on this project and the rationale behind my decisions, I hope to emphasize that my visualizations are one of many, many imperfect ways to represent this data and to highlight the tradeoffs associated with visualization. With that being said, I am excited about the visualizations I created this summer and hope that they draw attention to the history of sterilization in the United States.

Documented Reason for Sterilization by Year
Figure 2: Eugenics Rubicon data visualization
Figure 3: Eugenics Rubicon data visualization

The data I worked with in this project came from hand transcriptions of records found in California by xxx which were inputted into Redcap by a group of undergraduate researchers at the University of Michigan. I am not personally too familiar with Redcap because I never actually used it during my research, but from what I know it is a program that allows for research of confidential documents such as medical records by deidentifying data and phase shifting dates. The data that I used was a csv file from Redcap of deidentified data from the California files.

The data as I received it from Jacque was not particularly suitable for data visualization because it was not “tidy”. This means that each possible observation for one variable was spread out in a column of its own as opposed to one column that says ‘variable’ that is complete with the specific observation. I had to “tidy” this data before I could use it for the manipulations I wanted to do in R. I also renamed some of the variables which initially had long names. For example, the observations for who consented to sterilization were of the form “Who.consented. . . .choice.Mother.” and I changed them to the form “Mother.”

I also found some errors in the years entered for some of the records. The vast majority of the records were from 1919 to 1952, however there were a total of 8 entries from 922, 937, 939, 1022, 1834, and 4925. The one entry from 1834 may have been correct, however that would be surprising since would have been almost 100 years before the next record entry. The other entries, while incorrect, were plausible mistakes (922 probably meant 1922, for example), so I chose not to remove them from the dataset. Instead, I left them in for visualizations that were not dependent on year and subset my observations based on year to 1919 to 1952. When I did this, I chose to leave out 16 total observations that occurred in 1960, 1985, 1992, 1993, 1998, and 2014. This was to prevent distortion caused by large amounts of whitespace between years when I first visualized the data with these years, as shown below.

Diagnosis Share

While I believe this was the best way to visualize changes over time, it does leave some records out. I didn’t have enough time to this summer, but I would love to create a more qualitative summary of these later records in the future.

1935 Analysis

To decide what to focus on in 1935, I looked at the sterilizations by institution in 1934 and 1935. When I looked at these years, I saw that Sonoma and Patton had the most sterilizations in 1935 by far, followed by Napa and Stockton. All of those institutions had dramatic increases in sterilizations from 1934 to 1935.

Then I looked at superintendents for those years to see if there were any specific superintendents who oversaw a large number of sterilizations, particularly any new superintendents. G.M. Webster oversaw 973 sterilizations in 1935, followed by F.O. Butler who oversaw 664 sterilizations in 1935. I was stunned by the number of sterilizations that both of them oversaw, and it made me wonder what they were diagnosing in 1935 and if it was different than what they diagnosed in 1934.

When I looked at the number of diagnoses under G.M. Webster from 1934 to 1935, I saw that he oversaw the diagnosis of “Hebephrenic” for the first time in 1935 and there were 245 people with that. I also saw that the number of people diagnosed with “Dementia” under him increased from 18 to 373 from 1934 to 1935. When I looked at the data on the sterilizations Butler oversaw, I noticed that his diagnoses of “Feebleminded” increased from 108 to 312 from 1934 to 1935, and that his diagnoses of “Physically Negative” increased by 118 between those years. I have many questions about how and why the diagnoses changed that much in that year. I would love to do more research on these diagnoses as well as G.M. Webster and F.O. Butler. With the data, however, I decided to visualize the changes in the 1930s for all diagnoses.

I created a few figures based on this information to visualize how diagnoses changed in the 1930s. These figures clearly show the dramatic increases in diagnoses of “Hebephrenic”, “Dementia”, and “Feebleminded”, as well as the increase in the number of “Physically Negative” diagnoses. These visualizations show that G.M. Webster and F.O. Butler’s changes in the number of sterilizations they oversaw stand out in the data for that year. They show a peak in all diagnoses for 1935, indicating that 1935 was an important year beyond the changes in the actions of these two superintendents.

Eugenic Rubicon Project Blog

by Kirby Phares

Over the 2019 Summer Term at Dartmouth College, I worked on the Eugenic Rubicon project under Dr. Jaqueline Wernimont and Dr. Meredith Ferguson. Drawing upon sterilization records, the Eugenic Rubicon project seeks to create a widely accessible digital interface which will tell the story of forced sterilization in America during the 20th century. The first half summer, I focused on gathering records of HIPPA laws for each state in the United States and researching the archived records of Vermont and New Hampshire state institutions which performed forced sterilization. I then transferred my focus to analyzing the dataset of 30,000 sterilization records and creating visualizations to best represent that data.

At the start of the summer, Caroline, Professor Wernimont, and I met to discuss what we might be interested in working on this summer. Having never researched before this summer, the question seemed unanswerable.

When beginning work on research such as Eugenic Rubicon as a student, it is overwhelming to work around the people in the lab and work with the material every day. My knowledge of the subject grew tenfold simply from reading the grant proposal which is why it is so hard to know what to do because I don’t know what is important.

To introduce myself to the difficulties of research and the prospective impact it can have, I began tracking down state HIPAA laws regarding the disclosure of an individual’s medical records after death. There is a federal law which mandates that the state can release medical records fifty years after the death of the individual. However, this only serves as a baseline such that states have the power to lengthen the amount of time until the time of release. I was assigned the second half of the fifty states, spent almost two weeks researching and found definitive and complete records for four of the twenty-five. Through this work, I discovered the extreme disorganization of government documents and lack of transparency. In many cases, laws restated the HIPPA Privacy Rule verbatim, yet there was no mention of the timeline for releasing medical records. The frustration of this task stemmed from the question of whether the state defaults to HIPPA or if I was not “looking hard enough.” After a few weeks, I transitioned from searching the websites of each states’ laws to calling the offices of state medical boards. I only received a response from the Vermont Board of Medical Practice.

In mid-July, I was introduced to the data set of the sterilization records. During a team meeting, we scrolled through the extensive variables which the data set accounts for. For many of the questions, there was simply a check-box answer and missing was an explanation. To see that people were sterilized by a quick check of a box listed “Dementia” or “Alcoholic” without any extensive rationale for why the decision for sterilization was made was incredibly troubling for me. My past experiences with data analysis, I was given mundane data sets, like election results or football statistics, and wrangle them to ultimately find “meaning” through a correlation of visualization. When you regularly work with large data sets, it is easy to get desensitized to what is behind the numbers. But as I looked through the data to find what I wanted to focus on, I found myself shying away from looking at each patient because I could not find a way to tell their story the way I felt they deserved. Therefore, I turned to who I saw as the “enemy”: the superintendents. I decided to try to analyze each superintendent and the choices they made concerning the sterilizations. I found that some superintendents tended to be drawn towards certain diagnoses, such as feeblemindedness. I found this striking because I could not make a logical conclusion that one institution just happened to have more feebleminded patients than any other.

Initially, this seemed like a simple visualization, however, as I took a closer look at the data, I realized that one superintendent could have multiple aliases, in a sense. Therefore, different bits of data were spread among different variations of one individual. For example, N.E. Williamson was listed under multiple names and thus although he altogether had 240 patients, “N.E. Williamson” would have only accounted for 40.

Figure 1: Eugenics Rubicon data visualization

These inconsistencies were a result of human input, as the name of the superintendent was often written on each patient record. To fix this, I used regular expressions so that names could be quickly recognized as the same person and then all of that information could be attributed to one individual. The final figure shows the count of different diagnoses each superintendent signed off on during their time at the institution. To decrease clutter, I only included superintendents with more than 100,000 patients. The number of diagnoses exceeds expected values in which they should total to 30,000, but some patients had more than one diagnosis.

Occurences of diagnoses

The second figure was inspired by the superintendent figure because I noticed a significant proportion of diagnoses across almost superintendents was “Other”. All summer I tried to distance myself from the data, but the instance of “Other” completely occupied my thoughts. The list of specific diagnoses is in no way limited, yet for a significant number of individuals, “Other” is the reason they were sterilized. The figure shows “Other” compared to other diagnoses with high frequencies. The institutions shown are the top three institutions with the greatest number of patients or more than 100,000 over the years 1920- 1960.

Video Games Have Always Been Queer

by Isis Canti

Poster from the Queer Video Games event; text reads "video games have always been queer"

In March of this year, the Digital Humanities and Social Engagement (DHSE) lab were delighted to have welcomed Professor Bonnie ‘Bo’ Ruberg (Pronouns: they/them), who teaches at UC Irvine’s Department of informatics. Drawing from intersectional feminist frameworks, Ruberg’s work promotes social justice in and through digital media. They seek out to promote diversity in technology as related to pressing issue surrounding computing today. Their background in the humanities, technology reporting, and community activism allows for a multidisciplinary exploration of cultural implications of technology. During the talk “Video Games Have Always Been Queer”, Prof. Bo displayed the world of the emerging paradigm of “Queer Game Studies”, which claims that video games should be reconsidered through the lens of the LGBTQIA experience.  Though their work, Ruberg is attempting to foster community conversation across fields and makes space for queer people that work with and around games to thrive. 

In Prof Bo’s analysis, we examined the way in which people engage with the standard structure of a video game. Even in the CIS-Heteronormative community, there is the emerging practice of doing games differently such as using glitches and tactics to finish a game as fast as possible. 

This counterintuitive practice reminds us that there is no one way to do anything, even things that have a built-in structure. When we apply the lens of the queer experience, video games become an item to be reclaimed and owned by those that are often marginalized. As the talk progressed we examined different aspects of games such as the reinforcement of heteronormativity and American ideals of the family unit. The performance of gender in choice games in which your character has assigned sex (consequently an assigned gender). The objectification of women’s bodies, particularly their breast, in fighting games.  

However, not all games are created with today’s norms in mind.  Some independent video game makers are using the tool to change the culture around the topic and the way we approach it. I’ve had the chance to play “The Realistic Kiss Simulator”, which was discussed during the talk. It was created by two independent developers and involves two pseudo non-binary human-like individuals. Using the keys, players must coordinate for the two characters to kiss.  The game is dynamic and works on multiple levels—mainly that it gamifies the idea of kissing/sexual acts. However, it does not progress, nor is there a way of “winning”. Instead, the game focuses on the “goallessness” which consequently de-gamifies the act of kissing.   

Prof. Bo and all those in the that have united on this front are working diligently to reclaim spaces that have been denied to certain communities. I’m excited to see what lies ahead in that field of work and what the DHSE lab will bring to campus next. 

The Algorithm as Tamagotchi

A person works with paper labels.

This past February, Digital Humanities for Social Engagement and the Digital Justice Lab hosted a workshop called “Algorithms as Pets and Politicians.” Part of a series of events organized by Alex Juhasz (Chair of Film, Brooklyn College) on media literacy in this age of the digital proliferation of fake news, this particular workshop focused on how art practice can shed light on the political implications of the ways that Machine Learning systems view and comprehend the world. The workshop’s leaders were Alex, Orr Menirom (independent artist, NYC), and myself (postdoctoral fellow, Neukom Institute).

Orr screened an excerpt from her video work, “Clinton and Sanders Looking at the World and Naming Things for the First Time.” This piece interlaces footage from the 2016 debate between the two presidential candidates with decontextualized, fragmentary glimpses of multifarious objects and scenes, both quotidian (a pair of socks) and striking (a protest). Orr has edited audio from the two candidates to offer off-kilter descriptions of these things, giving the impression of an ill-trained Machine Learning system attempting but failing to label a material world whose objects are not nearly as straightforward as those in its “training set.” For instance, Bernie—or at least Bernie’s voice, made strangely robotic through Orr’s editing—seems to mistake a brick pinned beneath a door for a “human.” (To be fair, the holes in the brick do suggest a face.) Participants drew connections to the intertwined fallibility and creativity of computer vision systems; Google’s DeepDream, perhaps the most famous of these, can be trained to “see” faces in pictures where none exist. Orr’s work is not itself algorithmic, but the process of its creation can be seen as an attempt to see (or hallucinate) the world as if through the murky and error-prone layers of the neural networks that exert increasing power over our lives.

Next, Orr and I demoed a prototype of an interactive algorithmic text generation system called “The Speaking Egg.” Eschewing contemporary Machine Learning’s capacity to learn patterns from large, ready-made data sets, this system takes a decidedly more bespoke and laborious approach, one inspired by pet-like computational systems and interfaces that require care and attention to survive and thrive (e.g. the Tamagotchi, the Furby, or the virtual pets that scampered across desktops throughout most of the 1990s). To get this pet egg-bot to grow and to speak, one must manually provide example sentences. When the bot generates (simple and often nonsensical) sentences based on these inputs, the user must either praise or scold it, training the bot’s classification algorithm to help it produce more pleasing utterances. Workshop participants trained their algorithms to inhabit specific identities and ideological positions, exploring what it might mean to design algorithms that aspire to be intentionally political rather than “neutral” or “unbiased.” The feedback we received will be invaluable as Orr and I continue to develop this project.

The day concluded with an impromptu artist talks by several participants. Aaron Karp (Digital Musics) presented an agent-based digital music system based on the flocking of birds, Christiana Rose (Digital Musics) showed videos of her interfaces for sonifying the movements of acrobatic artists, and Josh Urban Davis (Computer Science) reflected on using deep learning to generate obituaries of people who never lived. These presentations, along with the lively participation of faculty, staff, and students, are evidence of the robust, interdisciplinary interest on campus in both the art and politics of computation.

Kyle Booten
Postdoctoral Fellow
Neukom Institute for Computational Science

Kyle Booten
Postdoctoral Fellow
Neukom Institute for Computational Science

Eugenic Rubicon

Many people are surprised to learn that in the 20th century over 60,000 people in the United States, mainly patients in state asylums and hospitals, were sterilized based on eugenics laws. A number of excellent books and articles and a few web resources on the history of eugenics and sterilization have appeared in recent years, but very little known is about the demographics and experiences of people sterilized, often against their will. Eugenics, the effort to shape and limit populations through sterilization and other forms of reproductive control, was popularized and institutionalized in 20th century America. While eugenics laws have largely been struck down and/or removed, the legacies of these practices have shaped communities and relationships between communities throughout the U.S.

Eugenic Rubicon is a developing prototype (http://scalar.usc.edu/works/eugenic-rubicon-/index) that uses mixed media and digital storytelling methods to share portions of this history. We have both examples of primary documents and newly authored materials that help to give a sense of children’s experiences of these practices, of different kinds of resistance enacted by patients, and how local practices at Sonoma State shaped this history.

This digital resource draws from and complements the demographic and social science research on eugenic sterilization in California being carried out by the  Sterilization and Social Justice Lab at the University of Michigan. Working with a unique resource — nearly 50,000 patient records from California institutions from the period 1921 to 1953 — our project seeks to make this history visible. Additionally we are working to make the dataset we’ve developed accessible and interactive. These records were microfilmed by the California Department of State Hospitals in the 1970s and only recently discovered; The Sterilization and Social Justice Lab has digitized these reels and are using them in compliance with state and university regulations to create a dataset that adheres to protocols around sensitive health data. These materials create opportunities as well as challenges for storytelling and possibilities for humanizing stories of reproductive injustice. It also raises important legal questions about how to balance the “right to know” with the “need to protect” in the realm of medical and health histories.

Eugenic Rubicon: California’s Sterilization Stories is a multidisciplinary collaboration between Arizona State University and University of Michigan and that includes digital storytelling, data visualization, and the construction of interactive digital platforms. This project employs creative approaches to history, digital humanities, and foregrounds commitments to social and reproductive justice.

The Digital Justice Lab is currently working on a new website (launching in 2021) to showcase three states worth of eugenics stories.

Corporeal Imaginations

In September, the Digital Justice Lab produced a exhibition of original work asking viewers to think critically about the relationships between their bodies and the data they produce.

Wernimont demonstrates the skin-like qualities of a piece. Photo by Brinker Ferguson.

Read the Dartmouth News recap and watch the video featuring lab director, Dr. Jacqueline Wernimont.

DJ Spooky

On Monday, October 14, the Digital Justice Lab brought Paul Miller, aka DJ Spooky, to Dartmouth College to speak about his latest composition Quantopia and to lead a student masterclass.

In addition to sharing parts of Quantopia, Miller took the audience on a journey through the founding algorithms of the internet, the histories of sound production and sound’s relationship to place, and explored some of the key themes throughout his work. Miller is currently focused on quantum physics, the politics of data and energy production and the ways that music can be a political force.

During the masterclass, students asked Miller questions about the democratization of music-making technology, the challenge translating science into music and art, and how he integrates his individual political practice into the projects he selects.

Check his website for more information about DJ Spooky’s upcoming events.

(In)tangible Violence: Poetry, Touch and Critical Making

Poems with nails through them.

by Whitney Sperrazza

My research explores the relationship between tangibility and intangibility. As a scholar of early modern poetry, this relationship informs my experiments with how digital practices help us engage differently with historical literary texts. During my visit to the Digital Humanities and Social Engagement lab this spring, I had a chance to talk about a work-in-progress on the haptic quality of poetic language. Feminist literary scholars have long argued that early modern sonnet conventions enact violence against the female body. In my ongoing experimental humanities work, I pose as my central question: how do we feel that violent language?

For some early prototypes, I constructed wooden board and nail versions of early modern sonnets, turning the poems into three-dimensional objects that made the reading experience a bit more perilous. I used the blazon convention as my starting point and marked (drove a nail through) any language that abstracted and catalogued women’s body parts. Ultimately, I produced poetic objects that were physically sharp to the touch. The poems were punctuated with moments of “sharp-ness” that not only forced the reader to confront violent descriptions of the female body, but also made those violent, and often penetrative, descriptions tangible. In addition to raising questions about the relationship between violent language and physical violence, these objects function as an artifact of my own reading—a creative edition of the sonnets that makes palpable my critical aims.

I shared these prototypes with the DHSE community and had a chance to discuss ideas about the project’s next steps as I shift from a handful of representative poems to the entire archive of early modern sonnet sequences. Increasingly, I am using this work to think about the long history of violence against women’s bodies and how new interactions with literary texts can help us better understand aesthetic manifestations of that violence. In other words, violence in early modern sonnets is a structural, not simply conceptual, feature. For the project’s next phase, I plan to combine my experiments in critical making with computational text analysis methods, tagging and cataloguing violent language across the early modern sonnet archive (roughly 1,500 poems) and then producing new objects based on that much larger accumulation of data points. By combining critical making and computational text analysis, I want to offer a creative repurposing of quantitative tools and use my curated data set as a starting point for new forms of engagement between texts and readers.

DHSE offered the perfect space within which to have a conversation about this work and its future directions, particularly given the social justice ethos so central to the lab’s mission and the highly interdisciplinary community the lab convenes for its programming. The DHSE audience (which included individuals from the Department of English and Creative Writing, Dartmouth Libraries, and the Neukom Institute for Computational Science) offered perspectives on different aspects of the work: the digital humanities methods, my close readings of the sonnets, and the project’s feminist critical framework. Additionally, in conversations with faculty, staff, and students throughout my visit, I learned about exciting new digital work at Dartmouth and gained valuable insight on project design and development.

Thank you to the DHSE team—Jacqueline Wernimont and Brinker Ferguson—for the chance to play and experiment in your vibrant lab space and for hosting an energizing and intellectually rigorous conversation about my work.