AMT Lab @ CMU

View Original

All the World’s a Stage and the Technology Merely a Player

By: Theresa Rudesill

In 2016, the Royal Shakespeare Company wanted to create something extraordinary to celebrate the 400th anniversary of the Bard’s death. In the words of Artistic Director Gregory Doran, “Shakespeare was touching on that new form of theatre. So we wanted to think about what the cutting-edge technology is today that Shakespeare, if he were alive now, would be saying, ‘let’s use some of that’” (Pollack-Pelzner, 2016).

In the two-year span between the initial thought and the opening of The Tempest in Stratford-Upon-Avon, a partnership between the Royal Shakespeare Company, Intel, and Imaginarium Studios began (Royal Shakespeare Company). Based in Stratford-Upon-Avon, The Royal Shakespeare Company is one of the world's leading theatres. Intel is an American tech company that aims to “create world-changing technology that improves the life of every person on the planet,” and Imaginarium Studios describes itself as a “pioneering Performance Capture and Virtual Production company based at Pinewood Studios, UK and Trilith Studios, USA” (Intel, Imaginarium Studios). 

Figure 1. The Tempest, Royal Shakespeare Company. Source: Topher McGrillis

The three came together to do something never before seen in live theatre: live performance capture using artificial intelligence. We have all seen motion capture used in some form, whether that be in film, video games, or television, but rendering had always been done in post-production. In this production of The Tempest, these companies sought to create a live avatar that would follow the movements of an actor as they were happening. Dr. Ceri Gorton, an experienced creative strategist, researcher, and cultural sector leader, outlines how the technology functioned (Bird & Gordon). Actor Mark Quarterly was placed in an Xsens suit which was laced with motion capture technology, which “enabled the team to create animated versions of Ariel live on stage. Quartley’s performance was also the first to showcase facial real-time tracking technology, which allowed him to control the facial expressions of his digital character as well as the movements of his body” (Gorton, 2019). In collaboration with the theatre’s lighting department, they could control the color and stage position of Ariel’s avatar.

See this content in the original post

A study was conducted by Dr. Ceri Gorton on the impacts of this technology on future productions and, on a larger scale, the field of arts management. One of the biggest takeaways discussed in the study was that “The Tempest identified key principles that can be developed in arts organisations to enable innovation across disciplines. These principles can make collaborative arts/tech R&D more feasible and extend its impact beyond the life of a single project to inform long-term opportunities”  (Gorton, 2019). 

As exciting as the technological innovations of The Tempest were, it is not easily accessible. The cost associated with this feat would not be possible for many organizations, especially as a one-time project. Accessibility of this technology proves to be a barrier to larger adoption, particularly in the field of education. 

In 2020, a study was conducted to see how students interacted with Shakespeare’s work through multi-modal learning. Harvey, Deuel, and Marlatt describe multimodal learning “.as a set of evolving approaches to navigating texts that accounts equally for technology innovation and the numerous exigencies of reading and writing that compose our lives” (Harvey et al., 559-68). This type of technology integration may allow for adaptive learning that engages students in a topic that they may have avoided otherwise. 

So why is this particularly important when teaching Shakespeare and similar texts? It is no secret that Shakespearean text is challenging and can feel like a different language. With the addition of immersive technologies like virtual reality (VR),  students can interact with characters and see stories play out in front of them. In the study, students not only engaged with immersive technology but sought it out and wanted to continue working with Shakespearean text in that context. (Gorton, 2019). When students were not using the VR headsets, many started discussions with their classmates about what could be happening on the headset and what they wanted to do when given the opportunity to use it. At the end of this experiment, students used VR headsets to perform scenes from different Shakespearean plays in the setting of the Globe Theatre (Harvey et al., 559-68). While this learning model showed many benefits, it is important to recognize that high equipment costs limit accessibility to VR for many students. 

Figure 3. Students use QR codes to access informational texts. Source: Harvey et al.

Figure 4. Student immersed in Globe Theatre VR experience. Source: Harvey et al.

Immersive technology is not the only thing affecting how we teach Shakespeare. In a more cost-effective model, we also see artificial intelligence helping students better understand the Bard’s language.

Mike Gioia is changing the world of Shakespearean learning with the use of AI as a “translator.” As Gioia mentions in his article, 56% of teachers say that their students found Shakespeare difficult to read, and 60% of those teachers said it was due to the Elizabethan language (Gioia, 22-24). 

With this considered, Gioia developed a large language model, called “Shakespeare Translator,” to “instantaneously synthesize, explain, and contextualize” Shakespearean language (Gioia, 22-24). The model has now been used by more than 500,000 readers. Not only can AI help students take a deeper dive into complex texts, it also might help historians uncover new information. 

In a different use of AI, historians are beginning to dissect Shakespeare’s work to find out the true author of particular scenes. It has been long speculated that Shakespeare did not act alone. You might be familiar with several popular theories: that Shakespeare stole work from different playwrights, that “Shakespeare” was actually multiple people, or even that Queen Elizabeth I wrote some of his work. Historians and thespians are now running Shakespeare’s work through AI tools to flag areas in the text that may have been written by different people. In particular, Henry VIII has been a target of such experimentation. It has been a common belief among researchers that this show has elements of John Fletcher’s writing style. John Fletcher was Shakespeare’s successor after his death (Morrison, 2019).  Petr Plecháč, an artificial intelligence researcher, fed work from Shakespeare and Fletcher to an open model AI program. He then put in Henry VIII and asked the AI to distinguish the writing. What it found was that Henry VIII appeared to have quite a bit of writing from Fletcher, specifically the second act. It also flagged areas where the two playwrights likely collaborated.