Writing has been around since cavemen used rocks to scribble on walls. Ink and paper became the norm, and later, words were most commonly written on a typewriter.
Today, writing is normally done on a computer keyboard or smart phone, or voicing words into another recording device. But converting those digital words into other languages may not be as easy using various platforms of modern technology.
“Since much is being done with digital literature generated by algorithms, it’s not as simple as traditional translation,” said Sandy Baldwin, an associate professor in Rochester Institute of Technology’s Department of English. “When you’re dealing with a digital object, whether it’s a website or a computer program, there are different layers of translation, be it computer code, readable text on a surface, a website it is presented on or videos.”
Baldwin is one of five co-principal investigators, all scholars of digital literature, named in an 18-month grant from the Mellon Foundation to the Fondation Maison des sciences de l’homme in France to explore issues and find ways to translate digital literature. He will work with partners in France, Denmark, Portugal and London.
Each of the five teams selected works from an author. Baldwin is studying the work of John Cayley, a professor from Brown University who is using the Amazon Echo to create literature. Echo, an interactive device that is activated by voice, uses “Alexa,” the artificial intelligence voice marketed as a companion to automate and aid our daily lives.
“Cayley turns interactions with Alexa into a sort of ominous short story that makes users think about surveillance and about how much ‘humanity’ we attribute to these artificial intelligence companions,” Baldwin said. “Cayley does this by creating a new set of interactions and adding a new set of voices, male voices that interact with Alexa and with the user.”
Baldwin said he’s planning to shift the tone of Cayley’s story from ominous to cheery. “We see the new story as being more of a marketing piece, trying to sell people on the Echo, as well as perhaps being ironic,” he said. “We will test these variations with a large number of users, gathering information about their understanding and emotional response.”
Baldwin said his work is important because it helps to understand how people engage with this new form of literature and how people engage with the ever-increasing number of artificial intelligence companions.
The teams will work together via Skype and hold workshops at each of their five locations, he said.
“Writers have always had certain structure that confined how they write, be it sonnet lengths, sounds of words or plots in a story,” he said. “Digital technology lets the author expand those constraints. With digital literature, you have more than just words to translate. You have images and animations, literature that emerges out of traditions and how people tell stories from generation to generation.”
Ethical questions can also be raised if an author uses digital technology to help write. How much of the finished piece is written by the author if technology assists to provide words?
The teams also will benefit from work of students from the National Science Foundation’s Research Experiences for Undergraduates program on Computational Sensing who are particularly focused on how people interact with and respond to the Echo.
When their grant ends, the teams plan a series of conference presentations and museum exhibits from their research, “and we’ll write a book about what other scholars and artists can do to translate digital literature,” Baldwin said.