| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Stop wasting time looking for files and revisions! Dokkio, a new product from the PBworks team, integrates and organizes your Drive, Dropbox, Box, Slack and Gmail files. Sign up for free.

View
 

Animation and Audio Poetics as Process

Page history last edited by Alan Liu 1 year, 6 months ago

     In “Deformance and Interpretation,” Jerome McGann and Lisa Samuels offer the concept of “derformance”—that is, various methods working toward the same goal of disrupting the text in an attempt to create new meanings. This methodology was posited as a supplement to traditional critical approaches to texts, where “[t]he usual object of interpretation is “meaning,” or some set of ideas that can be set in thematic form” (26). Allowing that there are infinite approaches to the deformative process, the process of deformance creates new text objects, ones which will, ideally, illuminate the formal aspects of the original text. Deformance is thus “physical and performative” (26), revealing language’s potential as “an interactive medium,” rather than acting as a mere “reflexive work of analysis” (28, 29). It attends to the "object"-ness of a text, not as "stable" and "inviolable" (180), but as material, physical, and therefore able to be de- and re-assembled. Criticism, McGann and Samuels argue, has the potential to be as artistic a process as translation or ekphrasis.

 

     In “If Meaning,” Samuels notes the inability of the print-medium to capture deformance at work and the myriad possibilities it must yield in order to claim consistent and effective critical practice. She points us instead to the possibilities of an electronically-based medium, which she calls "Being in a new space." Here, we argue, “being” can function as no longer just a presence, but a process. Through animation and sound, critics can introduce to their work a new dimension of experience: time.

 

     Initially published in the 1798 edition of Lyrical Ballads, Wordsworth’s “Old Man Travelling; or, Animal Tranquility and Decay. A Sketch” became subject to two significant revisions in subsequent editions (1800 and 1815) of the collection. In all three versions of the poem, the first 14 lines remain intact and unaltered; however, the Old Man’s speech, directly offered in the first edition of the poem, becomes an object of slow, gradated erasure in the second and third editions: what was previously direct speech in the first becomes reported speech in the second, and is edited out entirely in the third. The various editions already give us an illustration of the inherent instability of the text (what Lisa Samuels refers to as “anarchic” forces, in her essay "If Meaning, Shaped Readings, and Leslie Scalapino's way"), and the power of the deformances offered by the author himself through revisionary methods. By exploring the three versions with direct reference to each other, we hope to reveal the open-ended nature of the text, and of text itself. 

 

"Old Man Travelling; or, Animal Tranquility and Decay. A Sketch." (1798)

 

                    The little hedgerow birds,

That peck along the roads, regard him not.

He travels on, and in his face, his step,

His gait, is one expression; every limb,

His look and bending figure, all bespeak

A man who does not move with pain, but moves

With thought.--He is insensibly subdued

To settled quiet: he is one by whom

All effort seems forgotten, one to whom

Long patience has such mild composure given,

That patience now doth seem a thing of which

He hath no need. He is by nature led

To peace so perfect that the young behold

With envy, what the Old Man hardly feels.

--I asked him whither he was bound, and what

The object of his journey; he replied

"Sir! I am going many miles to take

"A last leave of my son, a mariner,

"Who from a sea-fight has been brought to Falmouth,

"And there is dying in an hospital."

 

 

"Animal Tranquility and Decay. A Sketch." (1800)

 

                    The little hedgerow birds,

That peck along the roads, regard him not.

He travels on, and in his face, his step,

His gait, is one expression; every limb,

His look and bending figure, all bespeak

A man who does not move with pain, but moves

With thought.--He is insensibly subdued

To settled quiet: he is one by whom

All effort seems forgotten, one to whom

Long patience has such mild composure given,

That patience now doth seem a thing of which

He hath no need. He is by nature led

To peace so perfect that the young behold

With envy, what the Old Man hardly feels.

--I asked him whither he was bound, and what

The object of his journey; he replied

That he was going many miles to take

A last leave of his son, a mariner,

Who from a sea-fight had been brought to Falmouth,

And there was lying in an hospital.

 

 

"Animal Tranquility and Decay.

A Sketch." (1815)

 

                    The little hedgerow birds

That peck along the roads, regard him not.

He travels on, and in his face, his step,

His gait, is one expression; every limb,

His look and bending figure, all bespeak

A man who does not move with pain, but moves

With thought.--He is insensibly subdued

To settled quiet: he is one by whom

All effort seems forgotten, one to whom

Long patience hath such mild composure given,

That patience now doth seem a thing of which

He hath no need. He is by nature led

To peace so perfect, that the young behold

With envy, what the Old Man hardly feels.

 

 

Animation

 

     Offering text- and image-based visualizations, “shaped” readings, and further deformances of the texts, we will challenge the idea of editing as textual (or authorial) authority, shifting power away from the author and into the minds (and hands) of the reader. Through a hybrid approach of visualization and deformance methods, we hope to open the experience of the poem to new perspectives. One of the critical aspects of this approach is through the use of animation: moving and moveable text, as well as images existing in motion. Rather than “treating the poem as an object,” which Samuels suggests “suppresses that which is excessive and anarchic in the work of language art,” we hope to challenge the very idea of “objectness”: when an image or text no longer exists as a static entity, but is put into motion, does it allow for further openness of interpretability? Samuels challenges us: “How do we stimulate criticism that will stop bracketing out its theoretical knowledge of…uncertainty, to develop more tools that don’t suppress either semiotic dynamism or that which is anarchic—fluid, unstable, changing, indeterminate, and so on—in art?” This idea of “stimulation”—of inciting movement, activity, responsiveness—is one of the key interests of our project. Using the iMovie to create videos that 1) open the poem to the dimension of time, and 2) derive from a platform that freely allows the physical manipulation and re-ordering of words, we intend to incite both the reader and the text to respond to the texts in new ways. “Electronic writing invites the quicker perception of potential variants,” Samuels writes. Movement disrupts the hierarchy imposed by the written word, the physical page, the structure of conventional reading. Archibald MacLeish once famously wrote, “A poem should not mean, but be.” We hope that animating the poem in various ways will create a fuller, more life-like sense of “being.”

 

Animation Principles: 

We want to develop multiple animations of our poems with the the following goals in mind (keeping mind that all of these are not collectively achievable with a single animation): 

  • The animation will write and overwrite itself, transitioning from manuscript to published form.
  • The animation will both discard the discarded portion of text, while maintain it elsewhere, demonstrating the tension between manuscript (handwritten) and published (typed) forms.
  • The animation will utilize ("recycle") the discarded material from both manuscript and earlier print versions of the poem, through a formal constraint termed "waste poetics"--poetry created from only the discarded materials.
  • The animation will acknowledge the spectrality of existant versions of the poem, meditating on how any version is "haunted" by other versions.  
  • The animation will attempt to convey voice and voicelessness through refraining from sound.
  • The animation, attempting a synesthesic effect, will be overlaid with audiopoetic sound-mapping to question whether one process can inform the other. 
  • The animation will reflect on the multiple interventions of mediation, both in the poem and self-reflexively.
  • The animation will meditate on the politics of sound and motion, in the poem, in the deformative (critical) process, and in representation on a larger scale. 
  • The animation will provide a useful way of disseminating academic materials more publicly.  

 

Tools

  • Pencil an open-source, free 2-D animation tool.

  

  • iMovie a Mac-based platform where stills of images can be re-ordered, manipulated, and shared through public web-based platforms. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Results

The project yielded three different processes, using animation as the common medium:

 

1. Manuscript to Print.  Every word, punctuation mark, and intentionally-left blank space was represented by a corresponding piece of paper. The first animation was shot to relate the process of the poem as it proceeds through manuscript form--a "reformance." This information was accessed through the Cornell University Press version of Wordsworth's poetry from 1797-1800 (ed. by James Butler and Karen Green). You can watch the rendered manuscript video here. Additionally, rendered versions of the 1798, 1800, and 1815 versions of the poem are available here: 1798, 1800, 1815.

 

The animation resolves strikeouts and other layered edits by recording the stricken word/phrase, immediately followed by the replacement word/phrase. In this way, all of the text is available to the viewer, and the resulting poem/animation becomes a record of all words, punctuation, and intentionally-left blank spaces that exist in the manuscript. One constraint of this animated approach is that only one word or punctuation mark can appear onscreen at a time, in order for the text to be readable, and the process to be consistent. Interesting, this levels the hierarchy of the page (space), though imposing on it the hierarchy of time (order). Time  itself was constrained by keeping each frame onscreen for the same amount of time, 0.5 seconds.

 

This rendering is meant only as a prototype--ideally, the process would be written in Flash, or processed through another platform that would allow all of the text to appear onscreen as it appears spatially on the page, only through an animated process rather than as a printed object. The poem would "write" itself, and when the first edit occurs, the screen would split in two: the first screen would continue editing itself until finally arriving at the 1815 print version. The second screen would preserve every deleted word or mark, demonstrating how our understanding of any one version of the poem is "haunted" by the "spectre" of the process.

 

2. "Waste Poetics." The concept of "waste poetics" is derived from the manuscript-to-print process: what kind of deformances can we create when formally constrained to the vocabulary of another poem (be it printed or discarded)? This concept plays with the image of digging through Wordsworth's trash bin, and recycling or reclaiming that material to create something new. The rendered animation of one example can be viewed here. This render used words, punctuation, and space that were discarded during Wordsworth's editing process and playfully attempted a "teenage" poem that Wordsworth might have created and thrown away--a nod to Wordsworth's own preoccupation with his growth as a poet, and his conception of poetry as a process.

 

The rendered animation is not the point of this process--indeed, an ideal platform for "waste poetry" would be an interactive text presenting the viewer with all available words, allowing her/him to "grab" the words s/he wished, and rearrange them on the screen. This process is how iMovie works: though the animation you see is itself finalized, it is only one render of an almost infinite number of possible animations. Even now, iMovie has preserved these words, marks, and spaces in frames that can easily be referred back to in order to create yet another animation.

 

3. "The Animated Argument." In addition to exploring the processes of poetry and deformance, animation provides a useful medium for creating and widely disseminating academic materials to the public. The animated introduction to this presentation can be viewed here. By utilizing text and images and setting them to sound and into motion, academics can publicly circulate materials to a wider audience. This video is less than 1:30 long, yet it acts as a self-contained module promoting the use of electronic space and multimedia production in the critical process. It contains about 300 still frames, and was shot, edited, and the voice-over was recorded in just one day. The process could be duplicated in order to promote books, conferences, or to present abstracts, and has been used (though with far greater expense and production value) for digital pedagogy (such as the RSA whiteboard animations, etc.)

 

Finally, the animation and sound aspects of this project culminate here, in our 1798 "Old Man Travelling" Remix.

 

 

Audio Poetics

 

This prototype for musical adaptation of poetry offers a logical process and a series of potential trajectories for extending the deformative possibilities of audio poetics. As deformance, the project tends toward an interpretive model that inhabits the fluidity and plasticity of the creative process rather than the judgment and interpretation of some shiny end-product. Wordsworth’s poem, in its multiple versions, is especially appropriate for this form of process-oriented deformance (discussed in more detail below). We initially developed a series of goals for the prototype, some of which we achieved; others were altered, deleted, or set aside as future possibilities (impossible to address with a single prototype). We will describe the project in more detail through a discussion of these goals.

 

1. The conversion process is governed by a logical rather than an arbitrary set of "rules."

We wanted to avoid a logic of adaptation that places formal poetics and prescriptive grammar in an analogous relationship to formal musical notation systems. Of course, these systems exist on either end of the translation, but they seem too abstracted and arbitrary to govern the middle zone, the actual process of conversion. 

 

2. The conversion process involves multiple sequential stages from text to music (e.g., vocalization, phonetic analysis, visualization, mapping/tracing, musical re-composition).

Stages:

  • Immediately destabilize any notion of the text as a fixed, static, and unchanging entity by moving to an audio recording of a specific reading of the poem. Instantly the text becomes about the specific reading, the choices the reader has made whether intentional or otherwise: enunciation, pronunciation, intonation, pacing, etc.

Instead of a particular human reading of the poem, we chose to work with text-to-speech synthesis (eSpeak) because it further destabilizes the binary between the stability of the written text and the instability of the individualized reading, which we see as inextricably bound to the problem of faithful/unfaithful adaptation. How? The synthesized voice is a literal aggregate of human voices, but it is also an inescapably mechanistic distortion of the human voice, a paradoxical synthetic outlier.

 

  • Generate a spectrogram of this audio recording using Praat phonetic analysis software.

 

  • Splice spectrogram according to line breaks, each representing approximately 2.5 seconds of sound. This task involves a certain degree of precision and is a process best described as close listening or reverse scansion.

 

  • Scale and map the spectrogram, line-by line, onto a tone matrix template. Here we move from one “sound system” to another. Certainly this process compromises the frequency range and with it the relative frequencies of different sounds. The range of the tone matrix is noticeably tighter than spoken language, even robo-speech.

 

  • Program the resultant patterns into the tone matrix on audiotool. Because of the limited space on the tone matrix, the smaller the spectrogram slice, the more accurate the mapping and the more faithful the resultant sound pattern to the original poetic utterance.

EASYDIFFICULT

An interesting result of this programming process came from viewing the pattern thumbnails, which revert the matrix pattern to the appearance of the original spectrogram.

 

  • Arrange multiple patterns into a single musical composition in Audiotool’s multitrack recorder. Each line of poetry or tone matrix pattern gets a single measure. Adjust beats per minute so that the time of the composition matches the time of the original audio recording of the poem.

 

  • Augment this raw version of the song with additional instruments and effects. Here we were concerned with adding polish and making the music listenable, while still considering the logic of adaptation wherever possible.

 

3. The music permits the recognition of properties of the text not previously considered while also suggesting that we reconsider what we consider when we consider a poetic text.

One of the goals of an elaborate, multimedia, multistage adaptation approach is that the process itself engages the text in new or unexpected ways. In this sense, any new process can be a new way of reading closely and becoming familiar with a text. The process does facilitate certain formal poetic modes of interpretation. For example, spectrograms visualize scansion, but they also call into question the rigidity of poetic prosody itself. What’s an iamb anyway? Isn’t this already a logic adapted from the syllabic to the accentual-syllabic? And what does it mean when the frequency of an accented syllable is actually lower than the frequency of unaccented syllables elsewhere in a poem? Indeed process-oriented deformance can and should confound these formal poetics, in the process asking strange and novel questions. For example, what does it mean when the song reveals (through a catchy hook) a certain climactic tone in the turn to direct speech near the end of Wordsworth’s poem?

Does this hook open up the difference in these lines and present an alternative, apolitical justification for Wordsworth’s later deletion of these lines (i.e., they sound different)? Or does it simply ask us to consider how this turn is climactic? Or perhaps it forces us to consider how the original poem appeals to the reader/listener at this moment.

 

4. The conversion process is simple enough to be repeated by others and with other poetry with relative success.

Achieved! Though without some serious software development, the process will remain extremely labor intensive, but, as we ask above, does the laborious process itself create knowledge?

 

5. The music does not ignore or discount the non-verbal material around the words.

Leaving silences completely silent (especially using only the tone matrix) made the music too jumpy to be listenable. We filled these gaps with minimal, mid-range frequencies, so the sound drops out and decreases in intensity. Does representing these spaces rather than simply leaving them blank suggest a valence that is maybe ignored when initially engaging the written text? And does the process of representing them encourage closer reading of these gaps and pauses?

 

6. The music is listenable.

We have already discussed this goal at some length. The use of percussive elements definitely factors in here, and one debate that surfaced is whether the use of percussion should follow a logic of adaptation or simply augment the listening experience. A logical approach might involve both representing accented syllables with an accented drumbeat and choosing different percussive instruments to represent different consonant sounds: bass drum for plosives, snare for fricatives, hi-hat symbol for sibilants, and so on. Initial experiments along these lines resulted in a less listenable song than the raw tone matrix version. Consider the rapid fire of diverse drumbeats across a single measure of music that would be necessary to accommodate the representation of a single line of iambic pentameter with multiple consonant sounds. Techno is dead. Less is more. Of course, in the future we could attempt some reduced representation of the most salient sounds and accents, but there is something to be said for steady and more consistent percussive elements. For one, it aids the recognition of melody and helps listeners call attention to tonal or vocal variations and hooks (discussed above). So, we have opted instead for the use of percussion in the prototype as a form of polish (though it does loosely parallel the rising rhythm in the poem) that also acts to determine musical genre to some degree. (The analogue here could be the style of reading, or the musical genre selected could hinge on the poetic genre.) We find the prototype to have resonances of electro-afrobeat.

 

7. The music acts as a reliable representation of the poem and isolates the sonic features from the actual language.

No doubt, the question of reliability is open to debate. Here, we offer one argument for sonic reliability in the form of a visualization: A comparison of the original audio spectrum and the raw tone matrix adaptation. Note the similar patterns in both examples. Indeed, the major points of differentiation are expected. One, the increased regularity of the musical spectrogram is to be expected in the standardizing process of replacing poetic lines of varied length with musical measures of equal length. Also, the music clearly has a fuller, deeper sound. This could be decreased by only representing the high and low frequencies on the tone matrix and hollowing out the rest.

8. The variances in the music for different versions of the poem will reflect key interpretive differences and will not simply vary superficially.

Again, this goal could be achieved in the future by altering musical styles and atmospheres, which would impose certain interpretations and evaluative registers commonly associated with different genres of music, but here once again deformative adaptation opens up a lot of questions about genre’s association with predetermined interpretive moves.

 

9. The music may be fruitfully combined with the animations.

See example.

 

10. The conversion process is complex enough that the resultant music may be called a translation rather than an adaptation.

11. The rigor and diversity of the process makes a meaningful contribution to translation theory.

Calling this deformance a translation may imply an audience that communicates through non-verbal sounds. Developing this idea would lead us to the question of cognitive diversity. Can we deliver the poetry to consciousnesses that would otherwise not understand it (or care about it) in its original form? Or in the musical form, will some understand it more fully? Adaptation, translation, or supplement? Here we need to consider animals, humans with limited communicative and verbal capacities, teenagers (and others) obsessed with music, the cosmos, and so on.

 

12. The music can represent the difference between versions without representing any single version.

A question left open. We welcome any speculation on this possibility.

 

Comments (0)

You don't have permission to comment on this page.