Arts Campus, Umeå University
kinect tracking, audio,
photographs, film, programming.
“As we spend more of our time staring into the frames of movies, television, computers, hand-held displays – ‘windows’ full of moving images, text, icons, and 3D graphics — how the world is framed may be as important as what is contained within that frame.” – Anne Friedberg, The Virtual Window: From Alberti to Microsoft p1
The concept of metalepsis, from Greek meta, “change” and lambanein “to take” (“to change the sense”) is used in narrative theory to explain a character addressing an audience member in a film, or a reader becoming part of the action in an interactive story. In digital media metalepsis has been applied to the numerous examples of users, viewers, players and readers negotiating and effecting the outcomes of narrative in games and fictional environments, often delivered via screens. In a current installation Fyr (Swedish – ‘Lighthouse’) the body of the person experiencing the work is meshed with the images and spaces it is composed of via the arrangement and interactive properties of the screens used to deliver it.
Fyr (2013) in HUMlab X, by Kent Gustafsson, Fredrik Oskarsson, Jim Robertsson and Johan von Boer, is delivered with a floorscreen that reacts to touch. By moving on its surface, images and responses are activated on the floorscreen. Facing the person on the screen is a wall projection that shows images and broadcasts audio. “The footprints and movements of people walking on the floorscreen triggers sounds, photographs and film sequences, and each participant will be surrounded by their own unique experience of the Lighthouse” (Elin Andersson). Furthermore a Kinect body-tracking scanner is attached to the computer running the installation and by moving around the screen the audio changes.
Up and down, under and over. These are basic concepts that orientate us to the space we experience every day. Anyone who has ever experienced a disorientating fun-park ride will understand that spatial orientation is a basic function of our senses. If the senses are challenged when it comes to up and down, or under and over, the feelings and thoughts we experience can change. Something that our senses take for granted is that the ground is below us, and above us is the sky (Thank you John Lennon). Buildings follow a similar predictable pattern, with a roof above and the floor below.
My initial impression of Fyr was slight spatial disorientation, followed by a serious feeling of immersion in the sights, sounds, and space of the far northeastern coastline of Sweden (where I lived for 12 years until recently). This is a rugged and beautiful place; made up of vast forests, grey stones and sandy soils, small islands, a brittle and tenacious coast and a sea that resembles a vast and disobedient lake (the Gulf of Bothnia). Within the space of Fyr the 1987 novel of the same name by Magnus Dahlstöm is evoked by the representation of the landscape around a lighthouse on an island in the Gulf of Bothnia. Birds, water, the shipping forecast, blend and react with images from the site itself.
What is perhaps most interesting about Fyr is the breaking of the frame mentioned in the opening quote from the late great Anne Friedberg, with an awareness of the window. The famous translation of what Leon Battista Alberti wrote in his treatise on perspective in 1435, “First of all, on the surface which I am going to paint, I draw a rectangle of whatever size I want, which I regard as an open window through which the subject to be painted is seen.” (De pictura, 1.19). Friedberg of course realized that “in the last two decades – markedly with the advent of digital imaging technologies and new technologies of display – did the media ‘window’ begin to include multiple perspectives within a single frame” (Friedberg 3). But how we can account for the perspectives that emerge from interactive and haptic (touch) enabled media with screens, tracking and audio to produce spaces around interacting bodies without frames in a complex multidimensional metalepsis?
Experiencing the space of Fyr (photo by Elin Andersson, more here)
By having more than ground or floor below us the dimensions of experiencing mediation change. When this experience is tracked with haptics and motion sensing, the body becomes part of the field of representation. As a sensory exercise this at first can be confusing, but it has a seductive power, and it becomes enjoyable to place the body in the field and play with a far away, but at the same time close place filled with objects, sounds and movements. Fyr by Kent Gustafsson, Fredrik Oskarsson, Jim Robertsson and Johan von Boer does this. It also provokes much thought about new forms of perspective and how stories can be told using augmented, media-rich, interactive environments.