|
WYSIWYG |
|
|
|
Prof. Sha Xin Wei — PI
Prof. Marcelo Wanderley — PI
David Gaultier — mechatronics & feature extraction programming
Freida Abtan — sound instrument creator (software)
David Birnbaum — sound instrument creator
Elliot Sinyor — sound instrument creator
Harry Smoak — assistant project technical coordinator
Doug van Nort — gesture/sound feature extraction, mapping
Rodolphe Koehly — physical materials advisor
| As
an extension of the research work conducted with the Topological Media
Lab (TML), Sha Xin Wei and his team are creating textile objects such
as wall hangings, blankets, scarves, and jewelry that create sound as
they are approached or manipulated. These sonic blankets can be used
for improvised play. A phonetic pun on the
old acronym for What You See is What You Get from the era of the
Graphical User Interface, WYSIWYG (for wearable, sonic instrument, with
gesture) draws on music technology, dance, children’s group games,
textile arts, and fashion. Created first and foremost to sustain social
play for people of all ages, WYSIWYG allows players to express
themselves whether enjoying time in a park, dancing at a club, passing
the time during a long car trip, or just playing at home. The research
is being carried out in collaboration with Marcelo Wanderley, an
associate professor at the McGill University Schulich School of Music
in Montreal, and draws on Wanderley’s research into the gestural
control of sound synthesis and new interfaces for musical expression.
|
 |
The custom-designed digital instruments embedded in the cloth sample
movement to transform ambient body movement and freehand gestures into
new sounds or “voices” associated with a player or transmitted to other
players in the vicinity. These devices can also be embedded into
furnishings or other types of objects. In addition, they can store and
re-synthesize sounds by nuancing them using data transmitted by nearby
sensors. The research project therefore targets the creation of a
series of devices – some made from soft material – that will react in
different ways to proximity and contact, movements, noise
characteristics, and the progress of the game itself. The sonic
behavior of the devices are designed in the spirit of games such as
hide-and-seek and blind-man’s buff and also work well with a variable
number of players in both ad hoc and rehearsed situations.
When the project was launched in November 2006, the WYSIWYG team
experimented with a prototype ”blanket” able to sense how it is
handled. During the presentation, eight people manipulated this
photo-sensitive blanket to produce a spatial sonic landscape. In July
2007, dancers performed a semi-choreographed movement improvisation
around a 20’ suspended “tapestry”and a 6’ “tablecloth” woven with
conductive thread on a Jacquard loom by Joey Berzowska’s XS Labs.
 
With WYSIWYG, Sha and his team intend to develop other
architectural-scale sensate cloths that function as agents
co-performing with dancers and as image-bearing, kinetic surfaces in
other performance contexts.
Dancer Marie Laurier with 20’ sounding cloth woven by Marguerite Bromley during Ouija workshop. © 2007 Topological Media Lab.
Custom electronics by Elliot Sinyor, McGill University. © 2007 Topological Media Lab.
David Gauthier with capacitive proximity sensor in the form of a bird woven from conductive fiber. © 2007 Topological Media Lab.
|
|
|
Performance Research |
|
|
|
Critical studies of performance
in responsive media spaces, based on installation-events designed as phenomenological
experiments.
Livia Daza-Paris
 |
 |
Wet Petal
|
Visible Silence
|
Wolfgang Reitberger
Affective Dynamics in Responsive Media Spaces
Harry Smoak
Thick/N responsive space
Sha Xin Wei
A TGarden As a Phenomenological Experiment
|
|
|
Hubbub |
|
|
|
Hubbub is an investigation of how accidental and non-accidental conversations
can by catalyzed in urban spaces by means of speech projected onto public
surfaces. Hubbub installations may be built into a bench, in a bus stop,
a bar, a cafe, a school courtyard, a plaza, a park. As you walk by a Hubbub
installation, some of the words you speak will dance in projection across
the surfaces according to the energy and prosody of your voice. We'll
capitalize on recognition errors to give a playful character to the space.
Hubbub is one application of TML research treating speech as a computational
substance for architectural construction, complementary to its role as
a medium of communication.
Success will be measured by the extent to which strangers who revisit
a hubbub space begin to interact with one another socially in ways they
otherwise would not. Hubbub is a part of a larger cycle called URBAN EARS,
which explores how cities conduct conversations via the architecture of
physical and computational matter.
|
|
|
TGarden |
|
|
|
TGarden is an investigation of how people make sense of and
navigate in rich and dynamically evolving media spaces. Given the rise
of ubiquitous computing and realtime media synthesis, we're anticipating
the need for coherent yet supple ways for designers to create such complex
interactive media spaces and for people to inhabit them.
In a TGarden space, visitors wearing instrumented clothing creates and
modulate video and sound based on their gesture and movement. In effect,
visitors write video and sound by their movement.
For 2001-2002, we concentrate on using wireless sensors on the body to
track gesture. We have built a state evolution system that responds continuously
to sensor statistics, synthesizes and marshalls media in realtime.
In TGarden spaces, we use a combination of costumes outfitted with sensors,
video tracking, realtime sound and video processing, and gestural pattern
tracking.
Research concerns include the design of continuously varying narrative
spaces, how people improvise meaningful gesture, and factors of tangibility
and coherence such as latency, temporal (musical) texture and rhythm.
Our goal is to come up principles of design that should be useful for
creating and inhabiting responsive media spaces.
This research thread parallels a series of international productions in
Europe and the United States.
Links:
SIGGRAPH2000 - New Orleans
http://sponge.org
http://www.f0.am/tgarden
|
|
|
