Prof. Sha Xin Wei — PI
Prof. Marcelo Wanderley — PI
David Gaultier — mechatronics & feature extraction programming
Freida Abtan   — sound instrument creator (software)
David Birnbaum — sound instrument creator
Elliot Sinyor  — sound instrument creator
Harry Smoak — assistant project technical coordinator
Doug van Nort   — gesture/sound feature extraction, mapping
Rodolphe Koehly  — physical materials advisor 

As an extension of the research work conducted with the Topological Media Lab (TML), Sha Xin Wei and his team are creating textile objects such as wall hangings, blankets, scarves, and jewelry that create sound as they are approached or manipulated. These sonic blankets can be used for improvised play.  A phonetic pun on the old acronym for What You See is What You Get from the era of the Graphical User Interface, WYSIWYG (for wearable, sonic instrument, with gesture) draws on music technology, dance, children’s group games, textile arts, and fashion. Created first and foremost to sustain social play for people of all ages, WYSIWYG allows players to express themselves whether enjoying time in a park, dancing at a club, passing the time during a long car trip, or just playing at home. The research is being carried out in collaboration with Marcelo Wanderley, an associate professor at the McGill University Schulich School of Music in Montreal, and draws on Wanderley’s research into the gestural control of sound synthesis and new interfaces for musical expression.

The custom-designed digital instruments embedded in the cloth sample movement to transform ambient body movement and freehand gestures into new sounds or “voices” associated with a player or transmitted to other players in the vicinity. These devices can also be embedded into furnishings or other types of objects. In addition, they can store and re-synthesize sounds by nuancing them using data transmitted by nearby sensors. The research project therefore targets the creation of a series of devices – some made from soft material – that will react in different ways to proximity and contact, movements, noise characteristics, and the progress of the game itself. The sonic behavior of the devices are designed in the spirit of games such as hide-and-seek and blind-man’s buff and also work well with a variable number of players in both ad hoc and rehearsed situations.

When the project was launched in November 2006, the WYSIWYG team experimented with a prototype ”blanket” able to sense how it is handled. During the presentation, eight people manipulated this photo-sensitive blanket to produce a spatial sonic landscape. In July 2007, dancers performed a semi-choreographed movement improvisation around a 20’ suspended “tapestry”and a 6’ “tablecloth” woven with conductive thread on a Jacquard loom by Joey Berzowska’s XS Labs.

With WYSIWYG, Sha and his team intend to develop other architectural-scale sensate cloths that function as agents co-performing with dancers and as image-bearing, kinetic surfaces in other performance contexts.

Dancer Marie Laurier with 20’ sounding cloth woven by Marguerite Bromley during Ouija workshop. © 2007 Topological Media Lab.

Custom electronics by Elliot Sinyor, McGill University. © 2007 Topological Media Lab.

David Gauthier with capacitive proximity sensor in the form of a bird woven from conductive fiber. © 2007 Topological Media Lab.

Joel Ryan, Chris Salter

Sha, X.W., Serita, Y., Dow, S.,Iachello, G., Fistre, J.

Matthew Warne