| Glove-TalkII
GloveTalkII is a system that translates hand gestures to speech through an adaptive interface. Hand gestures are mapped continuously to ten control parpameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-TalkII uses several input devices (including a Cyberglove, a ContactGlove, a three space tracker, and a foot pedal), a parallel formant speech synthesizer, and three neural networks. One subject has trained to speak intelligibly with Glove-TalkII. He speaks slowly but with far more natural sounding pitch variations than a text-to-speech syntesizer. Glove-Talk Glove-Talk was also a gesture-to-speech system. However, unlike Glove-TalkII, hand gesture were recognized using neural networks. The recognized gesture was mapped to a word which was then spoken using a text-to-speech synthesizer.
|
|
|
|
Iamascope The Iamascope is an interactive multimedia artwork. The Iamascope combines computer video, graphics, vision, and audio technology enabling performers to create striking imagery and sound. The result is an aesthetically uplifting interactive experience. At an installation, the user takes the place of a colourful piece of floating glass inside a computer generated kaleidoscope, and simultaneously views the kaleidoscopic image of themselves on a huge screen in real time. By applying image processing to the kaleidoscopic image, the performer's body movements directly control music in a beautiful dance of symmetry with the image. The image processing uses simple intensity differences over time which are calculated in real-time. The responsive nature of the whole system allows users to have an intimate, engaging, satisfying, multimedia experience.
|
| MusiKalscope
The MusiKalscope is an attempt to provide balanced computer support for an interactive multimedia experience. The two media sources are sound and imagery. The visual imagery of the Iamascope is combined with the jazz improvisational system called RhYMe so that a performer can focus on one media and the other media will maintain a certain minimum quality level. |
|
