Sonification as therapy
- creative field installation, interface
- Client Name Till Engert, Steffen Rummel
- Project URL vimeo.com
- Completion Date 11/22/2015
As part of a research project guided by Prof. Jörg Remy an improved rehabilitation program of arm motivity by real time sonification of arm movement by apoplexy patients is been intended.
Under supervision of Dipl. psychologist Dr. Daniel Scholz from the institute for music psychology and music medicine at the college for music, theatre and media in Hanover this therapy form is been developed by us. This way it is possible to improve the arm motivity based on musical memory traces. Additionally the motivation for the training is been improved. For this, we use a stereo vision distance sensor (LeapMotion), the IDE Max/MSP as well as the audio software Ableton Live.
The sensor captures the patients arm movement, which is then translated into MIDI data via Max/MSP. These MIDI-Data is used to control a virtual instrument in Ableton Live. The vertical movement corresponds to the notes of a scale which can be chosen by the therapist. By moving horizontal various instruments can be played. In a typical exercise the patient is trying to play a melody which corresponds to a certain arm movement and can be picked by the therapist. The resulting movement data can also be saved for statistical analysis.