Abstract

Atypical neuro-cognitive syndromes produce effects for people that are not often understood by neuro-typical people (due to the different neurological conditions). To fully understand the experience, there should be some form of simulation that demonstrates these effects. We can use the affordances of Digital Media and perform such a simulation depending on the perceptive systems involved in the cognitive process. If there are visual and auditory elements involved, we can simulate the mapping of one sense onto another. One example that illustrates this type of mapping would be a kind of music equalizer that maps the audio signals of the song to a visual representation in the form of an equalizer.

To develop such a project, I analyzed existing work on synaesthetic media experiences in order to develop a solid form of mapping. The next step was to computationally abstract the underlying cognitive process (of an atypical syndrome involving music perception) and built an interface through which people can experience the effects. I created an “Artificial Synaesthesia” device that performs such an interactive mapping and displays a visualization that allows users to gradually realize the result of the different mapping process. Through this experience, I hope to provide another perception of music for people by revealing to them the result of atypical cognitive processing (Synaesthesia) of music.

Code & Design Document

Code
Design Document