EMS Proceedings and Other Publications

Gesture and Sound Analysis: Virtual Instruments for an Interactive Composition

Simonetta Sargenti

Simonetta Sargenti, Departement of Music Theory, Conservatorio di Musica Stanislao Giacomantonio Cosenza, Italy


PDF - 439.2 kb
Download the paper


This project is focused on a description of most important actions related to playing a string instrument and allows us to build an interface simulating the playing actions needed to produce a sound and to compose a music piece in real time.
To obtain these results and to map the interface we analyzed:
1- the bow strokes used to play a string instrument
2- the partial components of violin sounds
3- the formal structure of a violin important composition (in this case a Caprice of N. Paganini)

With the interface we can also:
1- to use the same gesture and procedures used on the instrument to producing sound
2- to produce instrumental sounds simulating these of the real instrument
3- to compose a music piece with references to the historical repertoire of the simulated instrument
4- to compose a new musical piece or to improvise music

The interface used is a Nintendo-Wii controller simulating a violin bow to perform music with sound synthesis in real time.
Sound examples are built using frequency modulation synthesis and granular synthesis in a first version of the project. In a second one recorded audio files modified in real time are used too. In the first version of the project the formal reference for a new composition was the form of the Caprice in particular, Paganini’s Caprice for re-compose a new music piece. The second version is more oriented to improvisation. The following table shows the different kinds of analysis and their relation with the map of the interface.

Gesture Analysis  Bow movements map  Interface’s definition 
Spectral Analysis  Partials component map  Algorithm’s definition 
Formal Analysis  Piece’s sketch  Piece’s re-composition 

In an analytical point of view of the project we obtain:
1- a specifically analysis of the movements producing sound on a string instrument
2- a spectrum analysis of partial components of the violin sounds

In an aesthetic point of view we obtain:
1- the possibility of a re-composition of a musical piece starting from an historical example
2- the possibility to create a new composition or improvisation

EMS12 Proceedings