Task 3
Name: Singing to musical score transcription and music composition
Coordination: FEUP/UCP
Duration: 24 months
Task description
To a significant extent, TASK3 runs in parallel with TASK2, and the objective is to specialize the software environment developed in the context of TASK2 in order to offer new interactive functionalities, namely (real-time) automatic transcription of singing to music score and didactic music composition using singing voice only as input. It should be emphasized that the input is an acoustic signal (the singing) that is captured by a microphone, then it is converted to a digital format (Pulse Code Modulated samples, i.e., linear PCM) which is used by the digital signal processing algorithms. Thus, in this task the input is non-semantic but the output is semantic because the music score identifies the music notes and their parameters.
In this case the MIDI (Musical Instrument Digital Interface) protocol will be used to represent the symbolic notation of music. Since in reality this consists of a set of musical parameters, MIDI is editable which means that after the transcription of singing to music score, the user is allowed to modify or correct the automatically recognized melody line and all its individual music notes. The final score can therefore be played back using any synthetic instrument that is allowed by the MIDI protocol. This is the basic didactic functionality that will be implemented in the context of this task. It will also be enhanced in order to allow the repetition of singing voice exercises that are assigned to different music instruments. Combining the results together leads to a practical music composition functionality taking singing voice as the only input. This task will bring together engineers (FEUP), musicians and interactive system designers (UCP), and will be developed in close collaboration with Casa da Música in Porto, whose representatives have already expressed their strong interest in such a scenario for workshops, in the perspective of the educational mission of Casa da Música. These workshops will be organized for the general public interested in learning the basics of singing, music notation and music composition.
Expected results
The main results of this task are 4 reports (one report every six months), one international conference paper, and one prototype software environment.
Human resources: 31.85 person-month.
