Towards an Interactive Framework for Robot Dancing Applications

Download Thesis Last Updated in 07/07/2008

Abstract:

The process of music listening is one of continuous transformation, in which different agencies of musical expertise monitor the input signal and produce various output signals in the forms of behavioural responses. These responses may be overt, through dance movements or mere foot-tapping behaviour; emotional, as in tension and relaxation reactions; or entirely immanent, though recognition and classification of instruments, genres, and composers. Psychological and sociological research suggests that the embodiment of rhythm, through dance movements, is an elementary and pervasive organizing principle for social human interaction. In this dissertation we present the development of a robotic system using a humanoid robot, based on the Lego Mindstorms NXT, which tries to simulate the human rhythmic perception from complex audio signals and its reactive behaviour in the form of dance. To do so we developed a framework decomposed in three modules: Music Analysis, Human Control, and Robot Control, which are parallelly processed, through a multithreading architecture, to induce a robotic dance performance in a reactive behavioural-based approach. The Music Analysis Module performs a low-level rhythmic perception based on the Marsyas (an open source software framework for audio processing) onset detection function, with peak picking and adaptive thresholding. The Human Control Module consists on an interactive graphical interface for dance and audio parameters definitions, which grants the user deterministic role in the behaviour of this system, while assuring its dynamic reaction. The Robot Control Module incites the robot to react, in real-time, to the received rhythm and sensorial events (namely the colour stepped on the floor or the proximity to some kind of obstacle), embodying the previously defined dance. The resulting dance alternates in a seemingly autonomous manner between a diversity of motion styles coupled to the musical rhythm, and varying in consonance with the colour stepped on the dance environment, without any previous knowledge of music. A dynamic and interactive behaviour, in compromise with synchronism, assures an interesting relationship between a human and an artificial agent in the long-term.

FEUP 2008