Description

Audiomorph is an application that captures real-time audio from a microphone, analyzes it to extract both low-level and high-level features and sends these parameters to a web server. The webapp front-end then uses these features to generate dynamic visualizations that are synchronized with the audio, creating an interactive experience
Challenges, accomplishment and lessons learned: The challenges for the back end part were mostly about finding the right models to analyze the audio, and to make this process smooth. Regarding the front end, the main challenge was finding a way to represent the audio features graphically. A smooth, real-time analysis was accomplished, while the lessons learned there is space for improvement both in the analysis model (train it with new data, build a new model from scratch) and in the graphical representation, for example generating more unpredictable and representative visual.

Technology

• Back end: Python, librosa, pyaudio, essentia, flask
• Front end: Hydra Video Synth (JS library)

Students

Loïs Guerci: Elaboration of the idea, coding of the first draft of the prototype, development on the backend side

Richard Heikkilä, Madhav Gopi: Work on frontend side, generation of visuals, mapping of the audio/ HL values to the visuals

Manfredi Pletti: Backend development, project structure organization

Media