Crisis or Joy?

Not every message about crisis events we read on social media is negative. Many messages about climate catastrophes, war and global health crises are about hope, courage and cohesion.

We developed an AI language model and trained with real musicians to transform voices into musical instruments. Together with locals from over 50 countries, we transformed tweets read aloud about disaster and joy events in the respective countries into melodies of crisis and joy in their local language.

Where was it exhibited?

The project was exhibited as a highlight at the internationally renowned media art festival “Ars Electronica Festival” 2022 and presented to a large number of the 75,000 total visitors. For this purpose, an interactive game was created in which the festival visitors had to guess whether the tweets transformed into music had a positive or negative message for the selected country.

To train the ear for the melodies, oversized musical instruments made of recycled plastic were set up and visitors were encouraged to try out the underlying ToneTransfer algorithm live with their own voices.

Who and Why?

The project was developed together with the Johannes Kepler University Institute for Computational Perception and funded by the province of Upper Austria as part of a Linz Institute of Technology (LIT) Art&Science Call.

Curious? Here is the interview with the Ministerium für Neugier & Zukunftslust.