Making trees talk in Søndermarken

For this years Bloom festival we were fortunate to be given the opportunity to show an art piece in the beautiful setting of a small clearing surrounded by tall trees.

Through talks with the festival team and internally with each other, we arrived at the idea of a near-future scenario where humans have, using artificial intelligence, given the trees the ability to talk. A wonderful and playful idea with plenty of chance to build something really interesting and engaging, but of course also with many complications

Giving trees a voice

In our previous interactive LLM works where we used local text-to-speech synthesis, we had been using Kokoro, which is a really fast and open source model. It is great for many reasons, and especially for its vector based voices, giving us the option to making our own mix of voices. But creating the right mix for a personality was a trial and error process that took a long time, and the voices came out okay for most things, but not great.

For this piece we wanted something with more personality, for the lack of a better word. We wanted the audience to feel that they were talking to a tree and not an AI agent acting like a tree. So we sought for a new TTS model that could give us this, while still having low enough latency to be used in a real-time setting.

And the tool for the job ended up being Chatterbox. A new open source TTS model with zero shot voice cloning. This awesome project allowed us to, instead of trial and error, record our own interpretation of what a tree would sound like - resulting in some fun days in the office. A big thanks goes out to the programmers in our sister company NobodyWho who literally stepped up to the microphone, to give us their best tree impressions.

We think that what we arrived at really helps to give the work presence. Here is an example of the work, and you can enjoy the wonderful acting from one team member.

Reference audio:

Chatterbox audio:

TouchDesigner as the main controller

This is our go-to program for making installations. The flexibility and speed that it offers makes it ideal for us. You can never account for all things, and some things always need to be adjusted - lights need to be brighter, audio louder etc. And with this, we can do it on the fly and at runtime. The experience does not need to stop, we can monitor and improve where needed with no downtime.

The network ended up a little more messy that we would have liked due to last minute changes, but at the heart of it is three agent OPs that have their own self-contained sound, light and LLM sub-OPs.

There is a lot to cover in the network - whisper streaming, VAD, LLM api calls and token streaming and something that we built just for the project: TTS over local webrtc.

We wont go into it here, this post is already getting too long, but it did take ~0.8 sec off of the latency from our TTS implementation. Which is pretty good ! We might do a separate post on that another day.

Creating biotech amplification devices

For the sculptures, we partnered up with the wonderful team at 10-TONS, who are experts in creating sculptures of creatures, both real and fictional. We had some great talks about what would be doable and jammed ideas. After we had talked with them about the concept and looked at materials that would work well for the work, they did their magic and sent over the first 3D concepts.

The sculptures came out beautiful and really fit great with what we had envisioned for the biotech amplification devices, and with the light pulsing through them, they really set the scene nicely for the work. For images of the finished sculptures, have a look at the works page.

Installing in the forest

The trees in Søndermarken are protected, meaning that we were not allowed to damage them in any way. This meant no drilling, or hanging that could overly strain the trees branches.

So for the 6 speakers we used special tree mounts, that attach around the branches like belts. Then we used the mounts as a point to hang the sculptures from. Not easy work but the clean look was for sure worth it.

To end this post enjoy some images of the team installing in the forest.