April 28, 2023

don’t say let there be lights don’t say let there be lights

For the next step in sending data from Rack to Unreal, we’re going to start getting real-time with light updates. First, I need to get the position and size data for the lights over to Unreal like I’m doing for everything else, and while I’m at it, I’ll send the color and brightness of each light as it is at the moment I grab the position data. For now, I’ll render these as tiny point lights floating in space, but once we have actual 3d meshes these will likely be emissive (glowing) textures.

a screenshot of a computer screen showing a software modular synthesizer app on one side, and a crude recreation of that interface, including glowing lights, in a game engine on the other side.

let there be lights!

Next up, I need to continuously collect the current color and brightness values for each light and send them over to Unreal. I’ll compare the current values to the last ones I sent and only send an update if anything has changed. To do this, when I collect the initial data for each light I’m also storing a reference to the light so I don’t have to go digging through everything every time. Later on, I’ll need to manage this list to avoid null pointers, but the efficiency now is worth the tradeoff.

The main process loop in a Rack module runs at audio rates, which means, depending on the user’s settings, it’s running potentially tens of thousands of times per second. I need 60 FPS at best for the lights, so I throw a process divider in and only check for light changes every sampleRate / 60 runs. With that, the lights are a-blinking.

However, one last problem remains: Rack’s built-in CPU meter, which I had noticed typically sits around 2-4% with the default patch, is now at 30%. My initial, naive approach to gathering the light data and sending the updates is woefully inefficient. I can think of things that could potentially help remedy this, but I don’t know how to do them yet. In the end, I implement two major optimizations:

One, a threaded queue. I don’t have much (any!) experience with concurrency. But I know the main process thread in Rack, running at audio rates, should only be concerned with doing audio things. So I learn how to make a queue in C++, and I learn how to manage threads, and I have the process thread dump a command into a queue that another thread can pick up and do the work on. I have earned strange new programming scars. I feel like the King of Concurrency.

And two, bundled updates. This one was much easier. OSC can either send a single message, like a POST request in TCP/HTTP-land, or it can bundle a bunch of messages together with an optional process-these-at-this-time timestamp. Unreal happily pulls apart a bundle and sequentially processes each message contained within. I am currently sending each light update as a single message. So all I need to do is learn how to bundle the light update messages on the sending end. Easy.

The CPU meter is happy, I have learned things, and the lights blink better than ever.