Before complicating things with VR motion controllers, I want to make sure I have the basic logic behind user interaction in place using only a mouse pointer.

Each param type—knob, button, slider, or switch—is a subclass of the Param class. They share a common interface, comprised of three methods: Engage, Alter, and Release. Engage is called on click and handles initial setup, usually passing in the initial mouse position for reference. Alter is called each frame as long as the mouse button is held down, and we pass in the current mouse position each time, calculating the difference between frames and turning the knob or moving the slider. Finally, Release is called on release of the mouse button and handles any cleanup or resetting of state. For buttons and switches, only Engage and Release are used.

In addition to simply turning the knob or moving the slider, I also have to then calculate the actual value of the parameter based on the rotation or position of the mesh. This gets sent back to Rack where the actual value is updated. Careful attention is paid to the way click and release work for buttons and switches on the Rack side so I'm sending value updates at the same point those values would be changing if the user was using Rack directly.

For cables, it’s a matter of handling clicks on Ports, spawning Cable models on the Unreal side, and whenever a cable is connected or disconnected, sending a command to Rack to create or destroy the cable. I pay careful attention here to how Rack determines whether a cable is connected or disconnected (ie, when an audio signal or control voltage is flowing through it), and take care to exactly reproduce the behavior to avoid any surprises for future users.

This all takes about a month of work. I have gone beyond simply reproducing the state of Rack in Unreal to starting to recreate the experience of using Rack in 3d, which feels amazing. We can finally interact with the things! I’m beginning to think I might be able to make this work.