No more putting it off. Let’s get this thing running in VR. A partial to-do list: give some thought to performance, support a head-mounted display and motion controllers, adapt my existing controls to two independent hands, and decide how the user will move around the space.

I want to make several performance-related adjustments, guided by the official documentation’s XR Best Practices and XR Performance Features. While I do not intend to run this on a standalone headset, I do want it to be as lightweight as possible to leave room for Rack itself. I follow most of the advice but leave bloom on. I’m making extensive use of emissive materials for the lights, and I find that without bloom the light colors are very dull and washed out (I also turn off auto exposure for lighting for the same reason).

Next up is creating an all-new Character class, making it use a head-mounted display (HMD) as a camera, and giving it motion controllers to tie the input setup to. For input, I decide to try out the new enhanced input system. It turns out this comes with a very steep learning curve, and as I have found to be typical with Unreal stuff, the documentation ain’t great. I eventually figure out how to work with it. It does seem valuable for swapping out groups of controls for different situations. For instance, the trigger button needs to do something different depending on whether the controller is interacting with a param, a cable or port, or neither. I don’t have much experience with the older input setup, but I’m pretty sure this stuff would have been much more difficult there.

Finally, I get to what I consider to be the fun stuff: user experience. As I had hoped, having the interaction logic already in place for mouse controls gives me a huge leg up when switching over to the motion controllers, and re-implementing those controls goes about as smoothly as I could have hoped. The one major difference is that we now have two (motion) controllers instead of one (mouse), so now everything needs to work with either hand, and both hands need to be able to work independently.

Importantly, this is the first time you’ve been able to put the modules wherever you want.

What I end up implementing is a system of controls where it feels like you can grab the world itself and move it around your body. With one hand, using the grip button, you can pull the entire world toward you or push it away, or slide it side to side. With the grip button and a modifier, you can rotate the world around you like you’re standing at the center of a carousel. Gripping with two hands, the world will both rotate and translate as if you have picked it up (like a laundry basket?) to carry around.

These controls expand and improve upon the controls in SYNTHSPACE (notably, by adding one-handed options), which is the only place I’ve seen controls like this. I find them to be a delight to use compared to what one usually finds in a VR application or game. It seems easier to convince a body that it can move the universe than to convince it that something like an analog stick could invisibly make it move. I’m very excited for other folks to try it out.

Finally, I decide to throw in (ie spend another entire week working on) a tooltip widget that follows each hand and shows the same information the mouse tooltip does in Rack. This helps a lot with fine-tuning any knobs or sliders, and, along with a little haptic bump, provides additional feedback to the user as to when they’re interacting and with which port or param.

And one last tiny thing. We have a body now; we need a floor to stand on.

I know there is still a ton of tweaking to do to make the controls feel just right, but I’m well on my way and excited to do more. And so, friends, with what ends up being another couple of months of work, we are in cyberspace.