One of the final key components needed to have a meaningful experience with this thing is a way to load new modules into the patch: the library!

The library in Rack itself.

This turned out to be one of the more straightforward features to implement. The trickiest part was probably figuring out how to dump the library data from Rack, given that it is not considered part of the API and is therefore undocumented. I’d be more concerned about that if I wasn’t already locked into distributing a custom build of Rack—I don’t need to be worried about breaking changes catching me off guard.

I had been digging around off and on trying to get at the library data (ie, which plugins are installed and what modules they provide), and one day I stumbled upon a forum post mentioning exactly where to find it. From there it was a simple matter of looking at what happens when you click on a library entry in Rack to see how Rack itself spawns a module. I quickly had the fundamentals implemented—I could send a plugin slug and model slug (the required, unique, unchanging strings used to identify a given plugin and each of its modules) from Unreal to Rack, then have Rack spawn the module and send the info about it back to Unreal.

Now I needed to get all the library data over to Unreal. This includes the aforementioned slugs as well as the human-readable names, a list of available tags, the module descriptions and what tags they’re tagged with, and some additional metadata like favorites and usage statistics. I was initially sending this data in chunks with OSC the same as the rest of the data. However, this proved to be a lot of data to process that way which led to some hitches as it was loaded. In the end, I decided to dump all the library data into a single JSON file and then send a message to Unreal with the file’s path once it was created, resulting in a much quicker parse and near-instant-feeling load on the Unreal side.

The last step, then, is building out an actor with an attached widget to display all the library information and handle the UI for selecting, sorting, filtering, and favoriting. I had done some basic work with Unreal widgets for the tooltip widget, but this was a much more complex UI to tackle. The Unreal widget system, UMG, is weird! I’m not sure what it is based on if anything, but it is unlike anything I have seen before. And, as usual with Unreal, the documentation leaves some things to be desired.

That said, I am a guy who has done a lot of front-end work on the web, and if I can make CSS do what I want it to, then how hard can UMG be? There was a lot of trial and error, and I found a lot of excellent advice and tutorials from Ben UI. In the end, it wasn’t that bad and I even came to appreciate some aspects of it.

Along the way, I figured out how to take screenshots directly from Unreal. It’s nice to have the full render sometimes and not just the cropped, single-eye version I’ve been screenshotting up to now.

So we now have near parity with the features of the library in Rack. It is currently missing sorting, type-to-search, and preview images. Sorting should be pretty easy but might require a little more data collecting. Type-to-search is going to require a VR keyboard widget, and that’s a whole other thing, but I’m going to need it for file handling, at least, so it’ll be coming eventually. Preview images were made a lot more possible once I noticed that there’s a command line flag for generating screenshots of every installed module, which will come in handy. Coming Soon™ I guess!