At syGlass I have created VR tools used for intensive scientific study, completed an exhaustive overhaul of our VR user interface, and added developer tools such as a Python API interface (custom Terminal, source file organization, etc.) using the Windows Forms UI toolkit.

New Features

As of this writing my estimate for the number of features I have implemented is somewhere between 20 and 30 depending on how you count them. I have listed some of the more interesting ones here:

• Tutorial system: A set of features and tools for creating and playing back in-engine tutorials. This same system can be used by educators to create in-VR lectures for students.
• MP4 playback: Want to play a video inside your VR space? Import any MP4 file into syGlass and select to play it. The video plays on projectors inside the workspace. Videos can be added inside the tutorial system, whether to guide users on how to use syGlass or for educators to play for their students during in-VR lectures.
• Macro-molecular visualization: Choose files in PDB or mmCIF format to import molecular data representing anything from proteins to whole viruses consisting of millions of molecules. Point your controller at any given molecule for a better view of it and all the numbers associated with it, whether that’s its coordinates in space or the sub-structure it belongs to.
• Scatter plot tool: An interactive 3D/4D scatter plot. The user can reach out and grab individual points on the scatter plot to see more details about that particular data point. The source data for the scatter-plot comes from statistics of the current on-screen visualization or a user CSV file.
• Assisted tracing tool: Generates a set of points and paths along perceived lines in volumetric data, such as ones representing human nerves, and allows the user to trace desired paths with their VR controller. This greatly eases the work of scientists that need to annotate these types of datasets.
• 2D Segmentation tool: For scientists that need to easily perform axis-aligned slices of their data. Just grab the handle on the first slice that is generated and move it along the chosen axis to view any particular segment of the data. Switch axis using shortcuts on the VR controller.
• Python API interface: We have a Python API that allows developers and more advanced users to write code to interface with syGlass. This 2D interface makes it plain simple to use the API. It includes script file organization and a Terminal for viewing the output of the script.
• Generalized Histogram Thresholding: A one button press to instantly threshold any volumetric data, making it a snap to get correct visualization settings without spending an hour making manual adjustments.

Improvements

Besides new implementations, I have completed countless fixes and improvements to the following areas:

• Engine
• Rendering
• UX/UI
• Controllers
• Audio
• Networking
• Existing tools
• 2D WindowsForms interface

Menu Overhaul

There are few times in my career that I was as ecstatic as when I received word that we were going to redesign our virtual reality menu system at syGlass and I would get to lead the project. With no designers on board yet, this meant I had the special opportunity to not only code but to design. Here are pictures showing before and after results and in-between design concepts. I plan to do a more comprehensive writeup on the process sometime in the future.

Before and After:

Here are some pictures showing the progression, from early Photoshop and Figma prototypes, to an implementation that is starting to come together:

Looking Forward

I have started the process of modernizing our core engine by moving from OpenGL to Vulkan. Our primary goal for this modernization is to gain access to hardware ray-tracing to speed up our most intensive computation: ray-tracing large volumetric images. Since we have a separate render pass for the ray-tracing, we will first add Vulkan in the form of OpenGL-Vulkan interoperation, with Vulkan handling the ray-tracing render pass using hardware ray-tracing support. Once this is in place we can move the remainder of the codebase to Vulkan (as makes sense given our limited resources) to achieve additional modern features such as frame generation.