Developing software for HoloLens requires a workflow. Whether you start with modeling software or scripting or maybe writing C++, there’s some set of steps you follow from idea to implementation, where implementation in this case is viewing and using the software on the HoloLens.
As you start out there is a desire to create your own objects, not just view someone else’s creation however stunning their 3D work may be. After creating, you desire to start making objects move both with translations and rotations or other animations. Resizing and combing, and building upon previous creations are further desired changes to make, thoughts that all come in logical progression as you gain experience and confidence in the APIs. But there’s the workflow all the same.
In my case the workflow entailed Unity IDE to Visual Studio to HoloLens. It required certain versions of software libraries so everything worked. It required some time in each coding-to-testing iteration to flow the software through the build process, to go from end to end, time measured in minutes. And once running on the HoloLens minor imperfections could now in 3D space be readily seen, before hidden by 2D viewing limitations, often requiring yet another round trip to code and deploy the simple change. This issue gets compounded at times due to seemingly inconsequential issues or library incompatibility that may require lengthy troubleshooting to pinpoint. I became adept at using the various logging features available, a plus. But …
For me, this all resulted in the desire to remove that workflow as much as possible, it was just in the way and risky too. I wanted instead to work directly in the HoloLens environment, to develop directly in 3D virtual space. This would be my first minor epiphany on developing with the HoloLens: the back and forth needs to be eliminated, and can be eliminated entirely. HoloLens is a full fledged Windows 10 UWP device with a Holographic API that distinguishes it from other UWP devices. As well it’s got nice audio devices and matching APIs that work fairly well. Cortana, eh …, still needs some brushing up. But the HoloLens should support a development environment based on it’s hardware and APIs.
So this led to the question, “Why not work right on the HoloLens and eliminate much of the back and forth and as well eliminate the frequent software library conflicts?”.
Exploring that possibility led to development of a basic virtual-3D design and development environment running on the HoloLens as my first HoloLens app. This software offers 3D virtual spaces to create objects that can move and can be combined or resized in unlimited ways. With a built-in scripting engine even complex objects and systems can be created then recreated reliably and easily. This happens in a natural hands-free way requiring no awkward arm movements and no need to keep hands in the viewing area to work. The software adapts to how you use it by allowing you to tailor scripts with your vocabulary, your words. Repeating those words then runs the scripts and gives you full control of the virtual space. More 3D spaces can be added without disrupting existing ones, instantly, and combined into existing spaces maintaining their own movements and sizing.
The most visually stunning objects I’ve seen in 3D virtual space using a HoloLens are fractal structures, some of which are referred to as sacred geometries and frequently appear in historic documents and are found inscribed on or embedded in the most ancient structures found around the planet. These fractal structures can be more thoroughly visualized and explored in 3D virtual space than when using other available viewing media. And it is interesting to build fractal structures, to learn how a simple object or simple pattern so easily creates complexity when viewed at large scale. Add in rotations and other movements, instantly, and the possibilities start expanding.
At the start of developing the software, this interest in fractal structures led to the desire to easily display geometric objects such as a 64 star tetrahedron. But I struggled to write the code to create this object. Maybe there’s an algorithm somewhere giving coordinates of all the points but I didn’t look, I wanted to do it myself. Even building it out of pieces proved complex due to the various angles involved when fitting the pieces into exactly the right spot, lots of math, certainly possible but tedious.
While struggling with the math of it all, in the mindset that I’d need to provide code to create it, I just put it aside for a while and instead started playing. In effect I began experimenting in 3D virtual space to understand basic 3D commands and the moves needed to effectively, and simply, move the point of attention (a cursor more or less, aka POT, that moves around in 3D virtual space) to precisely combine and easily manipulate created objects.
During this time I discovered how simple the solution to this very complex math problem of creating a 64 star tetrahedron, rather how simple the solution when solved visually. The discovery really is that a small set of simple moves, especially simple when combined and repeated, succeeded in building a perfect 64 star tetrahedron. This happened using only a tetrahedron shape (4 star) with simple lateral moves and a single rotation, then combined and repeated in a simple pattern. The need for complex math slipped away, just not needed at all.
Understanding that solving some complex problems can become simple inside 3D virtual space is the major of the two epiphanies I’ve had so far developing with HoloLens and working directly in 3D virtual space.
This second epiphany gave further impetus to continue to develop the simple 3D virtual space design/development tool into something more reliable, easily expandable, and unencumbered by a 2D mindset (of monitors and using a mouse to rotate a 3D creation in 2D and the lengthy workflow to get to the device). And so did continue developing the tool, recently releasing it onto the Windows Store as a publicly available HoloLens application.
As an aside, through experimenting with both voice and hand interfaces, I found using voice commands to be more natural and more accurate as compared to using hands. Especially so given the limited field of view of the hand sensors and hands API performance along with the need for precision positioning required for perfect geometric reproductions, perfect even when magnified to 100X or 1000X normal scale, or reduced to 1/1000th scale, which with simple resizing commands can be done instantly in 3d virtual space. Voice on the other hand can be limited by noise, or improper pronunciation, but is usually more available and the voice command set more easily expanded than by using hand motions for the user interface.
What the software offers, if you have a HoloLens, is the immediate ability to create and manipulate objects in 3D virtual space, and easily compound your creations in unlimited ways. But most importantly, the result of the actual activity of creating in 3D virtual space is your mind releases life-long beliefs about constraints that don’t exist in 3D virtual space. It’s an experience that is mind expanding in ways you actually have to experience to understand. Read about it all you want, but experience it to truly understand, to actually open up and expand how your mind works.
Experience effective use of this software just for a few minutes and you start understanding the power of your own mind’s imagination has long been underestimated, and underused. Unleash your imagination!