Progressing into 3D Virtual Space

Experience effective use of this software just for a few minutes and you start understanding the power of your own mind’s imagination has long been underestimated, and underused. Unleash your imagination!

Developing software for HoloLens requires a workflow. Whether you start with modeling software or scripting or maybe writing C++, there’s some set of steps you follow from idea to implementation, where implementation in this case is viewing and using the software on the HoloLens.

As you start out there is a desire to create your own objects, not just view someone else’s creation however stunning their 3D work may be. After creating, you desire to start making objects move both with translations and rotations or other animations. Resizing and combing, and building upon previous creations are further desired changes to make, thoughts that all come in logical progression as you gain experience and confidence in the APIs. But there’s the workflow all the same.

In my case the workflow entailed Unity IDE to Visual Studio to HoloLens. It required certain versions of software libraries so everything worked. It required some time in each coding-to-testing iteration to flow the software through the build process, to go from end to end, time measured in minutes. And once running on the HoloLens minor imperfections could now in 3D space be readily seen, before hidden by 2D viewing limitations, often requiring yet another round trip to code and deploy the simple change. This issue gets compounded at times due to seemingly inconsequential issues or library incompatibility that may require lengthy troubleshooting to pinpoint. I became adept at using the various logging features available, a plus. But …

For me, this all resulted in the desire to remove that workflow as much as possible, it was just in the way and risky too. I wanted instead to work directly in the HoloLens environment, to develop directly in 3D virtual space. This would be my first minor epiphany on developing with the HoloLens: the back and forth needs to be eliminated, and can be eliminated entirely. HoloLens is a full fledged Windows 10 UWP device with a Holographic API that distinguishes it from other UWP devices. As well it’s got nice audio devices and matching APIs that work fairly well. Cortana, eh …, still needs some brushing up. But the HoloLens should support a development environment based on it’s hardware and APIs.

So this led to the question,  “Why not work right on the HoloLens and eliminate much of the back and forth and as well eliminate the frequent software library conflicts?”.  

Exploring that possibility led to development of a basic virtual-3D design and development environment running on the HoloLens as my first HoloLens app. This software offers 3D virtual spaces to create objects that can move and can be combined or resized in unlimited ways. With a built-in scripting engine even complex objects and systems can be created then recreated reliably and easily. This happens in a natural hands-free way requiring no awkward arm movements and no need to keep hands in the viewing area to work. The software adapts to how you use it by allowing you to tailor scripts with your vocabulary, your words. Repeating those words then runs the scripts and gives you full control of the virtual space. More 3D spaces can be added without disrupting existing ones, instantly, and combined into existing spaces maintaining their own movements and sizing.

The most visually stunning objects I’ve seen in 3D virtual space using a HoloLens are fractal structures, some of which are referred to as sacred geometries and frequently appear in historic documents and are found inscribed on or embedded in the most ancient structures found around the planet. These fractal structures can be more thoroughly visualized and explored in 3D virtual space than when using other available viewing media. And it is interesting to build fractal structures, to learn how a simple object or simple pattern so easily creates complexity when viewed at large scale. Add in rotations and other movements, instantly, and the possibilities start expanding.

At the start of developing the software, this interest in fractal structures led to the desire to easily display geometric objects such as a 64 star tetrahedron. But I struggled to write the code to create this object. Maybe there’s an algorithm somewhere giving coordinates of all the points but I didn’t look, I wanted to do it myself. Even building it out of pieces proved complex due to the various angles involved when fitting the pieces into exactly the right spot, lots of math, certainly possible but tedious.

While struggling with the math of it all, in the mindset that I’d need to provide code to create it, I just put it aside for a while and instead started playing. In effect I began experimenting in 3D virtual space to understand basic 3D commands and the moves needed to effectively, and simply, move the point of attention (a cursor more or less, aka POT, that moves around in 3D virtual space) to precisely combine and easily manipulate created objects.

During this time I discovered how simple the solution to this very complex math problem of creating a 64 star tetrahedron, rather how simple the solution when solved visually. The discovery really is that a small set of simple moves, especially simple when combined and repeated, succeeded in building a perfect 64 star tetrahedron. This happened using only a tetrahedron shape (4 star) with simple lateral moves and a single rotation, then combined and repeated in a simple pattern. The need for complex math slipped away, just not needed at all.

Understanding that solving some complex problems can become simple inside 3D virtual space is the major of the two epiphanies I’ve had so far developing with HoloLens and working directly in 3D virtual space.

This second epiphany gave further impetus to continue to develop the simple 3D virtual space design/development tool into something more reliable, easily expandable, and unencumbered by a 2D mindset (of monitors and using a mouse to rotate a 3D creation in 2D and the lengthy workflow to get to the device). And so did continue developing the tool, recently releasing it onto the Windows Store as a publicly available HoloLens application.

As an aside, through experimenting with both voice and hand interfaces, I found using voice commands to be more natural and more accurate as compared to using hands. Especially so given the limited field of view of the hand sensors and hands API performance along with the need for precision positioning required for perfect geometric reproductions, perfect even when magnified to 100X or 1000X normal scale, or reduced to 1/1000th scale, which with simple resizing commands can be done instantly in 3d virtual space. Voice on the other hand can be limited by noise, or improper pronunciation, but is usually more available and the voice command set more easily expanded than by using hand motions for the user interface.

What the software offers, if you have a HoloLens, is the immediate ability to create and manipulate objects in 3D virtual space, and easily compound your creations in unlimited ways. But most importantly, the result of the actual activity of creating in 3D virtual space is your mind releases life-long beliefs about constraints that don’t exist in 3D virtual space. It’s an experience that is mind expanding in ways you actually have to experience to understand. Read about it all you want, but experience it to truly understand, to actually open up and expand how your mind works.

Experience effective use of this software just for a few minutes and you start understanding the power of your own mind’s imagination has long been underestimated, and underused. Unleash your imagination!

Advertisements

Hololens First Impression

A bit disappointed.

It is a hologram viewer with an API and toolset that is for bringing holograms to the user’s environment.

The biggest disappointment is that I came with an expectation of a device and associated toolset (APIs, IDE, etc.) that would augment, or more specifically, enhance the user’s current environment when wearing it. The Hololens doesn’t quite do that, at least as in my vision, not yet and won’t anytime soon.

The Hololens, as per it’s name, is really for interacting with or just seeing holograms, rather than for enhancing your reality in simple ways as you go about life. It’s more meant to just augment some very specific moments in very specific (and unchanging) places, but not until lots and lots of development occurs first.

I’d envisioned the Hololens could detect and track moving objects that aren’t holograms. Out of the box it doesn’t come close to this. It may come, and various technologies such as OpenCV may be useful, but this won’t happen until lots of new development first occurs.

There’s a lot more to say but that is a good summary at this point.

Mostly the setup and initial testing and initial development went smoothly.

Do find it annoyingly difficult to scroll browser windows. All else seems to work nicely.

But how can it be useful immediately.?

Well, since it is wearable, then the benefit is having two hands mostly free and browsing with voice.

That means Cortana comes into play, and being a user of Amazon Echo and OK Google, I find Cortana woefully behind in most every way.

In short, Cortana sucks, and is really hard to get value from this poor technology that unfortunately is baked into Hololens (but can be turned off).

Anyway, Hololens can be useful for cooking. For instance for displaying recipes while preparing food. I’ll try this and report back.

Hololens is currently designed to be useful in static spaces which you spend a lot of time in. Wearing Hololens out and about is mostly good for nothing.

So that’s another summary statement that is mostly on the mark.

I saw a u-tube of a guy wearing Hololens while driving in his car. He wasn’t doing anything with it, just wearing it and driving. That is the extent of the value, which is to have a heavy and somewhat awkward hat on, while driving. He apparently was willing to pay $3000 for his high tech hat.

At the point of writing this I am in the 200 series of Hologram Academy, but haven’t finished it yet. It’s an easy to follow course, but the most time consuming parts of these apps if they were started from scratch is the graphics/visuals, and in the training samples graphics are completed already by Microsoft. This more or less hides the great amount of development that must occur to bring original holographic apps to life. Sure 3rd parties will likely fill the void, but they’ll need to. My issue here is the enormous time it will take to develop the most simple apps.

Having worked a number of years with Dynamics CRM, from version 3 up to 2015, I was always amazed by the development costs, when it’s touted as an easily customizable application. Yeah, once the customization is done, it still takes a lot of development (code writing) to make it useful. I think Hololens apps will follow this path.

Will finish the academy demos and come back and write more, but this sums up first impressions.