Commentary on HoloLens development.
It’s not just space to view holograms … 3D virtual space is where you expand your mind without limits, where you bring your imagination to life!
Commentary on HoloLens development.
Still challenged, but no longer confused.
This post describes the process I went through while solving a problem using HoloLens as a tool with which to manipulate 3D virtual space, but most of the post isn’t about that. However, this is the second time of quickly solving a complex problem using 3D virtual space to solve the problems visually. So the HoloLens has been of enormous value as it turns out, as initially I was somewhat skeptical.
The majority of this post describes what happened when I attempted to exploit some of the capabilities of the 3D engine used with HoloLens. In particular I planned to build an actual Earth model including the heightmaps of all the earth’s terrain derived from upper orbit high resolution satellite images. The rest is not what you’d expect.
Please consider watching this video before reading any further. That guy is talking about his life’s changing moment. It’s about 18 minutes. If you watch it all then all it means is you have an open mind. Truth does not fear investigation, and the HoloLens is what I used to investigate some of the truths about this issue.
In short, software developer here. Got interested in the HoloLens and ended up writing an app that is fairly simple in concept, but does a few things differently than other apps. It’s not flashy or with lifelike graphics, rather it’s a tool to open one’s imagination and illuminate possibility in 3D virtual space (yes, virtual space where absolutely life-like dolphins can instantly pop up out of thin air, for instance right in front of you in the grocery store parking lot or while walking along in the desert). The app, Instant 3D Builder, is more or less a basic design studio allowing the user to work directly in 3D virtual space. This is a better user experience and more efficient workflow than going back and forth from a 2D workstation where all the work gets done and then deploying to HoloLens for viewing and testing. This app makes the HoloLens the tool rather than just a holographic viewer.
To get the app to work comfortably over long periods of use, so it’s comfortable for the user I mean, it was necessary to figure out some user interface issues and I chose voice as the primary interface rather than hand motions. To efficiently do work with the app it was necessary to solve problems like how to move a cursor around with high precision in 3D virtual space using only voice. To make the app easily adaptable a voice driven scripting engine was added. The scripting engine with just a small set of voice commands ends up making the app flexible and powerful. One interesting action the app does nicely is to allow rotating objects to be combined simply and instantly, so animations such as solar systems can be quickly modeled, and these models can be instantly recreated and observed in real time with full motion.
The current public release, the Sacred Geometry edition, at this time offers only simple shapes and is focused on the tetrahedron as the primary 3D shape. The complete version is still work in progress.
While discussing progress for the complete version one day, going out on a limb so to speak, I boasted that I could replicate Galaxy Explorer, and make it even better in just a couple days using this tool. This with the stipulation that as long as all the graphics were pre-imported I could do it. And thus, I set about gathering up a good set of images of the planets, but in particular Earth, as well as mockups of the Milky Way and some good star patterns of deep space. I was hoping to get images with good depth information as well.
First, I decided to get actual satellite pictures of Earth from space rather than settling for the Photoshop mockups I’d find in all the sample code. While the mockups are really nicely done with clouds that move and night and day maps as the planet rotates around the sun, yes everything that makes the Earth very lifelike as per the movies, for instance as in the recent movie Gravity, I wanted the Earth for my app to be actually real, meaning visually real, real as in undoctored or unretouched photos. And especially I didn’t want the spherical mockups, I wanted to make the Earth truly identical to it’s true form, so to be pear shaped as described by the famous physicist Neil deGrasse Tyson. This would be the crowning achievement to make it better than Galaxy Explore and prove my app’s talents and capabilities.
Mr. Tyson says of the Earth, “It’s not actually a sphere, it’s an, it’s oblate. Officially it’s an oblate spheroid, that’s what we call it. It’s slightly larger below the equator than above it. … It’s pear shaped.” And so, YES, that is exactly what I wanted to create, as perfectly as possible. Everything else, all the other apps, just use perfect spheres much like the pictures NASA provided during the lunar landings and other space missions in the 1970s. I was going to make the pear. I now had my mission well defined.
In this story, here’s where things got strange. As my search started, and went on and on, I couldn’t find any sources of unretouched photos of Earth from space. It wasn’t just about missing depth data, rather, real and actual photos of Earth from space are simply not available. This was unexpected, I’d literally expected to find millions upon millions of such photos. I’d even thought about algorithms to piece the images together over the 360 degreed spherical Earth. And I’d hoped to find coordinated images, or correlated images that capture the whole Earth from 3 or 4 locations at once to get the full 3D image and with depth information. Nothing found, nothing of the sort. Not available. However there are lots and lots of claimed images, but dig in and it’s discovered they’re generated not actual.
So, as a human race with 50+ years of space experience, we have satellites galore, Hubble telescope, space missions, space stations, we have rovers driving around Mars, etc., etc., but not a single darn actual photo of Earth from space. What IS readily available through are totally awesome, totally realistic and perfectly tailored computer generated images. Yes, they’re oh so lovely, colorful, amazing to view, and just downright attractive and even more alluring, like a lovely mystery lady beckoning from afar to a horny sailor.
But, not lured in just yet to use generated images, I keep searching and investigating and keep ending up going down rabbit holes that would typically lead toward conspiracy theories about fake Nasa missions, fake Nasa photos and so on. I’d always believed that of course photos from Nasa were enhanced due to strange light physics in space or with the atmospheric barrier or something similar, or enhanced to bring out feature and color detail, but also thought surely raw photos would be available. Surely. Surely! But … No, no raw photos are available. And it turns out, somehow Nasa has lost all sorts of data, telemetry tapes of all the lunar missions, original footage of lunar landings. Just the sort of thing you’d think they’d preserve just like the Constitution or Bill of Rights. I mean it’s the original data of all the moon missions, and it’s all gone. All missing.
Ok, so I set this Earth image hunt aside for a while to focus on getting the Earth model right sized, meaning sized from a perfect sphere to becoming an oblate spheroid, just as Mr. Tyson said. And what do I find? No actual data. It’s all just a theory it turns out. And back to the images for a minute, I found the images Nasa keeps churning out today are still those of perfect spheres, exactly as provided back in the 1970s, not a single oblate spheroid to be found, just perfect spheres in all their generated images.
While Nasa does provide perfect spheres to represent Earth in all their images, Nasa doesn’t provide exact images in terms of continental dimensions, as some of their releases of Earth images have changed the dimensions of various of the continents. And for some reason, Nasa’s images of the moon landing never show stars in the background, just saying. Strange light physics I guess, perhaps Nasa could enhance those moon photos to add in stars to make it more realistic for us viewers? Also, there are conflicting statements made about whether or not stars are visible from space.
In the midst of all this, perhaps as I kept ending up on conspiracy theory sites, I did a sort of cheezy video about how the pyramids in Egypt and the pyramids in the Yucatan Peninsula line up perfectly to the vertices of a perfect regular tetrahedron centered perfectly on the center of a perfect Earth sphere. And created this video using the HoloLens and my app.
Also at the time I came across interesting info about how so many dignitaries of the world were suddenly, as in never like this before in history, visiting to Antarctica, all within just the last year. And also came across information about a supposed discovery of pyramids in Antarctica. And as it turns out the 3rd vertice of the perfect tetrahedron, centered perfectly on the perfect sphere of the perfect spheroid Earth points there where the supposed pyramids in Antarctica are located. The 4th vertice of the tetrahedron points to a location in the Pacific north of Australia, and I’ve not gone to research what might be there, but very potentially pyramids. All this is quickly tossed into the video.
So, where am I? I’m totally perplexed by the lack of actual Earth images from space, I’ve become convinced Nasa is up to something just not sure what. And I’ve got this tetrahedron pointing out how the pyramids are so well aligned, all very nicely visualized in 3D virtual space. And I’m at times in conspiracy theory land.
Being a tinkerer at heart, and an engineer by training, I started tinkering around with the Earth model I’d been using, tinkering around with it in 3D virtual space. And I began to wonder how to model or handle issues of zooming in on a curved body of water on the planet Earth from afar off in space, and slowly zooming in towards Earth to the point where it starts looking flat. In 3D virtual space you can do these things instantly, at least with the right tools. So, I was experimenting with very large planes, the flat planes not the flying kind. I’d make the planes very large, a few hundred meters at first, then many miles to test the extents of what the 3D engine coupled with DirectX 11 could reproduce or visualize.
And, in 3D space, it’s cool to visualize, to see in real time, how a perfectly flat and nearly limitless plane, absolutely and perfectly flat and precisely so because it’s made in 3D virtual space, even when located far below me, would still rise up to my sight level. This is a trick of the eye, well not so much a trick as just how the eye works with perspective. Exactly like when looking down a long hallway and the ceiling, the floor and walls all converge at a single point in the distance.
Next, I did this with a very large sphere, but something different happened than when using a plane. The large sphere didn’t quite act as the plane did, even when very large, even when so large that at shorter distances it should appear as a plane, or so I thought.
Consider, I’d position this gigantic sphere so as to locate myself precisely at the sphere’s highest point, which is a single tangential point. To do so, I’d be standing in a room, but in 3D virtual space the sphere could be visualized to be placed directly at my feet or much lower. Because it was a very large sphere, it did act partially as a plane, but absolutely not entirely so. The sphere wouldn’t quite rise up to my eye level like a plane. This because every point more distant than the previous was in fact ever so slightly lower (the formula for the drop in elevation as distance increases is 8 inches per mile squared, so four miles of distance, would be 8”/mile sq. * 4mi* 4mi = 128” or 10’8” of drop at 4 miles of distance on a sphere of about 8000 miles diameter).
So the large sphere doesn’t rise up to eye level like the plane does and this can be visualized. It’s very noticeable especially when the plane and sphere are created and visualized simultaneously. There’s no other way I know where it’s possible to truly visualize this phenomenon except in 3D virtual space, and HoloLens does this visualization very well indeed.
After doing these experiments with HoloLens visualizing a gigantic sphere and nearly limitless plane, here I am wondering why does the Earth somehow act like a plane to my eyes, but was of course a sphere, as we all know so very well even from our very first days of kindergarten.
Then I recalled any of the hundreds of flights I’ve been on, all over the world, where no matter how high the plane, the Earth’s horizon always rose to the level of my vision, or if not because clouds or muckish atmosphere would blot out the horizon.
I let my mind wonder back to my military days in the Navy. To our trips to Australia, Tasmania, then up through the Suez and Mediterranean, then South around the cape of South Africa and back across the Indian and Pacific Oceans to California, and always via Hawaii for that great R&R there. I recalled the vastly flat seas, and how I used the ships large binoculars called Big Eyes, to seemingly see way past the horizon to where vessels should no longer be visible. At the time, I always thought somehow the atmosphere would bend the light, always perfectly, to allow the vessels to be seen at many miles past where the vessels should be too low to be visible past the horizon.
I’m more or less rather jumping out of my brain at this point and to make matters worse, end up on more conspiracy sites, this time sites about Flat Earth. And I think it’s completely crazy. There are all sorts of sites publishing ludicrous theories about a Flat Earth spinning round on a flat plate, people flung off the side or buildings leaning sideways, and accelerating through space like a Frisbee someone has thrown only at incredible speeds and never slowing down just speeding up. I see pictures where they portray large ships going over the edge of an unbounded Flat Earth like a boat tipping over the top edge of a waterfall. No true answers on these sites, that’s for sure.
Just to be clear, in my research I found there is a Flat Earth map and well explained seasonal system. The sun and moon rotate around a flat earth and remain above it at all times. There are seasons, and day and night. Gleason’s Map is often referred to as an actual Flat Earth map. The sun and moon rotate around it, as is depicted here. There are various videos about this motion. Antarctica plays a crucial role, as it surrounds the whole of the terrain of Earth as we know it in all models. Just like on the globe earth, if you go far enough South from any more Northern point, you end up at Antarctica. On Flat Earth if you move away from the absolute center (the North Pole) in any direction, you will end up at the Antarctica border. The Antarctic border is a high ice wall, 30 meters or more, all around the flat earth’s terrain, everywhere around it. It forms the border of a flat earth.
I continue self-researching, using my HoloLens app, and started placing planes on various areas of the Earth model I was using to verify curvatures of Earth’s surfaces to known curvatures of various constructions. During this phase of the research I came across well-known engineering phenomenon, such as how bridges over 100 miles long and lakes over 1000 miles long have absolutely, and I mean ABSOLUTELY, zero curvature. But on the spherical Earth model depiction, these lengthy constructions are curved and have to be else there would be very strange curvature ‘makeup’ areas to overcome the flatness of the previous flat areas, as one moves along the structure and past it in one direction, and still result in a sphere. After a lengthy run of flatness, like in Kansas, there has to be some makeup of some sort, not more flatness, else the sphere can’t be formed or even visualized.
To add to this, I read that train track layers (the builders and engineers and metal makers of the actual rails) NEVER take into account the slightest curvature for the Earth when building railroads or tunnels, same for canal constructors. And it all works perfectly, just as if the Earth really is flat and not spheroid.
Flat Earth? No, I just can’t quite believe it, well, I simply cannot believe it at all. I’ve been too thoroughly conditioned by schooling, including Purdue University and Stanford University from which I hold graduate degrees in engineering. My well trained, or well indoctrinated, mind is fighting against itself. What I learned is not what I’m seeing.
Then somehow, and it wasn’t easy, I find a science based Flat Earth site. There are many sites that are there to obfuscate only. But on this science based site I found detailed experiments such as Michelson-Morley that required Einstein to come up with a way to explain away their experiment’s results using a new theory called Relativity. I’d always wondered what motivated Einstein in this particular endeavor and find it was the Michelson-Morley experiment. Who knew? I didn’t. Einstein is the one who first stated space is empty, or made it mainstream knowledge at least. A pure vacuum he made space to be, so that his Relativity theory would work. And he assigned light a constant speed, with speed totally unaffected by any ether or substance it might travel in. He had to assign the constant speed of light through any ether so his Relativity theory would work. Else it wouldn’t work out.
Then I found and read about the Sagnac experiment which repudiates Einstein’s theory of Relativity for light. And that experiment is well hidden from most of academia, as it’s in the unexplained arena of modern science (for modern science’s theories to be valid it just doesn’t fit in) and therefore relegated to the not valid category according to known experts, or according to believers in modern science. Yet for some reason Wikipedia chooses to include the Sagnac experiment and to correlate Quantam theory with Relativity in view of the results of that experiment, but with language that is incomprehensible. I only wonder why the authors feel so inclined to discuss it at all, and not ignore it like the rest of academia has.
The Sagnac experiment proves light speed is actually affected, and therefore the Michelson-Morley experiment also would be correct if Sagnac is correct. But the Michelson-Morley experiment isn’t about the size or shape of the Earth, rather it’s about the speed of the Earth as it rotates and moves through space. Both these experiments are simply related to measuring the actual speed of a physical body (Earth) moving through space, using light to measure the speed. And the answer they provide is the Earth is motionless, not just motionless as not moving through space, but also absolutely not rotating, just entirely motionless.
So, if the Earth isn’t rotating, what is it doing? Well, I don’t know, but if it’s not moving it certainly is not spheroid, is certainly not a perfect or oblate spheroid of any sort. Rather, it’s most likely flat, which is the conclusion I arrived at based on taking a hard science approach, which required hours of fighting my own ingrained beliefs, hours of fighting years of brain-embedded schooling and images retained from years of watching sci-fi movies of all sort showing Earth hurtling through space as a sphere.
And there is one other experiment that caught my attention, and that experiment measures the temperature of moonlight versus the temperature of moonshade. Now we all have been taught that the moon reflects sunlight. A reflected light is just a lesser amount of light than the actual source. As such, we’ve learned it’s cooler at night since the light reflected off the moon is much less intense than the light directly beamed from the sun. But, here’s the interesting part, the moonshade is warmer than moonlight. This means, if you wait for a full moon and take a digital thermometer and measure moonlight, you’ll find it is cooler than moonshade. This then leads to the conclusion the moon is not reflecting sunlight, rather it’s emitting a cold light. Very crazy at first thought. I’ll come back to this in the conclusion.
Really, until I picked up a HoloLens, I’d never even let myself imagine for a moment the image of a flat earth or how a flat earth could or would work. And yet a Flat Earth model does work, with Sun and Moon ruling over day and night. The HoloLens opened up my mind’s eye to places and sights my real eyes had seen many times before, but which my mind interfered with when always viewed through institutionalized-education filters of my training. Thus, I give praise to the HoloLens, for letting me see that which I could never before see.
Having real and significant doubts about what Nasa is doing, and the lack of real unretouched photos of any kind of Earth from space, just generated images, leads me to directly and deep into conspiracy land, and when I talk about it, people simply think I’ve gone crazy. As if leaping into 3D virtual space turned me crazy, and I’ve not been able to leap back out of that virtual make-believe world. Thus, not only do I give praise to the HoloLens, others think that it has turned me batschitt crazy. Or, as I see it, the HoloLens experience has entirely opened my mind, and confirmed my eyes really see just what they do see, rather than what I’ve been told by institutionalized training, which causes my brain processing to modify the view into something other than what I really see.
I’ll end this story in this way. Having used the Instant 3D Builder app for some weeks, I find it lets my imagination go places it never went before. I could never before immediately build a 64 star grid tetrahedron (one of the sacred geometries) and manipulate size and position and rotation all instantly, and see it all visually as if it truly existed right in front of me. Using the Instant 3D Builder app really has expanded my mind and done so without drugs involved. Just for this experience alone, I give testament to the potentials of 3D virtual space and to the HoloLens for visualizing it so well.
I also realize now that if I’d not gone through this mind expansion, I would simply not allow myself to ever have entertained the idea the Earth could be flat. But my mind has not just expanded, it’s become more relaxed around ‘taboo’ topics, around topics that used to be off limits to critical thinking due to my excellent institutional schooling. My expanded mind is more amenable to explore and question previously unthinkable things and topics.
Now able to allow myself this crazy little luxury, this luxury of allowing myself to visualize a flat earth, I wondered how application of this viewpoint could or would change my views about life and the meaning of life.
I’m not trying to push religion, just capturing here what I did do. At this point of these events, I decided to put to a test how my views might change by looking at the world through flat earth eyes, and picked up a Bible, and read Genesis Chapter 1. Chapter 1 is rather short, in my Bible it’s just a single page.
Before, whenever I read Genesis Chapter 1, it was all like some old and confused fairy tale and mostly not able to be comprehended, and strange terms are used that didn’t fit in at all with my knowledge, such as ‘firmament’ and ‘the deep’ . Thus, I would discount this text as gibberish and, so it follows, easily ignore the whole Bible.
Genesis Chapter 1, verse 16 states God created TWO great lights, not one. So if true, and the moon is a light rather than a reflector, a light that rules over the night, well, things are different than we were all taught in school.
With this newly opened mind, I find that what is described in Genesis Chapter 1 is the creation of a flat earth, not a spherical earth. In grade school, at my Mom’s insistence and partially for some babysitting, I’d attended Bible School for four or five years, each Wednesday after school. And in those lessons of the Bible, we might hear a version of creation, but no discussion of spherical versus flat, this subject was never entertained.
And for years, in fact every single year of my life from around 5 years old to about two weeks ago, I had in my head the singular and solidified and unquestionable view that the Earth was a sphere. I’d never, ever thought anything different all my life. And thus, it was rather easy to disregard the whole Bible, especially since that introductory page was so darn crazy when read from the viewpoint of a spherical Earth. Oh, but how different Genesis Chapter 1 becomes when read with a different view, when read with a Flat Earth view.
So picking up the Bible, for the first time in years, and now allowing myself this crazy little luxury of holding for just a short little while the viewpoint that the Earth is flat, I reread Genesis Chapter 1 again. Suddenly this single page, this introductory page, this very first page of the Bible’s text, reads entirely different, no longer so contradictory and confusing but now so much more interesting and perhaps entirely factual.
I challenge you to go try it. Go read that single page, Genesis Chapter 1, and see if it reads differently for you too. If that is, if you’re able to hold the crazy viewpoint that the Earth really is flat while reading it.
Oh, but what about the bomb you say, why am I suddenly loving the bomb? Researching Antarctica, one comes across the history of Admiral Byrd and his trip to Antarctica after WWII in 1948. Very strange that he went on a research mission with a battle group and not research vessels, and came back home with one ship, several men, and many planes short from when he left. Also, strange that upon his return, nuclear testing went from being on islands in the deep Pacific, or submerged deep in the earth, to high altitude nuclear testing, with even bigger and bigger nuclear weapons than ever used before.
This high altitude bombing went on for four years and stopped. Various rumors are out there about why the testing was done at all, but one rumor is this testing was actually used to ionize a large dome covering Flat Earth, and this ionization of this dome was done in order to repel something that wanted to pass through, and supposedly this something is still trying to pass through. The worrisome part is that the ionization is running out and must be done yet again, according to rumor. But what possible explanation can the governments give about more high altitude nuclear testing this time? Alien invasion or imminent asteroid collision? Yes, talk about conspiracy theories and this fits right in.
Who knows, I’m the batschitt crazy one here so why ask me. But if they, the establishment, need to save the world, well, I’m all for it! Yeah, at this point, I fully trust them. And I’m enjoying the Hillary Clinton Presidency too, just like I was taught by the Establishment during this past Summer and Fall would be the sure thing to happen.
Here’s a video – Truth Does Not Fear Investigation.
Experience effective use of this software just for a few minutes and you start understanding the power of your own mind’s imagination has long been underestimated, and underused. Unleash your imagination!
Developing software for HoloLens requires a workflow. Whether you start with modeling software or scripting or maybe writing C++, there’s some set of steps you follow from idea to implementation, where implementation in this case is viewing and using the software on the HoloLens.
As you start out there is a desire to create your own objects, not just view someone else’s creation however stunning their 3D work may be. After creating, you desire to start making objects move both with translations and rotations or other animations. Resizing and combing, and building upon previous creations are further desired changes to make, thoughts that all come in logical progression as you gain experience and confidence in the APIs. But there’s the workflow all the same.
In my case the workflow entailed Unity IDE to Visual Studio to HoloLens. It required certain versions of software libraries so everything worked. It required some time in each coding-to-testing iteration to flow the software through the build process, to go from end to end, time measured in minutes. And once running on the HoloLens minor imperfections could now in 3D space be readily seen, before hidden by 2D viewing limitations, often requiring yet another round trip to code and deploy the simple change. This issue gets compounded at times due to seemingly inconsequential issues or library incompatibility that may require lengthy troubleshooting to pinpoint. I became adept at using the various logging features available, a plus. But …
For me, this all resulted in the desire to remove that workflow as much as possible, it was just in the way and risky too. I wanted instead to work directly in the HoloLens environment, to develop directly in 3D virtual space. This would be my first minor epiphany on developing with the HoloLens: the back and forth needs to be eliminated, and can be eliminated entirely. HoloLens is a full fledged Windows 10 UWP device with a Holographic API that distinguishes it from other UWP devices. As well it’s got nice audio devices and matching APIs that work fairly well. Cortana, eh …, still needs some brushing up. But the HoloLens should support a development environment based on it’s hardware and APIs.
So this led to the question, “Why not work right on the HoloLens and eliminate much of the back and forth and as well eliminate the frequent software library conflicts?”.
Exploring that possibility led to development of a basic virtual-3D design and development environment running on the HoloLens as my first HoloLens app. This software offers 3D virtual spaces to create objects that can move and can be combined or resized in unlimited ways. With a built-in scripting engine even complex objects and systems can be created then recreated reliably and easily. This happens in a natural hands-free way requiring no awkward arm movements and no need to keep hands in the viewing area to work. The software adapts to how you use it by allowing you to tailor scripts with your vocabulary, your words. Repeating those words then runs the scripts and gives you full control of the virtual space. More 3D spaces can be added without disrupting existing ones, instantly, and combined into existing spaces maintaining their own movements and sizing.
The most visually stunning objects I’ve seen in 3D virtual space using a HoloLens are fractal structures, some of which are referred to as sacred geometries and frequently appear in historic documents and are found inscribed on or embedded in the most ancient structures found around the planet. These fractal structures can be more thoroughly visualized and explored in 3D virtual space than when using other available viewing media. And it is interesting to build fractal structures, to learn how a simple object or simple pattern so easily creates complexity when viewed at large scale. Add in rotations and other movements, instantly, and the possibilities start expanding.
At the start of developing the software, this interest in fractal structures led to the desire to easily display geometric objects such as a 64 star tetrahedron. But I struggled to write the code to create this object. Maybe there’s an algorithm somewhere giving coordinates of all the points but I didn’t look, I wanted to do it myself. Even building it out of pieces proved complex due to the various angles involved when fitting the pieces into exactly the right spot, lots of math, certainly possible but tedious.
While struggling with the math of it all, in the mindset that I’d need to provide code to create it, I just put it aside for a while and instead started playing. In effect I began experimenting in 3D virtual space to understand basic 3D commands and the moves needed to effectively, and simply, move the point of attention (a cursor more or less, aka POT, that moves around in 3D virtual space) to precisely combine and easily manipulate created objects.
During this time I discovered how simple the solution to this very complex math problem of creating a 64 star tetrahedron, rather how simple the solution when solved visually. The discovery really is that a small set of simple moves, especially simple when combined and repeated, succeeded in building a perfect 64 star tetrahedron. This happened using only a tetrahedron shape (4 star) with simple lateral moves and a single rotation, then combined and repeated in a simple pattern. The need for complex math slipped away, just not needed at all.
Understanding that solving some complex problems can become simple inside 3D virtual space is the major of the two epiphanies I’ve had so far developing with HoloLens and working directly in 3D virtual space.
This second epiphany gave further impetus to continue to develop the simple 3D virtual space design/development tool into something more reliable, easily expandable, and unencumbered by a 2D mindset (of monitors and using a mouse to rotate a 3D creation in 2D and the lengthy workflow to get to the device). And so did continue developing the tool, recently releasing it onto the Windows Store as a publicly available HoloLens application.
As an aside, through experimenting with both voice and hand interfaces, I found using voice commands to be more natural and more accurate as compared to using hands. Especially so given the limited field of view of the hand sensors and hands API performance along with the need for precision positioning required for perfect geometric reproductions, perfect even when magnified to 100X or 1000X normal scale, or reduced to 1/1000th scale, which with simple resizing commands can be done instantly in 3d virtual space. Voice on the other hand can be limited by noise, or improper pronunciation, but is usually more available and the voice command set more easily expanded than by using hand motions for the user interface.
What the software offers, if you have a HoloLens, is the immediate ability to create and manipulate objects in 3D virtual space, and easily compound your creations in unlimited ways. But most importantly, the result of the actual activity of creating in 3D virtual space is your mind releases life-long beliefs about constraints that don’t exist in 3D virtual space. It’s an experience that is mind expanding in ways you actually have to experience to understand. Read about it all you want, but experience it to truly understand, to actually open up and expand how your mind works.
Experience effective use of this software just for a few minutes and you start understanding the power of your own mind’s imagination has long been underestimated, and underused. Unleash your imagination!
This post uses the HoloToolkit components to configure a basic scene with voice control on an object. Then modifies the components to add a new voice command to rotate or stop rotation of the game object.
A video is here.
This post assumes you’re familiar with the Holotoolkit, so I’ve bypassed some of the steps to import the HTK into your Unity project. If you’re a new user, see this readme for the initial steps to use Holotoolkit.
The goal is to be able to use voice commands to control a game object.
Here is a simple tutorial video, using the FocusedObjectKeywordManager.
The toolkit provides both a SelectedObject and FocusedObject KeywordManager. It comes with pre-wired commands ‘Make Bigger’ and ‘Make Smaller’.
I add keyboard commands to allow easy testing from Unity.
Though my initial impression produced some disappointment, the past few days have been spent imagining uses where Hololens can enhance. And it creates some optimism to counter the disappointment.
While Hololens isn’t really for enhancing dynamically changing spaces (when driving or walking down a city street), it can enhance a static space in extraordinary ways.
For instance, imagine you are walking around an art museum and can now see holograms with informative displays next to existing paintings or other works of art. Maybe a hologram of Ben Franklin explaining the Constitution, or George Washington talking about an ax. A user could choose a particular style of art and get a guided tour through the museum, with a clear path to follow and prompts on where to go next, for instance if they were interested in paintings only from the 1600s. This type application can likely be mostly configured (not the Ben Franklin hologram, the path and informative displays could be) already. Also a user can easily display other similar art and quickly do side by side comparisons. Of course in a pure VR experience, there is no need for a museum at all, but Hololens is about MR rather than pure VR.
I imagined a Terminator-like experience, where the Hololens could detect, identify, track and enhance objects that came into view. This isn’t available, no tracking or object recognition yet, but there is ongoing work to bring in OpenCV software to do image recognition. Many people understand the value of this feature.
Rather the current Hololens experience is like opening a portal to a new world in your existing spaces. Very Harry Potter like, in being able to see what others can’t, but only if you put on the magical glasses called Hololens. So for gaming and self entertainment, the application is clear, create a 3D game environment around you, rather than just in front of you as in 2D gaming.
Another useful application that is being discussed on the Hololens forums is data visualization. One video of a 3D dataset, shows how it’s mapped to the Hololens format and displayed, and then the hologram is expanded and the user walks inside the hologram and views it from that perspective. I can imagine a doctor and patient both wearing Hololens(es) and viewing a 3D MRI or CAT scan of the patient.
In these three examples, (museum, open new visually stunning world, data visualization) Hololens opens new ways to communicate and learn. The holograms are good enough that this communication will feel natural and likely be very effective (visuals, audibles and perhaps some sense of feel will come soon enough). There are many other uses too, particularly for education and training, for faster and more effective comprehension. But alas, none of this comes for free, and that’s the current downside, which is the significant development efforts for the more visually stunning applications. But it’s true for most new development platforms, it just takes a while to ferret out the value from the chaff as the API evolves.
A bit disappointed.
It is a hologram viewer with an API and toolset that is for bringing holograms to the user’s environment.
The biggest disappointment is that I came with an expectation of a device and associated toolset (APIs, IDE, etc.) that would augment, or more specifically, enhance the user’s current environment when wearing it. The Hololens doesn’t quite do that, at least as in my vision, not yet and won’t anytime soon.
The Hololens, as per it’s name, is really for interacting with or just seeing holograms, rather than for enhancing your reality in simple ways as you go about life. It’s more meant to just augment some very specific moments in very specific (and unchanging) places, but not until lots and lots of development occurs first.
I’d envisioned the Hololens could detect and track moving objects that aren’t holograms. Out of the box it doesn’t come close to this. It may come, and various technologies such as OpenCV may be useful, but this won’t happen until lots of new development first occurs.
There’s a lot more to say but that is a good summary at this point.
Mostly the setup and initial testing and initial development went smoothly.
Do find it annoyingly difficult to scroll browser windows. All else seems to work nicely.
But how can it be useful immediately.?
Well, since it is wearable, then the benefit is having two hands mostly free and browsing with voice.
That means Cortana comes into play, and being a user of Amazon Echo and OK Google, I find Cortana woefully behind in most every way.
In short, Cortana sucks, and is really hard to get value from this poor technology that unfortunately is baked into Hololens (but can be turned off).
Anyway, Hololens can be useful for cooking. For instance for displaying recipes while preparing food. I’ll try this and report back.
Hololens is currently designed to be useful in static spaces which you spend a lot of time in. Wearing Hololens out and about is mostly good for nothing.
So that’s another summary statement that is mostly on the mark.
I saw a u-tube of a guy wearing Hololens while driving in his car. He wasn’t doing anything with it, just wearing it and driving. That is the extent of the value, which is to have a heavy and somewhat awkward hat on, while driving. He apparently was willing to pay $3000 for his high tech hat.
At the point of writing this I am in the 200 series of Hologram Academy, but haven’t finished it yet. It’s an easy to follow course, but the most time consuming parts of these apps if they were started from scratch is the graphics/visuals, and in the training samples graphics are completed already by Microsoft. This more or less hides the great amount of development that must occur to bring original holographic apps to life. Sure 3rd parties will likely fill the void, but they’ll need to. My issue here is the enormous time it will take to develop the most simple apps.
Having worked a number of years with Dynamics CRM, from version 3 up to 2015, I was always amazed by the development costs, when it’s touted as an easily customizable application. Yeah, once the customization is done, it still takes a lot of development (code writing) to make it useful. I think Hololens apps will follow this path.
Will finish the academy demos and come back and write more, but this sums up first impressions.