.jpg)
Let’s get this out of the way: Microsoft HoloLens is not exactly a holographic system, but don’t hold that against it. The reality of Microsoft’s surprise augmented reality system is that it is something special, extraordinary even.
It may be the first virtual-reality device to truly blend the virtual and the real in one seamless, head-mounted experience.
Following Microsoft’s ballsy “one more thing” presentation on Wednesday, the company gave me an impressive series of hands-on and up-close experiences with HoloLens.
Microsoft banned photos and videos from these demos, so I had to take copious hand-written (!!) notes to keep track. As a result, I have no images or Vines beyond what Microsoft provided in media kits and what I could gather during the original public presentation. Even without images, the allure of this technology is undeniable.
Headgear
Remember that slate gray, polished-looking headset you saw during the Microsoft livestream? Forget it. To get the full, or at least best possible, experience with HoloLens right now, I had to use something that resembled a skinned HoloLens. Instead of a polished gray exterior, the dev unit I wore had its guts exposed and was tethered to an external power source. The new holographic processing unit hung in a separate pack from a strap around my neck.
Before I could don this somewhat frightening-looking and still-quite delicate gear, I was given a brief training session by Microsoft. HoloLens is controlled with a combination of gaze, gesture and voice — and your head is the cursor. Basically, you look through a transparent stereo display where you want to interact or at what you want to control.
Many actions are performed with what Microsoft calls an “air tap,” which is done by making a fist in front of your body, where the HoloLens can see it, and then raising and dropping your index finger in one quick motion, as if you were tapping someone on the shoulder.
Sitting there, listening to these instructions, I felt my enthusiasm draining away. The product looked unfinished and the gesture seemed silly and hard to master. That feeling would not last.
Mission to Mars
IMAGE: NASA
Minutes later I was standing on the surface of Mars. The car-sized Curiosity Rover was just feet away from me, real enough to touch. All around me were spectacular Martian rock formations. I squatted down to take a closer look at a collection near my feet. All that was missing was the ability to pick up one of these stones and toss is over the ever-watchful Curiosity’s head.
Earlier, a pair of Microsoft representatives had gingerly placed the HoloLens dev gear on my head, reminding me not to touch any of the exposed technology on the front of it. Before my eyes a thin blue line outlined a rectangle. In its center was a Windows logo. The technician worked with me to position the headgear until I could see the whole rectangle — which would allow for an optimal experience.
Before this, they had measured the distance between my pupils with a special device straight out of an optometrist's office (future HoloLens will do this measurement on their own). They used this measurement to preconfigure the HoloLens and then asked me to push my glasses up against the bridge of my nose so they wouldn’t be in the way.
The headgear made a clicking sound that I initially mistook for my skull crackingThe headgear made a clicking sound that I initially mistook for my skull cracking — it was very tight. But I could see the whole square and was ready for my Mars experience.
The pressure on my head soon gave way to sheer wonder. Microsoft worked closely with NASA’s Jet Propulsion Laboratory (JPL) to develop the OnSite Curiosity experience software. The goal, they told me, is to revolutionize how JPL scientists drive this mission and how they walk about and explore the Martian surface.
IMAGE: MICROSOFT
The HoloLens software uses all the actual imagery Curiosity has collected and then renders it in three dimensions. To help illustrate the impact this could have on scientific exploration of these images and the data collected by Curiosity, the Microsoft technician had me walk over to a computer. I was still wearing HoloLens, but when I looked at the computer through the visor, which meant I was looking through the semi-transparent stereo displays, I could see the computer screen, keyboard and mouse perfectly. HoloLens was smart enough to cut out the Mars augmented reality around the computer.
On the computer screen, I saw black and white Mars landscape imagery. It was poorly stitched together and offered no context. When I used the mouse to select a rock on screen, its counterpart in the 3D-rendered landscape was also selected and a little flag appeared above it. The tech instructed me to use the mouse to drag my cursor off the physical screen and select something directly in the 3D environment.
It sounded like a silly idea: how could the mouse go off screen?It sounded like a silly idea: how could the mouse go off screen? Augmented reality makes it possible. I did as I was told and moved the mouse from one digital environment to another somewhat more unchartered territory and then I selected a rock. It was amazing.
Microsoft and JPL envision a world in which multiple scientists wear HoloLens headsets and collaborate in augmented reality. During my demo, a JPL scientist joined me in my Mars excursion and, to be honest, his avatar, which appeared just up the virtual hill from me, looked like an androgynous, gold Martian.
I could hear him through my headset (the headphones sit in the device and not on my ears) and watched as he pointed at rocks with a dotted line that basically shot out of one virtual finger. He showed me how to tell Curiosity which rocks I wanted the robot to zap with his high-powered laser. Throughout, I used the surprisingly easy-to-learn air tap gesture to select rocks for destruction or future investigation and to define areas for higher-resolution imagery.
It was one of the best demos I’ve ever experienced.
I could’ve happily stayed on Mars for the rest of the afternoon, but Microsoft insisted I move on to some of their other HoloLens experiences. I still wonder if Curiosity misses me.
Work together
IMAGE: MICROSOFT
HoloLens is a Windows 10 device and, according to Microsoft, should integrate smoothly with some of their other Windows 10 apps and services, including Skype.
For my second and almost equally impressive demo, I made a Skype call with a Microsoft representative who used HoloLens to help me replace a wall switch. After they put the headset back on my head and tightened it just enough to make my eyes bulge, I made a Skype call through HoloLens. A 3D-enhanced version of the app appeared in front of my face, I looked at the profile picture for a woman named Alice in the contact list, used an air tap gesture and initiated the call. Alice appeared in front of me in a floating call box and offered to help me.
While her call screen followed me wherever I looked, I was able to pin it to a virtual spot just above my gaze; I could look up to see her (I could always hear her). On her side, Alice could see everything I looked at and was able to guide me by drawing in the virtual space in front of me. She could draw illustrations, arrows or circle things she wanted me to pick up.
Even though I knew what I was doing, I found her guidance quite helpful. We did successfully install the new switch.
Game time
Microsoft spent $2.5 billion on Minecraft and with HoloLens may have found an implementation worthy of the expense.
The augmented-reality headgear is full of sensors, but the most powerful one may be the 3D depth sensor. It’s the same one you’ll find in the Kinect and it is capable of building a detailed 3D mesh map of a room and everything in it. Once HoloLens knows what’s in the room, it can essentially drape 3D imagery over it so that it looks as if the digital objects and textures are part of the same environment as real world walls and furniture.
In Minecraft, the result is pretty mind-blowingIn Minecraft, the result is pretty mind-blowing. Wearing the HoloLens headset, I saw a room full of Minecraft buildings, landscapes, and zombies. On one real bench sat a Minecraft digital wooden platform. I called for a tool using a voice command and then used air tap to destroy the wood. Soon I was looking through the bench to a molten lava pit below. I was also able to call for a torch and use it to blow up a bunch of zombies.
Throughout all of my demos, I could walk around these rooms without bumping into something or growing disoriented, mainly because I could always see the real and virtual environments at the same time. Instead of detracting from the experience, it finally made virtual reality practical.
3D builder
I didn’t get to use HoloStudio myself. Instead Microsoft had us watch someone else wear HoloLens and build a 3D object in real time. To see what he saw, we watched a playback screen that showed the Microsoft rep in the blended environment.
We watched as he built a couple of 3D characters using gestures, air tap and his voice. Everything he built could be placed in the augmented reality environment and printed out on 3D printers. I took note of the cool-looking, 3D-printed Star Wars X-Wing, which he had reportedly built in HoloStudio in an hour and a half.
While I wished they would have let me do some VR 3D building, it was a minor disappointment.
Different and better
Rarely have I tried something that looked so bad and worked so well. HoloLens belied its development state, delivering a truly blended virtual and reality experience. Unlike Google Glass, which also uses augmented reality, the initial use cases for HoloLens are indoors.
Google encouraged Google Glass Explorers to wear the head-mounted device everywhere. When HoloLens arrives later this year, don’t expect to see it out in the wild. It’ll live in companies, industry, living rooms and game rooms. That may help HoloLens avoid Glass’s fate. People who can afford what most expect to be a nearly $1,000 price tag won’t be labeled snobs because no one will see them walking through the airport wearing one.
It may be a little early to pass judgment, but my dev-stage experience has convinced me of one thing: there has never been anything quite like HoloLens and that is most definitely a good thing.
Sign up here with your email
ConversionConversion EmoticonEmoticon