Making the Invisible Visible with AR
Augmented Reality, or AR for short, will change how we experience, measure, interpret and understand our world and also how we interact with it.
AR will certainly challenge us – sometimes less, sometimes more significantly – but it also will bring with it a great value potential and an essential set of tools for our societies to master progress in various areas – may it be in an increasingly digitally driven industry, about infrastructure, mobility and logistics, urban planning, design, and governance, environmental and regenerative sustainability, establishing closed loops for a circular economy, about knowledge transfer, education, and learning, communication in general, tourism, socialising, as well as societal participation, and probably a lot more. And not to forget the right now seriously globally tested healthcare sector.
LIDAR Sensors Used for Tablet Devices as a Test Case for a Next Generation of Augmented Reality Solutions
On 20 March, 2020, WIRED Magazine speculated about the latest iPad Pro’s LIDAR sensor mainly being a test case for a wholly new category of products to be offered by Apple within the next years – Augmented Reality (AR) glasses.
WIRED author Brian Barrett, like authors of other publications, grounded his view in recent releases of iOS and Xcode or one of Apple’s more recent patents. And while not all technological developments, patented or not, necessarily have to lead to actual products available to consumers so soon or at all, AR as much as Mixed Reality (MR) or Extended Reality (XR) devices in general make much sense and will most probably see their breakthrough rather sooner than later. They will offer never before seen possibilities of perception of and exchange with one’s surroundings and also massively benefit an increasingly digitally interconnected, interactive world – at least for those who will be willing to make use of it and understand how.
Sceptics might point out how Google Glass did not really come to life, other approaches like Microsoft’s Mixed Reality device HoloLens only relatively slowly moving forward, and the once hyped company Magic Leap not being able to meet or even exceed way too high expectations.
Nevertheless, AR is already in use, and e.g. in manufacturing, companies experiment with first applications, eagerly waiting for the next technological upgrades and new possibilities enabled based on them.
Developments in Virtual Reality Being an Indicator?
On the other hand, after having been material of a whole decades old subgenre of science-fiction, namely Cyberpunk, Virtual Reality (VR) witnessed one revolutionary announcement after another, just to be followed by fascinating and yet still rather modest achievements.
However, those developments are definitely not to be underestimated, and most importantly, while indeed slowly, do in fact move forward unstoppably and will have built sufficient momentum eventually. Digital artist Goro Fujita’s work made with Oculus Rift and a software called Quill gives a very good idea of what great ways of what kind of visual and spatial experiences to expect in the cyberspace. Have a look at some of his animated 3D drawings that were made by drawing in virtual air and give an idea about what VR (and with it AR and XR) might have to offer to us in the not so far future.
Some Examples of Possible Augmented Reality Use Cases
The scenarios following were only some of the more obvious ones that AR will allow users to create, and some, if not all of them, are also already in the making somewhere. One might read about them in tech articles, hear people (the author of this article included) daydream about them, or experience prototypes at fairs, conferences, tech summits, etc.
An Urban Infrastructure Information Layer
Picture yourself walking in the streets of some megacity like e.g. Shanghai, possibly in Pudong district, not sure where and when to catch the next metro train but being under immense time pressure due to an important appointment to take in thirty minutes. You activate your AR glasses and suddenly see all needed information right in front of your eyes, as an overlay that extends the real, physical world by what otherwise would be invisible to human sight.
You see in which direction the next metro station is to be found, where the entrance. But you also see the different trains actually approaching and leaving the station, as if you were looking right through the ground under your feet; additionally the next arrivals and departures on schedule, counting down, how long it will take you to arrive at the right platform, which car will be closest to the exit to take at the arrival station to get to the place of your appointment fastest. Possibly, you will be able to directly book the proper ticket, too, by simply touching your spectacle frame swiftly two times when you selected the correct connection, thanks to eye tracking.
Of course such information layers would also work for all other parts of the urban infrastructure on your way around the city and tell about the nearest local government offices and their service times, stores and their products in stock, restaurants and their culinary offerings, hotels and vacant rooms, taxi or bus stations and fairs or ticket options, and much more.
An Augmented Operations Guide and Manual in Manufacturing
You are new to your job in a factory of an automotive company as well as to their process setups. Maybe you did not yet work with those specific robot models in use at the production line. Of course, you will also get the needed theoretical and practical training. But with your wearable AR device you get much more than that, as you will be guided by visual overlays, manuals that blend in next to the machine you want to operate. Step by step it shows you what to do and what to be careful about, also marking the areas around machines that you have to avoid while they are in service.
The Augmented Operations Guide also demonstrates how to maintain the robots, change tools and program them via the smart pad right in front of you. And once your shift ends, you dictate your report right into the documentation system.
A Museums Guide via Augmented Reality
When you go to a museum (or alternatively an art gallery) the AR device tells you all about what you see in the exhibition, provides you with everything available to know about a Chinese statue, Roman tomb stone, French painting, or any other cultural goods in front of you. You will learn about exciting background stories, vividly told, about scientific methods and the people who brought light into the dark around the exhibits over the course of history.
LIDAR technology – in combination with Artificial Intelligence and pattern recognition, as well as state-of-the-art 3D animation techniques – might be good for some pretty spectacular visual effects, as based on it a distinction can be made between background, foreground and what lies in between, for getting dimensions and perspective and inserting all kinds of visual input correctly. Possibly, scattered earthenware pieces on the ground will reconnect and suddenly a very alive looking Terracotta Army soldier marches around the corner right towards you, stopping closely in front of you, saluting and reporting about his and his comrades’ creation and purpose.
Never before should visits to any museum, art gallery or similar place have been so amazing and interesting.
An AR Guide for Tourists, Discoverers, and Adventurers
Last time you travelled and visited some place you might have come by some interesting looking buildings and asked yourself about when they were built, by whom, what influenced the architects in terms of style and construction methods and whom they influenced in return afterwards, what exactly the challenges were to make the concepts become reality, and so forth. Unfortunately, you had no tour guide by your side and those buildings were all out of sight too quick anyways, as the bus you were sitting in did not stop. Or you were on a hiking tour in the mountains and wanted to know the names of all those beautiful peaks nearby, their heights, what geological event formed them, what species populate them, etc.
Augmented Reality already helps with that today to some degree but it will do a much better job in the future, even more so when the device to deliver the information you want sits right on your nose and automatically unveils endless knowledge to answer every question you might have.
A Guide to Nature
Because of Augmented Reality and Artificial Intelligence you will never again ask yourself what that beautiful and also a bit rude bird in the park is called – this little Noisy Miner, or Manorina melanocephala for ornithologists.
But this specific exemplar does not want to reveal its typical sound to you when you are nearby no matter how much you would like to hear it? Your AR device coupled with your headphones can play it out for you. Lastly, you might also quickly want to take a picture, so just focus on the bird and blink twice. Again, thanks to eye tracking the job will be done right away, just a matter of a second before the bird flies away and into the next bushes.
More Will be Needed Than Sensors and Smart Devices to Make AR that Good
There will be a lot more use cases than described above but no matter how good any AR device might be and how sophisticated the sensors built in, they will need a complex system underneath, a kind of framework, to wire various complementary pieces of technology on the one hand and on the other unifying a lot of different information sources, databases, applications, and digital services. It will be key to generate a seamless, compelling and clear digital user experience and customer journey. Depending on the scenario, you might be confronted with including and welding together data points from ERP (Enterprise Resource Planning), CRM (Customer Relationship Management), BIM (Building Information Modeling), educational and learning solutions, content, document and media asset management systems, graphics and rendering engines, of course some Artificial Intelligence, and much more.
And of course, you also need to overcome still closed silos, make people of different but complementary backgrounds, disciplines, functions, departments, and institutions collaborate. Also this will be impossible without enabling processes and tools – like employee, customer, or partner portals, which at some point might then incorporate the very benefits of Augmented or Extended Reality described above.
Thanks for providing information about BIM. You have explained in detail in this article. Thanks again for this article.