Augmented reality - IBM

Augmented reality: More than meets the eye November 2003 Executive summary – Augmented reality overlays computer-mediated information on the real world. This ability enriches environments for action and learning and offers the potential for new kinds of shared experiences. Already used by the military to support action in the field, this technology is beginning to work its way into daily life in the form of everything from aircraft maintenance to providing drivers with better night vision. The front porch, with its spooled rockers and friendly flower boxes, is the last place you’d expect to find a murder victim. Which is why is it perfect for the film “New England Death.” It’s the sixth “perfect” spot you’ve seen today, but the others all presented problems – noisy aircraft, landmark issues, roads too narrow for the equipment trucks, trees that blocked key reverse angle shots. Serving both action and production concerns is the curse of the location scout. You stare intently at the front steps and trigger icons on your augmented reality (AR) glasses that illustrate how it will be transformed by shadows and light as the sun arcs through the day. Next, you stand on the top step and slowly turn 360 degrees. Distance measures are captured in realtime, with no red lights – plenty of room for all shots and setups. “Services?” you ask, and the distances and costs for crew housing, meals and even restroom facilities feed in. No show-stoppers are among them, despite the production’s tiny budget. The only hurdle left is making everything within a shot look like a part of the 1930s. As you look again at each house, each yard, each store, the elements for digital or physical revision – removal of satellite dishes, adding props, repainting –are noted and priced. The news is good. How this happens The old adage, “Knowledge is power,” takes on a new meaning with the coming of AR. Suddenly, we have a cheat sheet for living, with facts for better decisions, step-by-step guidance and even experience-enriching commentary right before our eyes (or ears in those cases where directional tones and narrations are used). In some ways, AR is already with us. As drivers, we routinely accept and use realtime information on our speed and the condition of our vehicles. And while we may not yet have access to the wealth of information found in the heads-up displays of fighter aircraft, we do have Global Positioning System (GPS) dependent maps for navigation and superimposed images detected by infrared sensors, now part of some production vehicles. Features in some high-end cars actually illustrate most of the characteristics of AR – mobility, awareness, context and personalization.

ibm.com/bcs

Executive technology report

1

Mobility. AR goes where you go, ideally in an untethered, unintrusive manner. For instance, some prototypes in development include wireless access to data coupled to a display that overlays the information of the realtime, real world view. A car or a jet fighter can have an onboard computer with storage, but this gets more complicated when AR moves to the individual, who needs to either carry a lightweight data source or have a means of dealing with data transmission delays. Awareness. How is the information presented? As the user moves about and looks in different direction, information changes depending on where he or she is looking. For accomodating AR in a vehicle, this just entails covering the windshield with schematics, icons and numbers. Directionality of information is based on direction of travel. For an individual to experience AR, head-tracking mechanisms are needed as a means of determining where the user is looking. Context. Where is the person and what are the conditions? In a car, the navigation tool uses data from the GPS to establish where the person is, and this approach has also been used for personal units (although additional equipment might be needed for sufficient precision). As we drive along, humans are already in the loop to identify the need for fuel and the directions to the nearest gas station, but all the pieces are in place to also automate such suggestions. The potential exists for adding physiological (provide a blast of cold air when the driver is getting sleepy) and affective (offer directions when the driver is confused) information. Personalization. How would you like it? Telematic systems for cars include subscriber profiling to facilitate concierge services (for example, to recommend restaurants on a trip). However, this might be taken in other directions, say adjusting both the content and presentation style of a heads-up display to meet an individual user’s needs and preferences. The data delivered to a student visiting a museum would be different from that presented to a security guard. Because of the many kinds of information available via AR – including commentary, advice, time, physical conditions, distance, facts, x-ray, map, diagram, shared views – people have let their imaginations run wild on applications. To give a sense of the range: • Tourism – Provide just-in-time narration, wire frame images of buildings as they existed in 1620, currency conversion, translation of signs and directions to the nearest restroom • Diminished reality – Replace unsightly construction with images of nature; overlay advertising with thoughtful quotes from Henry David Thoreau • Repairs – Highlight broken components and provide step-by-step instructions on how to make the fix

ibm.com/bcs


2

• Accessibility – Use narration and audio tones to help a blind person get the most out of a historical site • Education – Prompt students as they go through sales training scenarios; provide definitions during a lecture; overlay the model of a correctly held fist during karate training • Games – Take a fantasy game into the real world by overlaying fictional characteristics on real world objects, locations and people. Of course, there are limits to what can be done in AR today. For one thing, equipment today for mobile individuals is heavy and awkward. Working outside is limited by GPS availability and the precision of location determination. Even with the best head-tracking equipment, directionality is subject to lags and registration problems. See-through displays are not generally available. Equipment costs are too high for many applications. Still, progress is being made on all fronts, and there are clever solutions that jump past current difficulties. Clearly, AR will create new requirements for data transmission, analysis of contextual information and realtime management of content. AR ranks as a significant technology in the area of realtime, with important pervasive computing elements. While speculation and prototyping represent a wide variety of applications, the most complete products exist in the defense industry. However, in cars, electronic games and television broadcasts, pieces of AR are becoming familiar. In fact, expectations, driven by movies and games, have outrun what can currently be done in AR. Limits to bandwidth, the lack of appropriate displays, registration problems, availability of content, cost and lack of experience with users are just some of the reasons why the data-rich overlays seen in the Terminator movies will not be installed in a pair of sunglasses anytime soon. Breakthrough applications are more likely to provide a means for a backhoe operator to see fiberoptic cables (and avoid digging into them) than for a tourist to see Florence through the eyes of Leonardo da Vinci. As the first products acquire the label of AR, there will be a risk of the same sort of damage to the field, as was seen when virtual reality was hyped into oblivion. At the same time, the value propositions for AR are already common currency, reducing the amount of selling and explanation necessary to gain acceptance. In a sense, there is already a place in our culture for AR. In fact, under different labels, business and research institutions are investing in AR with tangible benefits in mind.

ibm.com/bcs


3

Two industries: Manufacturing and transportation The manufacturing industry is likely to lead the adoption of AR because of a combination of opportunity and practical concerns. The opportunities are in training assembly and repairs. For each of these, the ability to overlay schematics and provide step-by-step instructions can save time and improve efficiency. Who wants to refer to a repair manual when working underneath a machine with hands that are full of grease? The return on investment is relatively easy to determine and the setup is simplified because conditions are controlled and repeatable. Aligning the real world with the world of data is less of a problem if you can install markers (visual or electronic) over the area, which is easier to do in a fixed environment like a factory floor than across a cityscape. The retail industry, which is already on the way to turning stores into smart environments through radio frequency identification (RFID) and profiling, offers a variety of interesting possible applications. For instance, a glance around might tell a manager that stock of an item has been depleted or which zones of the store are dead, with little inventory movement. A sales person might be able to identify big spenders, their needs and, depending on privacy restrictions, biographical data that could help them provide a friendly greeting. Of course, if shoppers are eventually given access, through either a projected environment or glasses, then alerts, directions, suggestions, details about products and perhaps even commercials can be presented to them as they move through the store. Technology to watch Displays GPS Grid Computing Pervasive computing Sensors Virtual reality Wireless

Sites of interest Overviews Augmented Reality Page. http://www.se.rit.edu/~jrv/research/ar/ How Augmented Reality Will Work. Overview, with focus on key technologies http://www.howstuffworks.com/augmented-reality.htm Augmented Reality (Popular Science). An introduction to applications http://www.popsci.com/popsci/computers/article/0,12543,190327,00.html

ibm.com/bcs


4

Applications Battlefield Augmented Reality System (BARS). What the US Navy is up to http://www.ait.nrl.navy.mil/vrlab/projects/BARS/BARS.html Tourists may look to MARS for guidance. Focus on Columbia's prototype http://www.cnn.com/1999/TRAVEL/NEWS/12/31/future.of.travel/index.html ETC Lab. University of Toronto's program that includes AR http://vered.rose.utoronto.ca/ Infrared eye aids after-dark drivers. Car application http://www.cnn.com/TECH/science/9808/28/t_t/night.vision.car/ Shared Reality. Firm devoted to AR http://www.sharedreality.de/index.php?part=main&main=opener&sub=page&lang=EN Remote Sonar Sensor™. Application for fishermen http://www.humminbird.com/hb_Products.asp?ID=322 Wearable Computing Wearable Computing. AR work at MIT http://www.media.mit.edu/wearables/ Excuse me, is that a monitor on your head? Wearable computing, including AR display http://www.cnn.com/2000/TECH/computing/03/31/head.monitor.idg/index.html Steve Mann. The inventor of wearable computing http://www.eecg.toronto.edu/~mann/ MIT “cyborgs” bridge gap between man and machine http://www.cnn.com/TECH/science/9807/23/t_t/digital.gadgets/ Thad Starner's Home Page http://www.cc.gatech.edu/fac/Thad.Starner/ About this publication Executive Technology Report is a monthly publication intended as a heads-up on emerging technologies and business ideas. All the technological initiatives covered in Executive Technology Report have been extensively analyzed using a proprietary IBM methodology. This involves not only rating the technologies based on their functions and maturity, but also doing quantitative analysis of the social, user and business factors that are just as important to its ultimate adoption. From these data, the timing and importance of emerging technologies are determined. Barriers to adoption and

ibm.com/bcs


5

hidden value are often revealed, and what is learned is viewed within the context of five technical themes that are driving change: Knowledge Management: Capturing a company's collective expertise wherever it resides – databases, on paper, in people's minds -- and distributing it to where it can yield big payoffs Pervasive Computing: Combining communications technologies and an array of computing devices (including PDAs, laptops, pagers and servers) to allow users continual access to the data, communications and information services Realtime: "A sense of ultracompressed time and foreshortened horizons, [a result of technology] compressing to zero the time it takes to get and use information, to learn, to make decisions, to initiate action, to deploy resources, to innovate" (Regis McKenna, Real Time, Harvard Business School Publishing, 1997.) Ease-of-Use: Using user-centric design to make the experience with IT intuitive, less painful and possibly fun Deep Computing: Using unprecedented processing power, advanced software and sop-histicated algorithms to solve problems and derive knowledge from vast amounts of data This analysis is used to form the explanations, projections and discussions in each Executive Technology Report issue so that you not only find out what technologies are emerging, but how and why they'll make a difference to your business. If you would like to explore how IBM can help you take advantage of these new concepts and ideas, please contact us at [email protected]. To browse through other resources for business executives, please visit ibm.com/services/strategy Executive Technology Report is written by Peter Andrews, Consulting Faculty, IBM Advanced Business Institute, and is published as a service of IBM Corporation. Visit ibm.com/abi Copyright ©1999-2003 IBM Corporation. All rights reserved. IBM and the IBM logo are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Remote Sonar Sensor™ and Smartcast™ are trademarks of Techsonic Industries, Inc. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products and services do not imply that IBM intends to make them available in all countries in which IBM operates.

G510-3550-00

ibm.com/bcs


6