ARKit: Augmented reality ready for prime time

November 10, 2010

Author: John Wisolith


ARKIT augmented reality

Augmented reality (AR), the technology enabling virtual objects to be placed into a view of the physical environment, moved a step closer to mass market adoption in June when Apple announced the ARKit framework at their World Wide Developer Conference (WWDC). ARKit is a software development framework for Apple mobile devices that takes care of the complex low-level processing required by all augmented reality applications, allowing developers to concentrate on the features that distinguish their AR app from others. ARKit promises to make it cheaper and simpler than ever before to develop high quality AR applications for the largest number of existing mobile devices.

A brief history of AR

While augmented reality technologies have been available on smartphones since 2008, early implementations were limited to tracking the position of a predetermined two-dimensional marker, such as a QR code, and virtual objects could only be positioned around the space occupied by the marker.

More recent AR implementations are capable of tracking three-dimensional environments without predetermined markers. These newer implementations use a combination of sensors such as the camera(s), motion sensors, and, increasingly, a depth sensor, to track the movement of contrasting features within a scene, and derive their position in space. As the user moves their device through the environment, the AR software gradually builds a virtual model of the scene, known as a point cloud, containing the 3D position of each tracked point in space.

Once the point cloud reaches the necessary level of detail, the software can begin making assumptions about the user’s surroundings. If hundreds of points are aligned horizontally in relation to gravity, the software can safely assume that the points belong to a continuous surface, such as a table or floor. Virtual objects or scenes may then be placed on the surface and will remain fixed to it, regardless of camera movement. To view a virtual object from an alternate angle the user rotates their device around it as if they were looking through a magnifying glass. The camera can be panned away from the tracked surface and, when it returns into view, the virtual object will reappear in the same position it was left. Perhaps the greatest benefit of augmented reality is that it makes exploring virtual scenes as simple and intuitive as exploring physical ones.

ARKIT: A new era for AR

Commercial software development kits with these capabilities have until now been an expensive ongoing investment, with thousands of dollars in annual license fees. ARKit breaks new ground by making augmented reality available to developers at no extra cost. Hundreds of millions of existing iPhones and iPads containing an A9 chip or greater are supported, including the iPhone 6S. According to Apple, this makes ARKit “the largest AR platform in the world”. By comparison, Google’s AR technology for Android, Tango, is currently only available on two consumer devices, representing just a small proportion of smartphone market share.

Developers experimenting with the prerelease version of ARKit have already produced some exciting and creative demonstrations of the platform’s potential. One example showcases a virtual ruler with millimeter accuracy, while another measures the floorplan of a room. At a restaurant, a diner peruses the menu in 3D. Tiny basketball players compete on top of a developer’s desk. A life size virtual car is driven around a car park. Inter-dimensional portals are opened in the middle of a street, allowing the viewer to step through a doorway to explore a completely virtual world.

Apple demonstrated new Lego, Pokémon Go, and Wingnut AR games at its WWDC Keynote, and while games are the most obvious use case for AR, they are only one of many possible applications for the technology. In the retail space, IKEA is working on an AR catalogue which will enable customers to preview life-size homewares or furniture amongst the existing decor in their own home. Other applications in the category may allow amateur home decorators to create a scale model of a room in their home, and play with their design like a doll house. On construction sites, architects and builders will collaborate inside virtual models of their buildings. Medical students will be able to look inside life-size models of the human body and simulate injuries or diseases at the touch of a button. The potential applications for AR are limited only by our imaginations.

As an evolving technology still in its early days, ARKit currently has a number of technical constraints that are likely to be overcome as development on the platform continues. Due to its reliance on tracking still objects, AR is not suited to busy, high traffic settings with continuous background movement. The tracking algorithms work best in high contrast, textured environments, and will not be stable on featureless white walls or tables. For now, objects in the physical environment cannot obscure virtual ones; when a person walks in front of a virtual chair, the chair will appear to float in the foreground. However, devices with two rear cameras are already capable of calculating the depth of physical objects, and it is anticipated that in the near future ARKit will be able to use that depth information to obscure virtual objects.

The already impressive capabilities of ARKit will continue to grow as new features are added, and applications built with the technology today will benefit from subsequent improvements to the platform. Industry insiders have revealed that the upcoming iPhone 8 will include a new depth sensor, which will further enhance the quality of AR experiences and make them even more immersive.

The recent announcement that Google’s parent company, Alphabet, is relaunching its Glass headset for enterprise provides further confirmation that augmented reality is ready for real world applications. Technology pundits are speculating that Apple’s release of ARKit, combined with a series of patent applications, indicate that the company is working on an AR headset of its own. A decade after the debut of the iPhone, comments by Apple CEO Tim Cook that AR is “a big idea like the smartphone” hint at a not too distant future where smartphones are supplemented or replaced entirely by head mounted AR devices.

Augmented reality is a revolutionary technology with the potential to transform entire industries, just as the internet and smartphones have in recent decades. Businesses will use AR to improve efficiency by enhancing or reinventing workflows, and customers will benefit from more engaging and immersive experiences. As with previous technological shifts, incumbents will face competition from new challengers embracing the potential of AR. Apple’s ARKit provides businesses with an opportunity to begin exploring augmented reality today on devices that their customers already own, safe in the knowledge that they are developing for compatibility with AR devices of the future.

John Wisolith is an iOS Developer at Arq Group