Augmented & Virtual Reality (AR/VR) in mobile apps

A new platform comes every decade, the smartphone simple client golden era has passed and now makes way for other mediums where the user can be more engaged and the experiences more immersive. I’m talking of Augmented and Virtual Reality technologies, of course, as all the big technology companies (Apple, Google, Facebook, Amazon, Microsoft) have made serious commitments to developing new technologies in this domain.

Mobile applications are still our most personal and most used device, today we have over 2 billion mobile devices that support AR experiences and the number is increasing, as every new smartphone has AR/VR capabilities. The hardware is advancing constantly (eq. Apple’s new LiDAR sensor) as well as the software (new development kits), making it easier to integrate AR experiences in the apps.

AR is being used in a lot of domains already, for instance in the E-Commerce sector where shoppers are more inclined to purchase a product after they’ve inspected it with AR technologies or social apps where besides filters, the users can also add AR objects in their streams. Another domain in which AR is popular is the indoor or outdoor navigation solutions where directions are easier to be followed if they are placed in a real-world environment.

Big corporations also use AR/VR either for education or to showcase different tech features of their product, for instance displaying different motor parts or how a certain hardware component works. AR is also used in medicine, for instance, the patients who need to do recovery exercises can speed out their recovery time by also practicing at home with the aid of an app.

Due to the current state of the world, the education system is changed, new technologies are used for conveying knowledge and AR does a really good job in this domain as well, as it creates more engaging experiences without necessarily being in a classroom.

The big technology companies are all working on smart glasses to augment our real experiences but in the meantime, our way to the future is still through the smartphone.

So what is the current status quo in terms of what’s possible from an AR/VR point of view on mobile devices?

Virtual Reality

In terms of virtual reality, there are a lot of developments from both Google and Facebook where they created special goggles or devices to be attached to mobile phones for experiencing virtual worlds. Complex scenarios still require a lot of processing power, and they are usually used in games and those devices are most popular with kids.

In VR worlds, the users usually have additional devices (joysticks, etc.) for interacting with virtual objects. Since we are talking about mobile applications, VR is not that popular and used due to the nature of the device (doesn’t have a lot of power, and additional controllers).

Augmented Reality

Here, things are a little more evolved, and while companies are interested in VR technologies, a lot more budget and development go into mobile AR at the moment.

In essence, augmented reality is adding digital content to real-world anchors and being able to analyze the environment of the user.

From an AR perspective, we currently have the capabilities to do the following:

• placing a virtual object in the real world (it will remain at the same location)
• scaling, rotating, moving, and animating virtual objects
• interacting with virtual objects
• measuring real objects through virtual anchors
• sound effects based on the proximity of the source

AR is closely related to AI, it’s used a lot in combination with AI techniques to modify the user’s environment and interact with him. An example would be the recovery app I’ve mentioned before where AI would be used to detect the human body pose and AR would enhance the experience by showing different metrics or what needs to be improved to correctly do the exercise. To learn more about AI/ML, read our previous article.

From an AR perspective, we also have new sensors (most specifically the LiDAR), which can scan the environment with a higher accuracy making the object placing faster, easier, and more accurate.

Smartphones open the doors and the path to a new platform that is yet to emerge and are here to stay. There are a lot of possibilities in this domain and technological advances constantly open new possibilities. Just like in the years that followed the release of iPhone the market of mobile apps has exploded, this is what will happen with the AR and VR apps as well.

Do you have a mobile project which could benefit from AR or VR, or have general questions regarding those technologies on mobile devices? Drop us an email at cluj@appssemble.com, and let’s discuss more!