VR Technologies for Mobile Apps

The very first question was "how to create and show 3D and/or VR world in a native app?". If you are a game developer, then it may sound silly. We are not, but we had a knowledge about a few solutions, so we took a deeper look at them.

Unity

Really cool, if we’d develop a game, this will be the first choice. It’s scalable, it’s cross platform and you can write your code in a high level programming language, like C#. It's 3D engine, but it supports VR very well and out of the box. You can generate a project for Android, iOS and many other platforms.

However this is not the best solution for our internal project, especially because Unity is not free for all as its price depends on entire company income, so even if our application generates no revenue, you still need to pay. On the other hand if you're a single developer or a small game making company, then you may use free Personal edition (up to $100k gross revenue per year).

The second reason against is due a fact that Unity is a game engine, and we’re not interested in making games in Netguru at least at the moment. That means new skills we may gain during the development won't be helpful in the next projects.

Native VR

We were looking for a library that generates a 3D with stereoscopic view that is used by platforms such as Google Cardboard. We‘ve looked at most popular virtual reality methods and frameworks. Below is what we’ve researched.

OpenGL seems to be the only native solution for Android as Vulkan is not supported by GVRKit. You can read about the last one in the next paragraph.

It’s pretty easy to acquire similar effect by using native iOS SceneKit and ARKit. A big plus is that it’s easy to switch between stereoscopic and standard view. That means the app can be used without VR googles, especially if we add a software joystick or physically walk around the scene.

First, we thought that using ARKit along with SceneKit should be the best solution for native VR on iOS. Splitting screen for VR googles can be done easily with SceneKit and different angles for cameras, the same goes for implementation of walking and surround sound offered by GVRKit is not needed.

Unfortunately there are different types of lenses used in goggles, which make different lens distortion. To resolve it on your own you need to apply inverse distortion, but it can be difficult. Because of that we've decided to use GVRKit on both Android and iOS.

Now we use SceneKit and OpenGL for iOS. SceneKit is responsible for placing all elements in three-dimensional space and OpenGL is needed to render everything to make it work with GVRKit.

GVRKit

Google, as an owner of Cardboard idea, shares framework which works pretty good with both Android and iOS. It simplifies many common VR development tasks, including: lens distortion correction, heads tracking, 3D calibration, side-by-side rendering and even spatial audio. It's called GVRKit and you can find examples for Android and iOS. It required OpenGL renderer to work properly.

Cardboard button

An interesting thing is related with the Google Cardboard's button. We're going to use it to move inside VR environment and unfortunately the first version of Google Cardboard doesn't work with iPhones and even not all Android phones. It makes usage of a little magnet on the side that you can just press down. This made some developers to add virtual buttons which start walking when you look at them (i.e. at your feet). The second generation of Google Cardboard has a new input mechanism – a button that touches the corner of the screen. It's pretty easy to be detected in code and normally nobody is touching the screen in VR mode, so it fits perfectly. In addition it works on all devices.

Sum up

VR is not the easiest thing in the world, but definitely it's possible on both Android and iOS. It seems that the whole project is going to be a really interesting lesson and we're going to learn a whole bunch of cool stuff.