Augmented Reality-Based Mobile App for iOS Users

Add art to your reality with Kunster - Augmented Reality-based mobile app

Ever wondered how Picasso saw the world around him? Or how would your room look if it was painted by Munch?

One of our latest R&D projects aims to show people that no matter where we are, art is all around us. It’s just a matter of perception.

We’ve developed an Augmented Reality-based mobile app that instantly shifts your surroundings based on a given painter’s style.

What was the challenge?

At Netguru our vision is to integrate our research capability into solving real-life problems. In addition to our commercial activities, we spend time creating educational and non-profit projects that can benefit the community.

By creating Kunster we wanted to bring art closer to people and engage both gen Z digital natives and older generations.

Now you don’t have to leave your house to enjoy the art and see the styles of the greatest world painters. Our app will help you to experience it without having to go to the museum, if your access is limited.

We decided to pick the most characteristic paintings and famous artists, including:

Van Gogh and his Starry Night
Leonardo da Vinci, the author of Mona Lisa
Edvard Munch, famous because of his Scream
Pablo Picasso and his The Weeping Woman
Witkacy and his Fight (Tree Felling) - in partnership with Muzeum Sztuki in Łódź

Enjoy art wherever you are

See for yourself how it works!

How did we do it?

Our iOS application is based on Machine Learning and Augmented Reality. We used ARKit, CoreML, and TensorFlow Lite to achieve our goals.

PyTorch allowed us to quickly develop a pipeline to experiment with style transfer - training the network, stylizing videos, incorporating stabilization, and providing the necessary evaluation metrics to improve the model. coremltools was the framework we used to integrate our style transfer models into the iPhone app, converting the model into the appropriate format and running video stylization on a mobile device.

How does it work?

The app uses AR to overlay a portal over the user’s physical surroundings. Step through it, and you can view the world transformed to look like it was painted by the artist you selected.. Users can record videos and share them with friends and/or social media. They can also learn something more about the painter, take a quiz about his/her work, or find museums where the artworks are being displayed.