Who does not know them, the two cult ibexes from Graubünden? The children's books about the adventures of Gian and Giachen are bestsellers and are in the living rooms of many Swiss families. In the Progressive Web App (PWA) "Kinderwelt Graubünden" developed by us, the book pages are now brought to life and digitally animated with the help of artificial intelligence.
The lynchpin of the digital world is the animated mountain panorama: it enables the discovery of different places. Through the control by means of device movement as well as the built-in dolly zoom and parallax effect, the experience for children and adults is maximised. The panorama leads to the individual favourite places of Gian and Giachen: These include numerous sprite-based, interactive animations and audio files. The coordination of audio and animation is based on a specially created dialogue framework based on Typescript. So nothing stands in the way of an expansion for further locations or a spring version of the book!
«The progressive web app «Kinderwelt Graubünden», which we programmed, won bronze in the technology category of the Best of Swiss Web Award 2020!»
At the heart of the Grisons Children's World is the Machine Learning based scan function, which enables a seamless transition from book to platform. Using Deep Learning, scanned or photographed children's book pages (or excerpts thereof) are recognised and users are then guided to the corresponding animated book scene.
The decision for a Progressive Web App and against a native, mobile app was a conscious one: Users can use the highly compressed app offline and place it on their personal home screen. Despite the challenge of lacking support for PWA functionality on iOS, we managed to offer users an equivalent app experience thanks to clever user guidance.
The technical challenges we have faced in classifying images with TensorFlow are described in detail in this blog entry .
The creation, project and campaign lead in this project was with WIRZ Communication AG.
.jpeg?v=63810243574)
