As the global pandemic continues, everyone is affected by social distancing. To close the gap between people, Hands digitalizes the famous Shifumi (or rock-paper-scissors) and allows anyone around the world to share a timeless moment with their digital "neighbor". This unique digital experience is based on artificial intelligence and gesture recognition from the user's webcam.
At a time when brands are seeking to conquer new audiences and are seizing digital opportunities, Hands offers a new way of understanding gaming and puts the most advanced technologies at the service of creativity. Faithful to its mission of accompanying brands in their digital interactions, Hands has produced a project with AirShifumi that is as surprising as it is exciting.
Closing the distance through gamification
Physical distance and digital proximity
It’s been a tough year that has brought a lot of profound changes to the way we interact as a society. We’ve been separated for more than a year now with no end in sight. Over time, this physical isolation has created new online uses to try and feel close again.
We organized digital alternatives for our social activities and found new ways to create group dynamics - from Houseparty after working hours to digital dance clubs. Digital has been a lifeline to join people together again and mend lost proximity.
AirShifumi is the answer to this void. As Zoom meetings have replaced drinks with friends, Hands wanted to offer a space to feel close again and provided a reason to get lost together in a dreamlike universe and to explore new ways to interact - through a screen, but closer than ever.
Transcending our relation to gaming
Brands have been playing with the gaming universe more than ever lately - some as a way to explore new codes and maybe attract new communities, others as a bold statement to push the limits of brand perception, challenging their brand territories themselves. On Twitch, Discord, Animal Crossing and proprietary platforms, luxury brands are having a field day with interactive experiences and reinterpretations of classical games to an always broader audience that knows no limit of age or gender. With AirShifumi, Hands created an immersive universe around the gestures that are most likely to link us together. Far from the go-to 8bit aesthetics of others, Hands have taken the principle of gaming to its roots: how do you immerse players in a different reality, for minutes or hours? What stories are they about to live? As an answer, AirShifumi offers a playful and meditative experience.
With AirShifumi, Hands created an immersive universe around the gestures that are most likely to link us together.
Travelling through space in a world without frontiers
Out of this world visual identity
AirShifumi, a game with a cosmic and dreamlike feel, allows us to interact together but at a distance with people from all over the world. Air is what helps us to materialize space, so this revisited "rock-paper-scissors" takes place from earth to cosmos through the Milky Way. An aerial lexical field for an experience full of emotion and lightness.
Through soft colours punctuated by saturated touches, the user is plunged into the heart of a surreal and futuristic universe.
For the illustrations, Hands partnered with Myriam Wares
Surreal colors & grounded effects
Hands wanted to have a surreal and fantastic play of scale between the elements and chose a soft and poetic colour palette. Textures and grain were added to bridge the gap between the tangible and intangible worlds. Finally, sounds and music was integrated to accentuate the emotions. Hands partnered with Press Play On Tape for the sound design.
Light as air experience
Hands chose to use the following typefaces: MadeMirage and Geomanist: thin serif typefaces to highlight a world where gravity is no longer a constraint. The typography was animated by playing with the height and fade. With thin and textured UI elements and anchored illustration elements (rocks, clouds, planes) the overall feeling is one of lightness and density brought together.
A glance into the future
Hands decided to have a permanent rise that indicates the passing of time and an advance into the future. The visual codes are inspired by the world of space and retrofuturism (planets, asteroids, sky, universe, etc).
Cutting-edge technology for human emotion
We’re long past the point of defiance towards machines, and finally embracing the emotion they can provide. The Rijksmuseum proved it time and time again by appropriating best-in-class technology to play with their artwork. Artificial Intelligence has the legitimacy of real interaction. Learning as it goes, AirShifumi adapted to its opponents and provided the emotion of a close-by encounter with strangers.
AirShifumi adapted to its opponents and provided the emotion of a close-by encounter with strangers.
Months of training and countless reworks have allowed the experience to provide one simple message to its players: ‘We are still together’. It was true for the development of the project: from across the world, working remotely, or together in a room.
This R&D project that aimed to test new ways of interacting with each other in times of distancing ended up being our project held most at heart, and our latest best achievement, as we put it into the world for everyone to play with.
Frontend Frameworks and Libraries:
Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
For the 3D, all the modeling, texturing, and rigging have been done in Cinema 4D
Substance Painted has also been used for specific elements such as the hands.
Modeling, texturing, and rigging done in Cinema4D
Backend Technologies:
Tracking the user’s hand position and signing was quite tricky. Hands ended up using Tensor Flow, particularly in its javascript version. All of the Machine Learning analysis and training are done directly on the client side.
TensorFlow, regardless of how fast it is, has been the real bottleneck performance of the rest of the experience, taking up almost 90% of the framerate budget when activated, so the WebGL rendering pipeline has to be flawless.
Rock-Paper-Scissors game
Server Architecture:
To allow for multiplayer games, Hands powered the experience with WebSocket.
All the WebGL rendering has been done natively to increase performance. Simplicity and lightness was the goal of the code design. A specific animator and rigged mesh shader/processor have been written from scratch to be able to tweak absolutely everything.
The shading is mainly unlit, most of the time simply plane with an atlas texture. Still, there are some custom shaders for the trickiest parts, as the backface reflection on the platform, or the hand halo.
Tools
Development
- UI: VueJS - open-source JavaScript framework used to build user interfaces and single-page web applications
- 3D rendering: WebGL - dynamic 3D programming interface specification for HTML5 pages and applications
- Protocol: Websocket - web standard for an application layer network protocol and a World Wide Web programming interface for creating full-duplex communication channels over a TCP connection for web browsers
- Artificial Intelligence (Deep Learning): Tensorflow - open-source machine learning tool developed by Google
Design
- 3D Modeling: Cinema4D - 3D design software
- Texturing: Substance Painter - 3D painting software
- Illustration: we partnered with Myriam Wares
- Sound design: we partnered with Press Play On Tape
A good experience can draw a crowd
The welcome was as good as we hoped (+60K visitors in less than 2 weeks thanks to word of mouth), and knew no borders (visits located in 150 different countries). Mostly, it taught us to keep on pushing limits: always shake further.
With an average time spent of 2.5 minutes and a bounce rate of 36%, AIRSHIFUMI is a perfect case of interaction, which has generated 10K visits to the agency's website. The experience has received numerous awards including SOTD Awwwards, and FWA and cssdesignawards named it the site of the month for January 2021.
Who we are
Hands is a creative agency that promotes know-how and heritage in the digital age. We offer both technical and creative expertise based on an innovative project approach. We integrate all technological production and audiovisual production activities necessary to shape our projects.
Discover AIRSHIFUMI 👊🤚✌️ : https://airshifumi.hands.agency/
Follow us on instagram: @hands_agency