Magical Reflections by makemepulse wins Site of the Month February 2022

Congratulations to makemepulse, the winners of SOTM February for Magical Reflections, thanks for all your votes, the winner of the Pro Plan in our directory for designers, can be found at the end of the article.

Magical Reflections is born from a collaboration between Meta Creative Shop and The Alte Nationalgalerie in Berlin. We worked with them to produce a virtual gallery that would exist as an extension of a real life exhibition featuring Johann Erdmann Hummel’s work; with particular focus on one of his most iconic pieces: “The Granite Bowl in the Berlin Lustgarten”.

The build itself was every bit as interesting because it was so technically challenging. Meta’s idea was to extend the use of VR to experimental web with the experience being available through the Occulus browser.

Building an imaginary museum

The project consists of two components: a virtual museum and an immersive painting. The brief for the virtual museum was to create a unique, dreamlike space to showcase Hummel’s work.

The brief for the virtual museum was to create a unique, dreamlike space to showcase Hummel’s work”

Whatever we’re doing, we always want to push the boundaries of what can be achieved with digital production, and this project was no different. For example, to help us create a dream-like feel, we added special effects and post processes such as lens flares, Chromatic Aberration and real time reflections on the floor.

Beautiful but challenging special effects to work with as not all mobile or computer operating systems can handle them. We had to implement a dynamic quality adjustment to compensate for this. The average fps is sampled and after a while it adapts the quality by removing an effect. It then repeats the process until the frame rate is considered stable enough.

The lighting in each room was created using Houdini which enables a non destructive workflow to generate on the fly lightmap uv with good texel density scattering which is great for quick iterations. And since it’s node based, it enables other things such as the ability to clean up the scene, to add and place interactable frames, to generate the navmesh used in XR mode and to colour parts of the scene. All at the same time!

Recreating a masterpiece

One of the most exciting parts of the project was taking one of Johann Erdmann Hummel’s best known works ‘The Granite Bowl in the Berlin Lustgarten’, and turn it into an immersive real-time 3D destination; allowing users to get up close and personal with the piece and explore every detail within the work itself.

We weren’t that familiar with his work when we took the project on but the museum was kind enough to educate us on relevant art history so we had a better understanding of the historical context around this artist who was renowned for his hyper realistic paintings. All the more impressive considering photography had yet to be invented in his lifetime. The challenge for us then was to accurately reproduce Hummel’s art with realistic 3D real time renders, no easy feat!

The challenge for us then was to accurately reproduce Hummel’s art with realistic 3D real time renders”

Every single aspect of the painting has been reproduced in 3D. It involved placing volumes, hand painting all the content in Photoshop, out and out creating missing parts from scratch and modelling all characters... There are 16 in total (including the dog 🐶). Performance wise, we needed to reduce textures and draw call count so we used Simplygon to aggregate characters. It can merge selected mesh, generate a new uv set and create new textures based on sources according to the new uv set. This meant we were able to reduce character and texture count to 4 each.

Special FX

The main focus of the painting is the granite bowl’s reflection. Because it’s a flat surface on top of a pillar, solely using a cubemap resulted in a floating bowl with unrealistic reflections. We decided instead to take a hybrid approach, melding real time reflections on the bottom and cubemap for the sides and top. Using only one shader we used vertex colour to blend these two reflections according to the red level.

The real time reflection is done by reflecting the camera matrix from an arbitrary axis and rendering the scene from that point of view. To gain extra fill rate performance, the bottom of the bowl is used as a stencil mask to render only the pixels that matter.

Enhanced experience through virtual reality

Working in VR is always a bit of a challenge as it requires us to approach the project differently in terms of our ideas around navigation. We needed to consider mobile, desktop and VR iterations of the experience and the need for each version to be equally and as powerfully optimised.

This meant we had to render the scene up to four times (one for each eye + real time reflection) which was quite challenging. It is important to minimise WebGL state changes between renders and where possible to only update MVP uniform between eye renders. This is where nanogl-state really shines. This module stack GL only states and applies necessary changes between renders.

And finally, maintaining a solid 90fps in WebGL and in XR is a hard task, and the motion sickness generally comes from unstable fps. Oculus supports a feature to set at runtime the target fps. We setted it to 72.

Company Info

makemepulse is a global interactive production studio that partners with global brands striving to make authentic connections. We turn aesthetics into experiences. We bring standout immersive experiences to everyone, everywhere.

The winner of the Pro Plan is @thenes17, please send us a DM and we’ll upgrade your account.