demo.gif

Smile transition

Smile transition is an application that recognize human facial expression and trigger the transition between different contents. The transition is transformed into random 3D geometry shapes.

2017. 07

Role: Interaction and Visual Designer, Creative Technologist

Tool: Three.js, Java Script, OpenCV

 
 

The goal for this project is to designed an eye-catching interaction for the display window on the street. The design of change a 2D image into a 3D geometry shape and size through a folding movement gives the interaction a pop up effect from the flat window. We chose smile as the trigger for the transform, this facial expression can seamlessly add on a delightful feeling to the experience

 

When you pass by the interactive window on the street it will recognize your facial expression through the camera behind the window. Once the camera detected your facial expression is a smile, it will transformed into a random 3D geometry and then show you the next images.

 

The technology behind this installation is mainly computer vision and facial recognition. Using machine learning to recognize the facial expression and the environment and then to trigger the interaction.

I use Three.js to get the vertices of a plane and pull them into a 3D shape to create the visual of the interaction.

The algorithm of the graphic changing:

Screen Shot 2018-01-28 at 2.45.48 PM.png
Screen Shot 2018-01-28 at 2.46.10 PM.png
 

Mock-up on website, mouse enter serves as the facial expression

demoweb.gif