Invasion #withGalaxy
Samsung’s holiday season partnership with Disney’s Star Wars: The Rise of Skywalker. R/GA decided to create an Augmented Reality (AR) social activation and invited users in the U.S to watch the spaceship from the Star Wars movie landing in different cities on their Samsung device.
2019.10 - 2019.12
Role: Interaction and Experience Design
Tools: Unity, Three.js, Figma, Spark AR
The Client: Samsung x Disney
I led the design for the AR experience in preparation for the activation launch together with the new Star Wars movie: The Rise of Skywalker (December 2019). I collaborated with the creative directors to refine the initial idea, explored multiple proof of concept prototypes with different AR platforms and successfully delivered the final product together with a remote development team in London.
Goal: To take advantage of the Star Wars partnership and amplify the campaign in the U.SSamsung tapped R/GA to make an impact in a culturally-relevant way, and drive post-purchase engagement by giving back to existing Galaxy owners for the holiday season.
With the amazing partnership with Star Wars, and the upcoming new movies releasing around the holiday season, R/GA came up with the initiative idea to bring the starships from a Galaxy far, far away to skies across the country.
Challenges and opportunities:
Use the Disney designed 3D assets for the AR activation
Use Star Wars as the attraction for the social activation
Only 12 week for the project
Use Samsung devices for the AR experience
High Level User Flow
After several rounds of discussion across different teams, we designed a high level user flow as a guideline to communicate within the team and share with clients to hear their feedback. We wanted to communicate the important and critical points in the experience with Samsung and Disney in order to be on the same page at the early phase of the production.
Journey Mapping
With the concept approved by both Samsung and Disney, we started to design the general journey map for the user. Meanwhile, we discovered an interesting concept of spatial computing and ran several tests in order to have the best visual effect.
We collaborated with different teams on this project, including design, marketing, strategy and development team. I created the journey map in order to demonstrate all the key moments, pain points and the critical decisions. In this way, people from different backgrounds can communicate more efficiently.
Important Features in Exploration
01. Geo Fencing
Assigning different spaceship to different cities based on the rolling out timeline.
02. 3D Model Optimization
The stunning spaceship model from Disney are too big for AR experience.
We rebuilt and washed down the polygons in Maya to make it AR friendly.
03. Spatial Computing & Sky Segmentation
Since all the spaceships are arriving in cities, in order to have a real magical AR experience Spatial Computing and sky segmentation is very important. It can match the real distance between AR content and real word objects.
In order to create the best AR experience, we decided to create 3 different proof of concept AR prototypes and present to the Samsung clients at the same time. The client picks the direction they feel excited about and then we move forward.
Challenges and opportunities from these 3 different prototypes:
WebAR prototype, which can help Samsung reach a broader audience since WebAR experience did not require additional downloads or preparations. But it was not as stable and reliable as in App experience, and it required a lot of engineering effort.
Samsung native App prototype, which was created in Unity, embedded into an existing Samsung members app. This was the most reliable and exclusive experience, but it was only accessible for Samsung users.
Snapchat prototype, which has the best visual effect since it has the sky segmentation feature. However the develop platform was not open, any customized requirements or feature will cost a fortune, it was over the budget.
WebAR Prototype
Within the very challenging timeline, I was in charge of the production of three prototypes and implementation of the WebAR prototype.
I use Three.js to load the spaceship models into the webXR framework, and enable the geo fencing feature in Javascript. I also tried to import an openCV to achieve the sky segmentation effect, however the hardware and browser was too slow to have good enough outcome. This fun exploration actually showed the huge potential of spatial computing: it is the key to make AR and XR experience really magical by mapping the data from the physical world in real-time. The prototype also showed the possibilities of WebAR: it’s convenient, it’s fast, and it has huge potential in future.
Once we got the green light from Samsung and Disney, we started to dive deeper into the user experience. We prioritize the tasks and features based on the timeline, talent resource and key requirements from different teams. The following UX wireframe presented the experience with the essential features.
Final Product Website:
