Chefing 1.0: Mixed Reality Cooking
Removing the hassles in the kitchen with hands-free cooking
Timeline: Mar-Jun 2020
Sponsor: Magic Leap
Teammates: 3 developers
My role: UX research, UX/UI Design, Prototyping
video is 1:18 long
Project Overview
Context
Chefing started a a class project for the AR/VR Capstone, a computer science program at UW. I worked as the only designer on the team with 3 developers. Chefing has become an incubator project at the UW Reality Lab in summer 2020 and the research will be published to academic conferences in 2021.
Problem
Cooking can be a lot of pain for those who are new to it because there are so many elements you need to balance at the same time. Smart phones has replaced makes finding recipes easier but it also creates challenges for users as an extra element to work with in the kitchen.
Outcome
Chefing is a mixed reality cooking app that helps you to make hands-free cooking possible by displaying a holographic recipe interface right in your kitchen. No more grease on the phone screen because you only need to touch the air to interact.
How might we create a mixed reality experience to provide hands-free cooking experience and help people learn cooking?
Features
Mixed reality interface: no more phones in the kitchen
No phones, laptops, tablets, cook books, or even moms needed in your kitchen ever again because Chefing is here! Chefing provides a streamlined and enjoyable cooking experience by removing the hassle of multitasking in the kitchen when you follow a recipe. Everything is right in front of your eyes and you don't need to physically touch anything.
3D GUI
Don't worry, we don't overwhelm you like a microwave interface or disappoint you by putting a flat screen in front of you. Chefing's interfaces are designed to have depth and stay simple at the same time. There's no buttons you don't need to use and hopefully there's no buttons you might want to add to the interfaces.
Simulation: practice makes perfect
Not ready to cook? New to cooking? No worries, we got you covered. The recipe-specific simulations allow you to practice steps in the recipe that might be hard or dangerous for new-comers. Practice makes perfect.
Research Insights
Although this is a computer science driven project, I was able to make a case for user research and surveyed 21 participants and had 8 in-depth interviews. Here are some key insights.
1. Phones are annoying but unavoidable in the kitchen: The invention of smart phones changed how people cook. We use to use recipe books and cards, now we can access everything on our phones, including countless recipes. However, phones do require people to take their hands off from cooking to operate. It often results in grease phones and the hassle of transitioning between tasks.
2. People needs step-by-step instructions: Because cooking is a linear process and can be complicated and difficult sometimes, it's important to break the process into small steps to let people follow with ease.
3. Visual elements facilitate fast understanding: Although we are used to reading text-based recipes, seeing images and videos help us to associate the instruction with our cooking more easily.
Define
After synthesizing my research insights, I defined:
Target audience: people who are new to cooking
Design opportunity: create a mixed reality tool to help people learn cooking in a hassle-free and fun way.
Ideation
Initial Concepts
I started designing by creating various ways this mixed reality interface can exist in the kitchen and how users can interact with them:
1. A virtual phone that follows your hand and gets activated when you flip your hand;
2. A floating stand-along interface always in your vision;
3. A wall-mounted interface that doesn't get in the way of your cooking;
4. A big floating interface with 3D animation showing each step.
hand menu-activation
hand menu
floating interface stand-alone
wall-mounted interface
floating interface + 3D animation
Refined Concept
After discussing with my teammates and showing the visualizations above to some research participants, we decided to go with a floating interface that follows users loosely. I iterated 3 versions in Unity. Click on the images to see pros and cons of each version.
Final Concept
We went with Option 2, centralizing all the information users will need to one hub, to minimize user's effort in using the app and to make it easy to find the dashboard.
Design
Initial Design
At first, I designed the interface to be big and simple. As you may see above, the final prototype for concepting has many buttons that sit right next to each other. Because the prototype was of a size of a phone, we found ourselves accidentally tapping on the wrong buttons during testing.
The important lesson: field of view
However, I forgot to consider a very important factor of MR design: field of view. Unlike virtual reality in which people have almost as much field of view as we do in real life, MR and AR glasses only give you very limited field of view.
Final design
I redesigned the interface to match the ratio of the field of view of the headset we are using and prototype in Unity relentlessly until I found the right size for the interface: big enough to make the smallest text legible to users, but small enough to fit the whole thing into the field of view.
Reflection
Mixed reality is powerful, but also full of constraints
Working with mixed reality is fun! Spatial computing blends virtual objects naturally into the real world, seeing your living environment through the lenses helps eliminate motion sickness, and it has unlimited potentials in training and professional practices. However, the limited field of view, the bulkiness of the headset, and even the tint of the lenses can easily take away the impressiveness of the experience. I learned to work with limited FOV on Magic Leap by relentlessly prototyping the interface until the size is appropriate.
It's more important to ship a complete product than a half-finished product with an amazing feature
Believe it or not, we ditched out biggest selling point after finishing MVP. We worked on representing recipes in 3D animations for 4/5 of our time because we wanted to created a truly immersive, fun, and novel way for people to learn cooking. However, our minimal viable product made us realize the graphics constraint of mixed reality: it doesn't do a good job representing 3D objects and animations. We went back to our initial user research and found out video tutorials are many people's go-to. So we ditch the animation idea last minute. Although we felt attached to the adorable animation we made, we were glad to made the decision that led to a more practical and desirable product.
