What Happens in the Lab, Stays in the Lab User Research Imagine 2014 Session
This session is titled What Happens in the Lab, Stays in the Lab: 10 Secrets to User Research, presented by 2 user research leads from eBay Enterprise: Elizabeth Zietlow and Palak Desai. eBay Enterprise has two eye-tracking labs and one mobile/tablet testing lab for user research. Will discuss why research matters and the right research methods and when to apply them, as well as how to get the biggest bang for your research dollars.
#1 - OTB - Pick your Objectives, Timing and Budget. What's your objective - who are you studying, what's the purpose, what's it going to inform? What's your timeline? Where should you insert your user research? Finally, budget - understand what your budget is, or even figuring out how much you research you can do without a budget. There's an approach for all teams, you just need to start with the OTB questions.
#2 - You can't please everyone. Focus on your defined objective. Competitors can be used for inspiration, but don't use them for validation. Use user experiments for validation. Look at business needs, consumer needs and technical needs.
#3 - Technique matters. Apply the right type of interview technique. Three techniques - reflect back, think aloud and observe. Think aloud technique is great for evaluating a discrete piece of an interface and you want to understand how users process that interface, but isn't a great blanket technique for everything. Observe technique is great for evaluating an overall design. Create real tasks for your user group to complete so you can get out of the way. Reflect back is an interesting technique - you interview users about a recent experience and ask them to reflect back on it.
Using these techniques allowed eBay Enterprise to recommend 2 checkout page changes that increased the page's conversion rate by 34%.
#4 Beware of smoke & mirrors. Online testing, mouse tracking and heat mapping vendors push their products as a cheaper, better, faster approach to usability testing vs in-lab research approach. However, these methods help with specific questions, but can't replace in-lab research. Mouse tracking does not always work - in some cases, there is a 10% or less correlation between where someone's mouse is and where their eye is. Heatmapping is good for answering a single question - i.e., where is someone looking.
As an example, a video was shown of an eye-tracking study of a user shopping online, and the vast majority of the time, the user's mouse was not in the same location as the user's eyes were focusing on.
#5 Keep it real. Remember that we are too close to the industry, so you need to get out and talk to 'real' people. For instance, Google interview users in NYC and asked them if they knew what a browser was, and only 8% of the general public had a correct idea of what a browser was. Also make sure the users involved in your research aren't too far away from the industry - i.e., don't ask vegans for feedback on Omaha Steaks website. And don't find people who are too biased - i.e., existing customers or fans.
#6 Cheat a little. Cheat on complex audience, interface or insight to keep testing simple.
#7 Seeing is believing. Bring all stakeholders into the lab - designers, developers, managers - so that they can see real user reactions to their work. Seeing is believing - watching real users can be better than trying to guess based on analytics data.
#8 Sometimes people can't explain themselves - for instance, an eye-tracking test showed that when men and women were showed a baseball player and asked who it was, women looked at the face. However, men looked at the face and the crotch. They wouldn't have told an interviewer this - what people say they do and what they actually do are two different things - so eye-tracking and using the right observe techniques is important.
#9 Make the right connections - use a multidisciplinary approach to get viable, deep results. Bring user research, strategy and analytics together. Use analytics to pose questions that user research can then help answer - for instance, if analytics shows users aren't using layered navigation, use user research to find out why not. Then go back to analytics to determine the value of solving that problem.
#10 It's not all or nothing - continual, incremental research activities are valuable.
#11 Visit eBay Enterprise at booth 93 to see a live demonstration of user research and usability studies.
All in all it was amazing to hear all the different industries eBay Enterprise has worked with for user research - they've helped everything from swimsuit sites, a major toy website, clothing brands and even a hosting company's server configurator.