HC Lai, User Experience Researcher, Rushani Wirasinghe, User Experience Researcher
Digital advertising is the financial backbone of the Internet industry, enabling many companies to monetize the traffic to their digital products and provide customers with free access to this incredible content. At Yahoo, our user experience (UX) researchers not only work closely with the teams designing consumer products, but also with those managing our advertising platforms. We have identified best practices for conducting effective user research in the context of advertising products, because it’s important to recognize the unique nature of the participants.
Yahoo Gemini Advertiser Council
The B2B UX research team has shifted from one-time interactions to longer term relationships with users of advertising and publishing tools, and actively works to build these relationships. Partnering with the product marketing team, the B2B UX research team developed the Yahoo Gemini Advertiser Council, which engages Yahoo Gemini advertisers annually with the goal of enabling users to be part of the product development process. Yahoo Gemini customers participate in diverse activities such as research studies throughout a product lifecycle, quarterly communications, and biannual in-person meetings with our product team. These activities not only positively impact advertisers’ perception of Yahoo Gemini, but they facilitate deep collaboration between our users, product team, and sales team. Based on direct feedback, Yahoo Gemini Advertiser Council members feel more confident that Yahoo can meet their advertising needs.
Flurry’s Corporate Sponsorship
Another example of a user panel is Flurry’s corporate sponsorship of Hacker Dojo [1]. Hacker Dojo is a community of app developers and startups located in Silicon Valley. We originally reached out to Hacker Dojo to conduct a one-time usability study on their premises. When we learned that we could expand the relationship from a one-time engagement to a longer-term relationship in the form of a corporate sponsorship, we did not hesitate. The sponsorship has provided Flurry with a user panel that they use to collect user feedback with minimal effort. It has also been a good opportunity to promote Flurry within the local community.
Building a user panel not only helps minimize the lead time to recruit third party users but it also allows researchers to follow a community of users and observe how their usage of a product evolves over time.
The Yahoo UX Research and Accessibility (UXRA) team is heading to the ACM SIGCHI Conference on Human Factors in Computing Systems (otherwise known as CHI 2017) in Denver!
This year we have a series of great talks from our researchers and interns about the research we have conducted across consumer and advertising products. If you’ll be at CHI, you can find the room numbers for talks listed below. We’re also hiring, so stop by our booth at the breaks. If you can’t make it to Denver, links to download our papers are available below.
CHI is an example of how research in industry builds off of the best academic research available and contributes back to the research community with our own findings and systems that we create. Each of our papers below cites dozens of studies from other corporate labs or universities, and we hope contributes to future scholarship in these venues. We are an applied research group, and the findings from this research have a direct impact on our products and research culture within the company, as well as helps to put our users first in creating new product experiences at Yahoo for our billion+ users.
By Andrew Schulte, Chief of Staff to the CEO, Yahoo
A few weeks ago, we received nearly two dozen handwritten letters from 7th grade students at Chaboya Middle School in San Jose, all addressed to Marissa Mayer. These incredibly thoughtful and inquisitive letters penned questions about leadership, STEM, being a female CEO, the future of tech and general advice on working for a company like Yahoo. These letters touched and inspired Marissa and the team here, so what better way to answer their questions than a field trip to Yahoo headquarters!
We kicked off the day with an assembly that included an engaging Q&A discussion with Marissa addressing all the students’ top questions. Afterwards, we took them on an all access behind-the-scenes tour of what it’s like to work in Silicon Valley, including what it takes to launch some of our latest mobile products.
The first stop on their tour was our User Experience (UX) Lab where students had a chance to learn first hand about the importance of user feedback in consumer product development. Students met our UX and product teams, and learned how real world testing is critical to innovating and improving on products.
Next up, was a tour of Yahoo Studios, where students got the inside scoop on production, and how we put together a lot of our original video content. A few students even showed great promise as future on-air talent!
Then it was off to the Accessibility Lab, where they learned about the importance of building products that serve all our users, regardless of their physical capabilities. The students had an opportunity for hands-on problem solving to help individuals with hearing or visual impairment, or limited mobility use of our products.
We finished off the tour with a stop by our Publisher Products team, which works on some of Yahoo’s most popular products, like Yahoo News, Yahoo Sports and Yahoo Finance. Here, they got an inside look at a day in the life of an engineer – from coding the personalized homepage stream, to the data and analytics team that helps us make data-driven decisions, to the release engineers that manage quality control for final publishing.
We had just as much fun as the students, and we’d like to send our deepest thanks to Chaboya Middle School, Mr. Joe Ennes, and the 7th grade class for bringing their best and brightest energy to Yahoo. We had an amazing time hosting you, and have tremendous optimism for the great things you’ll achieve in the future!
At Yahoo we move quickly. Our product teams are often shipping new features every week or two and designers are constantly exploring new solutions. While User Research methods have traditionally been fairly slow to execute, especially in the academic traditions of anthropology and human-computer interaction, we have a need to move more quickly and fit our work into the weekly sprints of our product teams.
For Yahoo Mail, we tried an experiment last year where we conducted weekly user research every Friday. This gave teams Monday through Thursday to work on their new designs and implementations and Friday for us to conduct research and for them to observe and plan for their next week from our research labs. Concepts could be tested in a design state iteratively from week to week, and then tested again in a working build in a subsequent week once it was implemented. This allows us to rapidly test new designs, iterate each Friday until we get something that both provides value and is usable by our participants, and test final implementations once they are ready. We augment this with larger scale surveys and field studies as necessary for bigger features and strategic directions.
Along with testing prototypes, these sessions also include in-depth interviews on particular topics of interest, including walkthroughs of current behavior and understanding of recent experiences. These questions help us to understand particular domains over several weeks and dozens of participants and help to set strategy when combined with other qualitative data from surveys and usage logs.
These weekly studies have transformed the design and product process to truly be user focused. We test a wide variety of concepts and implementations each week. Product Managers and Designers don’t have to worry about setting up a special study for their topic, getting users recruited, and waiting several weeks for studies to be executed. They can rely on having spots in the weekly study for last-minute additions of new prototypes or qualitatively testing topics that have appeared in logs or online user feedback.
All of this has helped bring the Yahoo Mobile Mail app to its highest ratings ever, hitting and then staying at 4.5 stars on both iOS and Android for most of the year, while shipping a wide variety of new features for our users.
As a UX researcher working on advertising products, a decent portion of my day is spent observing how advertisers use advertising solutions. During various studies I have conducted, it came to my attention that their frustration often rooted in time and efforts required to complete a task. Additionally, we heard search advertisers’ needs in achieving even better results with their search campaigns on Yahoo Gemini while spending less time and effort managing multiple platforms. We came to realize that “time is money” to our advertisers.
Informed by these insights, Yahoo Gemini recently introduced an account sync feature that addresses advertisers’ needs. Account sync is a powerful tool to help search advertisers manage large-scale search campaigns across multiple search engines. The ability to sync accounts can make it simple for advertisers to import their entire eligible portfolio of campaigns from other platforms into Yahoo Gemini, with just a few clicks. By seeking a deep understanding of users’ values and beliefs, UX Research helped to inform product opportunities and we hope to continue improve the Yahoo Gemini experience.
