Visits Program Helps eBay Learn About Our Members
by Nino
eBay Staff Member

A critical part of designing any product is understanding the environment in which users will interact with the product. That's why automobile makers test new vehicles on driving ranges that simulate different road conditions, and baseball bat manufacturers head to the batting cages to test their new bats.

Similarly, eBay has many User Experience (UE) research programs aimed at making our websites easier for our members to use. Most methods involve real eBay members and help our User Experience researchers understand the different ways in which people use eBay. Techniques such as usability lab simulations and surveys help our UE researchers identify specific aspects about eBay's site features or navigation that do or don't work well for our members. These findings are widely distributed within the company, and help us improve the site for our members.

While simulations in a usability lab setting provide valuable insights about how people use eBay, it's every UE researcher's dream to look over the shoulders of eBay members as they're using the site. There's just one small difficulty – most eBay members access eBay from the privacy of their homes! Many eBay sellers operating from home have separate rooms dedicated to their eBay activity. Many buyers also access eBay from home, using a variety of strategies to get the best deals on eBay.

In short, eBay fits into members' lives in different ways in their own homes. For a long time, eBay didn't have first-hand knowledge about how our members used the site from home – all our information came from indirect sources such as member interviews and surveys,or from statements from members in the Voices program, posts on discussion boards, or comments and emails to Customer Support.

Then in 2002, Krisela Rivera, a Senior Usability Engineer in the User Experience Research Group, spearheaded an initiative that has evolved into the comprehensive “Visits Program.” As part of this program, teams of eBay employees visit actual eBay members in their homes and businesses to observe them interacting with eBay and to gain a better understanding of their challenges.

The Visits Program began on a relatively modest scale, with visits to members' homes in Oregon and upstate New York. Over the next few years, the Visits team extended their visits to members' homes across the length and breadth of the eBay world. Members across the country, from Gilroy, California to Buffalo, New York, graciously opened their doors to the eBay team, letting them observe how they browsed, bought, and sold on the site and sharing their opinions about what they liked or didn't like about eBay. Realizing the program's benefits in terms of understanding members' needs, eBay soon extended the program to international eBay sites. By 2004, the Visits team had visited the homes of eBay members in Germany, Italy, Taiwan, China, and India. Members visited represented a microcosm of the eBay Community – they included newbie users, occasional buyers and sellers, volume buyers, PowerSellers, people who use eBay as a hobby, as well as people who make a living from eBay.

The team followed a rigorous and non-intrusive process to ask eBay members if they wanted to be included in the study, only contact eBay members who had “opted in” to receive surveys from eBay. Members were then asked if they would be open to having a small eBay team at their home for a two-hour visit. If they agreed, a two or three person eBay team visited their home. During the visit, the team asked the member questions about their eBay experiences, through structured questionnaires as well as informal interviews.

Staff observed the member perform eBay tasks they were interested in and recorded the member's experience and ease of performing the task. In some cases, with the member's permission, the session was recorded on video, to help the team do a “show and tell” to other groups back in San Jose. At the conclusion of the visit, the eBay team would huddle in a nearby café or park to compare notes and compile a preliminary report about the visit. These reports summarized the member's needs, challenges, suggestions, workarounds to an eBay-related problem, and other insights related to the site's features and usage.

After Krisela and her team shared the aggregate findings from the first Visits with other teams within the company, the program's popularity grew quickly. Soon, members from other eBay teams, such as Product Management, Categories Marketing, User Interface Design, and Creative Design started accompanying the UE teams on subsequent visits. Everyone was eager to interact with members directly and understand first-hand their challenges using eBay. To ensure that the Visits were as non-intrusive as possible to the members whose homes and businesses were being visited, eBay instituted internal training sessions for the Visits teams. Before being part of a Visits team, an eBay employee has to undergo a two-hour training session, in which experienced team members share do's and don'ts from previous Visits.

As they pooled their findings from different visits, the teams were struck by the consistent themes that emerged from members with different profiles and interests across different locations. Rivera shared some of the insights.

“No matter where we went, a strong finding that emerged was – people love eBay. So many people shared their stories of how eBay had enabled them to succeed in new ways, and how they were able to supplement their incomes from their eBay businesses. Or even about how eBay had helped them pursue their collecting hobbies.”

However, not all the comments were rosy, Rivera admitted. “Unfortunately, what we've learned is that, for many people, eBay is not an easy site to use.” She went on to cite some of the usability problems that the team consistently observed: sellers having difficulty using the category selector, buyers having trouble finding the right item (whether through searching or browsing) and sellers being unsure about what to do next in the Sell Your Item (SYI) process.

The team was surprised that many members found some eBay concepts difficult to understand – for example, the idea of bidding and reserve price. Many were daunted by eBay's terminology and overwhelmed by the sheer volume of information on the site. Newbies who wanted to get started buying or selling on the site were frustrated that there was “too much to learn” before they could get started.

On the positive side, the teams found that members had adapted to fit eBay into their lives in many interesting and innovative ways. Rivera laughingly recalled how a seller dealing in shoes had streamlined his supply-chain. “One of the rooms in his house is his warehouse, with the window serving as the loading dock. When he has boxes to ship, he simply stacks them on the window sill, and the delivery service just picks them up from there. Very efficient!”

Many of the Visits findings were revelations to eBay. Rivera recalls that several people at eBay were surprised by learning about members' ways of using eBay at home. “I didn't know this!” was a common reaction that Rivera encountered from other eBay employees.

The good news is that the Visits findings have had a positive influence on eBay Product design. Rivera recounted how one big success was the Comparison Shopping feature on the search and listings page. “We saw many people open multiple browser windows to compare items from different sellers. This set us thinking – wouldn't it be a great idea for members to select and compare items that they were interested in, in a side-by-side view? Everyone at eBay was excited, and a few months later, we implemented the project.”

Other improvements that were influenced by the Visits program included streamlining the Sell Your Item (SYI) process. Many members had expressed frustration that there were too many steps in the SYI process and that there was inadequate Help content to guide them through the process. Now eBay is using these findings to improve the SYI process.

The Visits program was so well-received internally at eBay, that it is now a regular exercise every quarter. It is also being extended to other aspects of eBay, such as eBay Motors and Seller Tools.

On their travels, the Visits teams were exposed to a variety of cultural differences across different geographies. They experienced narrow lanes and small houses in the old quarter of Milan, Italy. They learned to take off their shoes before entering someone's home in India and Taiwan. They learned that a restaurant chef in China could take offense if you stuck your chopsticks in a bowl of rice. But everywhere they went, the differences also reinforced the similarities within eBay's worldwide, diverse Community. As Rivera said, “no matter where you go, people may have challenges...but they overwhelmingly love eBay!”