Design Lead for AT&T Mobility, UI Designer, Tellme 2005-2009

I was the dedicated designer for AT&T Mobility Directory Assistance (411), which received 1 million calls a day. We took our billionth Directory Assistance call during my tenure, celebrated our 100th release, and brought novel functionality like location-based services to improve phone searches. Directory Assistance accounted for over one-third of Tellme’s revenue and Tellme handled over 60% of all 411 calls in the US.

The goal of AT&T Directory Assistance was to ensure that callers, in their quick 30-second interaction, received the requested listing. This was done either entirely through the voice system (automated call) or by gathering the listing information (city/state and listing request) and passing that information to the operator. Regardless of whether the listing was found by the system or by the operator, the user would return to the automated system for call completion.

I worked on Directory Assistance when text messaging was still novel, Blackberry and Palm ruled smartphones, Google maps was in its infancy, and people still called 411.

What I Did

I worked as the sole designer on a small interdisciplinary team with an account manager, project manager, one or two developers, and one or two QA testers. I was an integral part of the account team, meeting frequently with AT&T, our client, to ensure they were happy and heard.

We worked in 2-week sprints called “application update” cycles, with a release at the end of the sprint. It followed much of the principles of Agile long before agile methodologies became fashionable.

Tellme’s goal was always to balance the benefits of automation with the needs of the user. We prided ourselves on the great user experience we delivered. Our other differentiator was our network, allowing frequent releases rather than infrequent updates of on-premises software.

User Research

I conducted various forms of user research. When doing a usability study, I filled all roles, from creating the initial research plan through facilitating the study and synthesizing the results. The system was very well instrumented and I crunched logtags to determine areas for improvement, projections of automation gain and other caller experience impacts of proposed features, and places where bugs or other issues were impacting the user experience.

I also conducted call listening, where I would listen to the audio of a caller and the system to identify kinks and gaps. One interesting finding was that some users misinterpreted the initial prompt. The system said, “411 Info. Say a city and state. Or say Other Services.” Some users interpreted this as a binary, where they said, “City and state” (which contrasted with the other option, “Other Services”).

The system had a clever prompt to gently redirect the user. After a user said, “City and state,” the system would say, “What city and state?” rather than sending the user to the operator. Most interesting, when we launched into a new market, we saw a rapid decline in this behavior. It seemed this was a first-time user mistake. The red line above is the percentage of “city and state” utterances in an existing market. The blue lines represent 2 new markets. The first month saw a rapid decline of “city and state” utterances and the lines continued to trend downward over subsequent months, approaching the percentage of the existing market.

Street Disambiguation

In Directory Assistance, the goal is to give the caller the phone number and other details of the best listing for their request. When callers would ask for a common business, like Home Depot or Starbucks, they were sent to the operator because the system didn’t have a way to disambiguate based on street location.

I first worked on a feature to read out two to three streets to the user.

The feature had been proposed by a previous designer, but was cancelled when the quality assurance person from AT&T found a few cases where the data was incorrect. I listened to his concerns, worked with our data team to ensure that the issues he found had been mitigated, and provided him with additional test cases of further permutations of his concerns to prove that the feature would not expose any issues to callers. I also made sure to test cases in his home area (Redmond and Puget Sound) rather than the Bay Area to ensure other test cases he tried would work. He tested the feature, approved it, and we launched street name disambiguation, improving automation while maintaining a great user experience.

Location-Based Services for Disambiguation

Street name disambiguation was good, but location-based technology was available that let us determine the caller’s location and offer the nearest locations first. This was new technology used cell-tower triangulation because very few phones had GPS chips in them at the time. The majority of people calling 411 had “feature phones” in 2008.

Since location-based services (LBS) were new, we needed to know what the average person’s reaction would be to being located and having that location used. We conducted in-the-wild usability testing in San Francisco. I instructed the product team, including account managers and engineers, on usability best practices and everyone on the team tested a few users.

We went to Union Square and Dolores Park. We used a Wizard-of-Oz (WOZ) system, where a team member (the “wizard”) pretended to be the system and looked up directory assistance listings. We told the wizard ahead of time our general location (e.g., Dolores Park), which was sufficient accuracy for the test purposes.

I found that minimal prompting (“searching in and around your location…”) made the LBS feature barely noticeable, yet users were pleasantly surprised that the results were organized by distance from their current location. When told about the feature explicitly, users said they were open to giving up some privacy for obvious benefit. LBS was an obvious benefit.

Quarterly Summits

I presented the application performance analysis and pitched recommended improvements during quarterly in-person summits.

The images above are from the summit after we launched LBS disambiguation. These meetings were critical for maintaining rapport, keeping the client engaged and happy, and meeting our automation targets.

What Shipped

We shipped many subtle and some large improvements to Directory Assistance. My major accomplishments include:

• Adding new call centers. Cingular and BellSouth were acquired by AT&T, which added two new call centers. These operator banks used different versions of the operator workstations, requiring us to ensure our data handoff of the locality and listing information worked across each software configuration. These technical details were part of my pervue as the owner of the caller experience from end-to-end.
• Modularization of the call completion options. This project, which create a variable structure for piecing together the various options available (e.g., call, text, get address), had been put off as too complicated and risky for years. I designed, received internal and external buy-in, created the specifications, and launched this feature to the delight of the Tellme and AT&T teams.
• Automating street disambiguation. This was a major addition to the call flow and included multiple iterations, first through street name disambiguation and then through location-based services.