Sr. UX Designer, Microsoft, 2012-2013
I created the Speech Interaction Framework as part of the team at Microsoft dedicated to speech interactions.
I am a subject matter expert on speech design and had been working with teams across Microsoft for years to apply speech best practices and align experiences. The Speech Interaction Framework was a codification of that knowledge so everyone within the company working on speech experiences could speak the same language and share the same concepts. It was a successful design project with lasting effects today.
The Problem
Microsoft is a large company and, at the time we created the Speech Interaction Framework, there were multiple teams across Windows, Windows Phone, Xbox, and incubation projects interested in natural user interfaces (NUI) and speech in particular. These teams would assign designers to create the speech experience for their particular screen and use cases.
The speech experts within the company, including me, were not on these project teams, but rather we worked together with the program managers and engineers working on the speech recognition engine. To ensure a cohesive speech experience across all Microsoft products, we would partner with the various product teams to share best practices and keep them up to date on what other Microsoft divisions were doing.
We realized that it would be helpful to have a consistent framework which encapsulated our best practices, as well as ideas of where the technology was going, so everyone at Microsoft working on speech would have similar concepts and use similar terminology. The goal was to arrive at similar experiences as well.
What I Did
I had worked on speech experiences with Xbox for three years. I joined with my counterpart from Windows Phone, who had worked on various speech experiences including the burgeoning idea of a virtual assistant known as Cortana. We also included other industry design veterans and our studio producer to create the framework.
Evolution of Speech Best Practices
We moved from writing about speech best practices to more graphical representations. The final speech interaction guide simplified our expertise into a simple graphic which could be explained in just a few minutes. Our team ensured that the program managers, speech technologists, and executive leadership all understood the Framework and solicited their feedback so they would be bought into the sharing and use of it.
It also served as a jumping off point for the actual design process of creating speech interactions for various screens.
Using the Speech Interaction Framework
I evangelized the Framework across the company, including presenting it to Windows Phone, Xbox, Windows, and HoloLens.
I also modified the Framework into a working table to design the speech experience for those various products.
I was even able to use The Framework as the anchor for a Build 2013 presentation on speech design as part of that year’s Bing platform announcement.