Figures: Interaction map demonstrating the flow of interaction through educative.io.
University of Washington, January 2018 through Present
Collaborators: Samantha West, Jihyun Lee, Christopher Collins
Educative is in the early phases of personalizing their courses for individual students' needs. Our group is working with Educative to identify user needs and uncover metrics that can help Educative appropriately adapt to individual differences. The primary users of Educative.io have introductory or intermediate understanding of software development and have little to moderate experience with online courses. Some variation in past experiences these will help us evaluate how users of different skill levels interact with the courses. We are in the process of recruiting participants and planning the usability study.
Sketch (by Samantha West) of the wearable homework tracking device.
University of Washington, September to December 2017
Collaborators: Samantha West
Middle school students struggle to keep track of their homework assignments. There are several reasons why this is the case: First, students often try to keep track by memory, and may forget assignments. Second, for students who use handwritten notes or journals, writing down the assignment and track of the notes can be burdensome. Third, a handwritten journal does not remind them when it's time to get started on the homework. And finally, technological solutions present the opportunity to better connect students and teachers. Design Question: How can we support elementary school children with the tracking and completion of their homework assignments using a conversational interface? In order to test our design direction, we created a hi-fidelity prototype that used the wizard of oz method to control its functionality. The wearable was created with an existing fitness tracking band, a strip of color changing, wireless, remote controlled LEDs, two buttons made of tape, and pre recorded audio feedback on a laptop that was played in response to users commands. One major design direction present in this specification of the TellMe device is screenlessness. During the course of our user research, we found that parents in many cases will restrict their childrens' use of screened technology such as smartphones, in favor of a low-tech cell phone. TellMe should be inclusive for all students. Our screenless design is non-distracting, minimal, and does not conflict with the values of parents.
Figure (by Mike Harwell): Wireframe of app screens for transcribing speech, filtering voices, and reviewing transcriptions.
Collaborators: Samantha West, Mike Harwell, Jessica Son, Sandy Tsai
Working with a group of students, I designed an auto-captioning mobile app to provide real-time captions and translation. The app uses a phone's built-in omnidirectional microphones and combines automatic speech recognition (ASR) with machine learning to determine which voices may be of interest to a person and captions only those sounds. Auto-Captions features speaker identification and filtering, so captions are labeled and only relevant speakers are included. Auto-Captions eliminates the need for a professional stenographer, and allows the user to freely capture speech-to-text directly from their smartphone. Auto-Caption can filter out ambient noise and other distracting noises, providing a clear transcript of any conversation right to the user's smartphone in real-time. Finally, Auto-Captions features conversation storage, search, and retrieval, allowing users to look into their past conversations for important information.