(Ellie is a persona created to protect the identity of the real individual I worked with)
I worked with "Ellie" from May 2024-June 2025 as her full-time, in-school aide. During that time, she completed over 20 courses ranging from Calculus II and Neuroscience to Liberation Theologies and Art History. Not to mention Poetry and Painting class and her Independent Study course. Throughout her senior year, she used her eye gaze to complete Calculus assessments, write essays and poems, to create posters and presentations, and to contribute to classroom conversations.
My job was to notice, to listen, and to collaborate with her to translate her thoughts and ideas so that the world could hear them too. We primarily used left/right selections with hand signals to communicate. Multiple times a week I asked her to assess my efficacy as her aide with the following questions on a scale of 1-5: "Did I interpret your ideas correctly?" "How am I doing as your interpreter?" "How well does this sentence represent what you meant to say?"
ComBoard was used to write essays, chat during free time, and answer free response prompts.
Methods
Recurring user interviews
Competitive audit of AAC (augmentative and alternative communication) devices and tools
Key insights
How might we…
design an AAC app that minimizes eye strain, integrates AI for predictive text and smart gaze tracking, and adapts to the unique needs of the individual so that they can communicate efficiently and independently without obscuring their face?
Quantitative Goal:
Double instances of spelling to communicate from 2x/month (spring 2024) to 4x/month (spring 2025)
Results:
980% increase in spelling instances using ComBoard 5x/week by spring 2025 on aide's phone and student's school tablet compared to 2x/month spelling instances when relying on heavy, bulky, plexiglass board.
Qualitative Goal:
User preference rating at least 20% higher than other methods.
Results:
Method preferences were scored by the user Tobii (1/5), Plexiglass board (3/5), ComBoard (5/5)
Based on observational data. Mid-summer 2024, the student began consistently choosing to use the digital ComBoard over the plexiglass. Once school started, this became the exclusive choice for spelling tasks in school, but it is my understanding that the plexiglass solution was still used at home.
Draft 1 was a communication board on google slides. Ellie would use eye gaze to select a category and her aide would click it leading to a new page with more options. This is a common format for communication boards.
This design is limited and not scalable as I would have to manually add each category and word option. The keyboard design puts more power back in the hands of the user.
After learning that Ellie struglles with spelling, we decided that small icons on each key of the board would help.
I sketched up some ideas on paper, illustrated them directly in Figma, and began to test layouts with Ellie to find the right balance between the letter, the icon, and white space on each key.
Atomic design allowed me to apply changes in real time in collaboration with Ellie - maintaining consistency at each iteration.
Ellie indicated the following preferences for the key layout:
1. Alphabet-based organization to help her find the letters where she would expect them to be.
2. Diverse clusters to limit misinterpretation by an eye tracker, predictive text, or a communication partner.
3. Include all english sounds as keys, like "TH," to speed up spelling.
With Ellie's go-ahead, I got to work making the text field interactive enough to test the usability of the design.
She rated the transition from main board to "selected card" view a 5/5, verifying that the reduction in clutter decreased her strain significantly.
Every button, tile, and card layout, along with their interaction states, used in both the mobile and desktop designs. This sticker sheet makes modifying and personalizing the ComBoard hassle-free, allowing me to continuously provide the student with a tool that best suits her needs as she is able to communicate them to me over time.
Every button, tile, and card layout, along with their interaction states, used in both the mobile and desktop designs. This sticker sheet makes modifying and personalizing the ComBoard hassle-free, allowing me to continuously provide the student with a tool that best suits her needs as she is able to communicate them to me over time.